Gateway policies hub
The Gateway policies hub is your central location for managing API traffic and monitoring performance within the LLM Gateway ecosystem. This hub lists the available gateway policies under two main categories: traffic control, and observability and monitoring. These gateway policies help you efficiently manage your API traffic, enforce quotas, prioritize requests, and track key performance metrics.
Traffic controls
The traffic controls category provides comprehensive policies designed to regulate and manage API traffic. These policies ensure optimal API performance, helping you prevent overconsumption, manage high-traffic environments, and enforce access control policies.
-
Client-side limiting gateway policy
This gateway policy enforces quotas within the client-side application, regulating API consumption by limiting the number of requests that can be sent. It is ideal for controlling overuse and protecting API provider resources.
-
The priority queue gateway policy manages API requests based on their priority. It ensures high-priority traffic is processed first, with the ability to queue lower-priority requests until their time-to-live (TTL) expires.
-
Endpoint access control gateway policy
This gateway policy enforces access rules based on domain names and custom headers, enabling you to permit or block traffic from specific domains or IP addresses.
Observability and monitoring
The observability and monitoring category lets you define and track custom metrics for API operations. This section is focused on providing detailed insights into API performance, usage, and rate limits, with built-in support for exporting metrics to Prometheus for real-time analysis.
Custom metrics collection gateway policy
This gateway policy lets you define custom metrics to track specific API behaviors, such as request size or response times. Metrics can be customized using labels and exported to Prometheus for visualization in tools like Grafana and Datadog.
