Skip to main content
Feedback

Configuring your API quotas

LLM Gateway quotas provide a centralized method for managing and configuring API quotas. Quotas encompass both internal and provider-defined limits, establishing precise controls over API consumption.

Quota management is defined in a quota.yaml file. This file is shared across all Gateway policies within an LLM Gateway, providing a scalable method to allocate and monitor API usage without shared state issues.

Quota policies configure limits based on specific provider limits and internal limits that reflect the API consumption profile, such as monthly, daily, or other intervals. The system updates quotas dynamically as consumption occurs, maintaining real-time management across Gateway policies.

The following topics describe quota configuration procedure:

Quota file structure

The quota section defines the main constraints on API usage by specifying rate-limiting strategies. This primary quota acts as the overarching rule for API requests, applying to all requests that match specified filters, for example, URLs or headers. The quota's filters apply conditionally based on endpoint patterns or request headers, making it adaptable to different usage scenarios and customer types.

The internal_limits section sets specific constraints within the main quota by defining nested internal limits that inherit from the main parent quota. Each internal limit can have its own filters and strategies, enabling precise control over distinct user groups or endpoints.

Benefits of LLM Gateway quota management

Key benefits of quota management include:

  • Real-time API Usage Monitoring: Quotas update dynamically as clients consume bandwidth, providing accurate real-time usage statistics.
  • Granular Control: Set quotas based on URLs, headers, and user groups. Nested quotas provide more detailed control over specific API resources.
  • Scalability: Manage and track quotas across multiple API Gateway policies and user environments.

Quota example

The following configuration example defines a quota rule in a quota.yaml file:

/etc/lunar-proxy/quotas/{fileName}.yaml
quotas:
  - id: MyQuota
    filter:
      url: api.website.com/*
    strategy:
      fixed_window:
        static:
          max: 100
          interval: 1
          interval_unit: minute

The following configuration applies a quota rule to a specific IP address rather than a URL pattern:

/etc/lunar-proxy/quotas/{fileName}.yaml
quotas:
  - id: MyQuota
    filter:
      url: 15.459.124.21/*
    strategy:
      fixed_window:
        static:
          max: 100
          interval: 1
          interval_unit: minute
On this Page