Internal limits with allocation percentage

The allocation_percentage field allocates a specific percentage of a main quota to different internal quotas, ensuring flexible and precise control over quota usage among different user groups or environments.

Allocation scenario

The following scenario demonstrates a primary quota (MyQuota) limiting API consumption to 100,000 monthly requests. LLM Gateway allocates 20% of this quota to staging and 80% to production:

• Primary quota: Limits API usage to a maximum of 100,000 requests per month.
• Staging quota: Allocates 20% of the primary quota, up to 20,000 requests.
• Production quota: Allocates 80% of the primary quota, up to 80,000 requests.

Allocation YAML configuration

The following configuration example defines allocation percentages for staging and production environments:

/etc/lunar-proxy/quotas/{fileName}.yaml

quotas:
  - id: MyQuota
    filter:
      url: httpbin.org/get
    strategy:
      fixed_window:
        static:
          max: 100
          interval: 1
          interval_unit: month
          monthly_renewal:
            day: 1
            hour: 11
            minute: 0
            second: 0
            timezone: UTC

internal_limits:
  - id: LimiterStaging
    parent_id: MyQuota
    filter:
      header:
        key: X-Group
        value: staging
    strategy:
      allocation_percentage: 20

  - id: LimiterProd
    parent_id: MyQuota
    filter:
      header:
        key: X-Group
        value: production
    strategy:
      allocation_percentage: 80

note

• Combination Restriction: Configurations cannot combine fixed_window and allocation_percentage strategies within the same internal_limits block.
• Zero Percent Restriction: Assigning an allocation of 0% returns a validation error.
• Total Allocation Requirement: The total percentage allocations must equal exactly 100% for the quota configuration to be valid.