Core Settings

The LLM Gateway relies on the key settings to strengthen its security, flexibility, and performance. While not all settings are mandatory, we strongly recommend configuring them to ensure secure, efficient, and reliable management of your API traffic.

The following section outlines each configuration, its purpose, and its relevance to your system setup.

Security and communication configurations

Securing and structuring communication between clients, gateways, and API providers is essential for maintaining data integrity and performance.

The following configurations help ensure your API traffic safely and efficiently:

• mTLS Configuration

  Mutual TLS (mTLS) guarantees that both the client and server authenticate each other before establishing a connection. This is critical in environments that require end-to-end encryption and secure communication.

  Use this configuration when you need to enforce secure communication between your application and the API provider. It’s particularly valuable in highly regulated industries where data security is a priority.

• SSL Communication with LLM Gateway

  SSL encryption secures the traffic between the LLM Gateway and external API providers. This protects sensitive data from being exposed during transmission.

  Implement SSL communication to ensure encrypted traffic between your infrastructure and third-party APIs, preventing unauthorized access to data.

Traffic structuring configurations

To efficiently manage and optimize traffic flow, it’s essential to structure and segment your API traffic.

The following configurations help organize traffic routes and provide flexibility in routing and request management:

• Path parameters configuration

  Defining reusable URL patterns makes managing API endpoints more efficient. This is particularly useful when handling large numbers of similar API requests.

  Apply this configuration to manage path parameters and reduce redundancy when managing multiple API endpoints. This is especially helpful for simplifying complex URL structures.

• Identify your API consumer

  Identifying and segmenting API requests is crucial for monitoring usage and traffic patterns across different applications or environments.

  Use the x-lunar-consumer-tag header to label API requests based on the application or user, providing granular insights and tracking capabilities for multi-tenant environments.

Runtime configurations

Optimizing runtime settings is vital for ensuring smooth operations, especially during high traffic or system shutdowns.

The following configurations focus on fine-tuning the behavior of LLM Gateway components:

• Environment variables

  Configuring environment variables allows you to tailor the operation of the LLM Gateway and Interceptors based on your infrastructure’s specific needs.

  Use environment variables to adjust runtime behaviors like connection settings, logging, and fail-safe mechanisms. This gives you more control over how the gateway operates in different environments.

• Failsafe mechanisms

  To ensure uninterrupted traffic flow during system failures, the Failsafe Mechanism reroutes traffic directly to API providers, preventing downtime.

  This configuration is essential in maintaining high availability, ensuring your system continues to function smoothly even if the LLM Gateway or Interceptor experiences issues.