Skip to main content
Feedback

LLM Gateway Pro installation

To scale LLM Gateway, you can run multiple instances concurrently. This setup ensures that traffic is balanced across the instances, enhancing both performance and fault tolerance.

Getting started

1. Obtain the LLM Gateway Pro image:

  • The lunar-proxy-pro image required for LLM Gateway Pro is not publicly available. To obtain this image, you must contact LLM Gateway directly to request access. Once you receive the necessary access key, make sure to replace the standard lunar-proxy image with the lunar-proxy-pro image.

2. Set up Redis for shared state:

  • You’ll need a running Redis instance to store the shared state between LLM Gateway Pro instances. You can quickly set this up using the redis image from Docker Hub.
  • Once the Redis instance is running, configure each LLM Gateway Pro instance to connect to Redis by setting the REDIS_URL environment variable. The URL format should be redis://<host>:<port> or rediss://<host>:<port> if using TLS.
note

The redis will use lunar as its default prefix for the gateway connection with Redis, to configure the value please change:

  1. REDIS_PREFIX when running using Docker
  2. redisPrefix when using Helm Chart

3. mTLS for Redis:

  • For enhanced security, LLM Gateway Pro supports connecting to Redis using mutual TLS (mTLS). By using mTLS, you ensure that communication between LLM Gateway Pro instances and Redis is encrypted and secure.
  • Follow the steps in the mTLS with Redis Configuration guide to configure mTLS for your Redis connection.

Redis connection status

During startup, LLM Gateway will display a message indicating the Redis connection status, including the prefix and URL used. A "Successfully initialized Redis Client" message confirms a successful connection.

For a full breakdown of startup indicators, refer to the LLM Gateway Logs page.

On this Page