Skip to main content
Feedback

AI Gateway overview

Boomi AI Gateway lets you secure, govern, and observe every interaction between AI agents, APIs, and data sources. Whether you're connecting large language models (LLMs), orchestrating agentic workflows, or managing custom MCP servers, AI Gateway helps you do it safely and at scale. It brings together three components: AI Gateway, MCP Gateway, and Skills.

Why Boomi AI Gateway

AI infrastructure is no longer just about connecting to models; it's about controlling the entire agentic ecosystem. AI Gateway gives you that control across your AI stack:

  • Observe: Gain real-time visibility into every model, API, and tool invocation.
  • Enforce: Apply policies, role-based access, rate limits, and cost controls.
  • Secure: Prevent prompt injection, data leakage, and untrusted agent behavior.

AI Gateway powers both AI traffic governance and MCP tool governance under one roof.

LLM Gateway

LLM Gateway centralizes control of all LLM and API traffic in your organization. It gives you visibility, policy enforcement, and identity-aware routing with minimal latency overhead.

  • Production-ready for enterprise workloads.
  • Cost- and latency-aware routing across model providers.
  • Inline request inspection and data redaction.
  • Full observability and policy-based enforcement.

Core components of LLM Gateway include the following:

  • Flows: YAML-based configurations that define how API requests and responses are processed in real time.
  • Quotas: Centralized limits on API consumption, set by URL, header, or user group, for real-time tracking and control.
  • control plane: A graphical interface for installation, traffic monitoring, performance review, and quota and team management.

MCP Gateway

MCP Gateway governs the tool invocation layer for AI agents using the model context protocol (MCP). It acts as a single endpoint aggregating multiple MCP servers, with full security and audit controls.

  • Register multiple MCP servers under one governed endpoint.
  • Enforce authentication and scoped permissions per user or agent.
  • Forward secrets securely and record full audit trails.
  • Integrate with LLM Gateway for end-to-end traffic visibility.

As agents and AI-powered applications connect to more MCP servers, MCP Gateway helps teams stay in control through:

  • Centralized access: Aggregates and exposes any number of local and remote MCP servers through a single endpoint.
  • Policy enforcement: Controls which tools and methods are accessible to which agents.
  • Customization: Tailors tools to specific workflows.
  • Full observability: Tracks usage, latency, and errors across all tool calls.
  • Authorization management: Centralizes secrets, API keys, and OAuth tokens.

How LLM Gateway and MCP Gateway work together

Together, LLM Gateway and MCP Gateway form a complete governance stack for AI applications. LLM Gateway manages model and API traffic. MCP Gateway manages tool and agent interactions; and both share unified policies, identity, and observability. You can deploy them independently or together on Kubernetes, alongside your organization's identity provider and monitoring stack.

Skills

MCP Gateway lets you package a task-specific capability as a Skill. A Skill bundles instructions for an agent with the exact tools and prompts it needs to act on them. It follows the community SKILL.md convention extended with a capability group. This group is a curated set of tools and prompts that travels with the instructions as one object. Once created, any connected agent can discover a Skill through MCP itself or invoke it explicitly as a slash command.

On this Page