# Model gateways and inference APIs

One address in front of many models: gateways and routers that add fallbacks, caching, spending limits and logging, and inference services that serve open models behind an API.

57 tools, in alphabetical order. Nothing is ranked by popularity.

- [1MCP Agent](https://modelmoot.com/tools/1mcp-agent.md): A unified MCP server that aggregates many MCP servers behind one connection. (1MCP; open source)
- [Agent Router](https://modelmoot.com/tools/agent-router.md): A project, formerly Envoy AI Gateway, that gives agents one OpenAI-compatible API for many models and one router for MCP tools. (Agentic AI Foundation; open source)
- [agentgateway](https://modelmoot.com/tools/agentgateway.md): An open-source proxy for traffic from AI agents, covering LLM, MCP, A2A and plain HTTP calls. (agentgateway project; open source)
- [AI SDK](https://modelmoot.com/tools/vercel-ai-sdk.md): A TypeScript toolkit for calling models, streaming results, using tools and building agents, with UI hooks for React and other frameworks. (Vercel; open source)
- [Amazon Bedrock](https://modelmoot.com/tools/amazon-bedrock.md): An AWS service that gives access to models from several providers through one API, with agents and guardrails. (Amazon Web Services; not open source, hosted service)
- [any-llm](https://modelmoot.com/tools/any-llm.md): A Python library that gives one interface to many model providers, with an optional gateway server. (Mozilla.ai; open source)
- [Baseten](https://modelmoot.com/tools/baseten.md): A hosted platform for serving models with model APIs and custom deployments, with an OpenAI-compatible endpoint. (Baseten; not open source, hosted service)
- [Bifrost](https://modelmoot.com/tools/bifrost.md): An AI gateway written in Go that provides one OpenAI-compatible API to many providers, and also acts as an MCP gateway. (Maxim AI; open source)
- [Braintrust](https://modelmoot.com/tools/braintrust.md): A hosted platform for running evaluations, logging traces and managing prompts for AI applications. (Braintrust; not open source, hosted service)
- [Cerebras Inference](https://modelmoot.com/tools/cerebras.md): A hosted inference API for open models that runs on Cerebras wafer-scale hardware, with an OpenAI-compatible endpoint. (Cerebras; not open source, hosted service)
- [Claude Code Router](https://modelmoot.com/tools/claude-code-router.md): A local proxy that routes the requests of Claude Code and other coding agents to different models and providers. (musistudio; open source)
- [Cloudflare AI Gateway](https://modelmoot.com/tools/cloudflare-ai-gateway.md): A Cloudflare proxy for model API calls that adds caching, rate limiting, retries, fallbacks and logging. (Cloudflare; not open source, hosted service)
- [Cloudflare Workers AI](https://modelmoot.com/tools/cloudflare-workers-ai.md): A Cloudflare service that runs open models on its network, called from Workers or over a REST API. (Cloudflare; not open source, hosted service)
- [DeepInfra](https://modelmoot.com/tools/deepinfra.md): A hosted inference API for open models with an OpenAI-compatible endpoint and dedicated deployments. (DeepInfra; not open source, hosted service)
- [Fireworks AI](https://modelmoot.com/tools/fireworks-ai.md): A hosted inference platform for open models with an OpenAI-compatible API, fine-tuning and function calling. (Fireworks AI; not open source, hosted service)
- [Gram](https://modelmoot.com/tools/gram.md): An MCP gateway and control plane that turns APIs into curated MCP servers, hosted by Speakeasy or self-hosted. (Speakeasy; open source, hosted service)
- [Groq](https://modelmoot.com/tools/groq.md): A hosted inference API that serves open models on its own language processing hardware, with an OpenAI-compatible endpoint. (Groq; not open source, hosted service)
- [Helicone](https://modelmoot.com/tools/helicone.md): An LLM observability platform that logs requests through a proxy or SDK, with cost tracking and prompt tools. (Helicone; open source, hosted service)
- [Helicone AI Gateway](https://modelmoot.com/tools/helicone-gateway.md): A gateway from Helicone that routes requests to many model providers with logging, cost tracking and fallbacks. (Helicone; open source, hosted service)
- [Higress](https://modelmoot.com/tools/higress.md): A cloud-native API gateway based on Envoy and Istio, with an AI gateway mode and an MCP server hosting feature. (Alibaba; open source)
- [Hugging Face Inference Providers](https://modelmoot.com/tools/huggingface-inference-providers.md): A Hugging Face service that gives one API and token for serverless inference from several partner providers. (Hugging Face; not open source, hosted service)
- [IBM ContextForge](https://modelmoot.com/tools/mcp-context-forge.md): An open-source gateway, registry and proxy that puts MCP servers, A2A agents and REST or gRPC APIs behind one endpoint. (IBM; open source)
- [Kong AI Gateway](https://modelmoot.com/tools/kong-ai-gateway.md): A set of Kong Gateway plugins that route and govern traffic to model providers behind a unified interface. (Kong; open source, hosted service)
- [LiteLLM](https://modelmoot.com/tools/litellm.md): A Python SDK and proxy server that calls many model providers in the OpenAI format, with keys, budgets and routing. (BerriAI; open source, hosted service)
- [llama-swap](https://modelmoot.com/tools/llama-swap.md): A proxy that starts and swaps local model servers on demand behind one OpenAI-compatible endpoint. (mostlygeek; open source)
- [LLM Gateway](https://modelmoot.com/tools/llm-gateway.md): A gateway that routes requests to many providers through one OpenAI-compatible API, hosted or self-hosted. (LLM Gateway; open source, hosted service)
- [Lunar MCPX](https://modelmoot.com/tools/lunar-mcpx.md): A self-hosted gateway for MCP traffic with governance, logging and access control. (Lunar; open source)
- [MCP Gateway and Registry](https://modelmoot.com/tools/mcp-gateway-registry.md): An open-source gateway and registry for MCP servers and agents with OAuth authentication and tool discovery. (agentic-community; open source)
- [mcp-proxy (Python)](https://modelmoot.com/tools/mcp-proxy-py.md): A Python bridge between streamable HTTP and stdio MCP transports, in either direction. (sparfenyuk; open source)
- [mcp-proxy (TBXark)](https://modelmoot.com/tools/mcp-proxy-tbxark.md): A Go proxy that aggregates several MCP servers and serves them through one HTTP server. (TBXark; open source)
- [mcp-proxy (TypeScript)](https://modelmoot.com/tools/mcp-proxy-ts.md): A TypeScript proxy that serves a stdio MCP server over streamable HTTP and SSE. (punkpeye; open source)
- [mcp-remote](https://modelmoot.com/tools/mcp-remote.md): A local bridge that lets a stdio-only MCP client use a remote MCP server, including OAuth sign-in. (punkpeye; open source)
- [MCPHub](https://modelmoot.com/tools/mcphub.md): A self-hosted gateway and control panel for connecting and managing several MCP servers. (samanhappy; open source)
- [MCPJungle](https://modelmoot.com/tools/mcpjungle.md): A self-hosted registry and gateway that manages many MCP servers and serves them to clients from one place. (MCPJungle project; open source)
- [Microsoft Foundry](https://modelmoot.com/tools/azure-ai-foundry.md): A Microsoft Azure platform that hosts models from several providers behind one endpoint, with agent tooling. (Microsoft; not open source, hosted service)
- [Microsoft MCP Gateway](https://modelmoot.com/tools/microsoft-mcp-gateway.md): A reverse proxy and management layer for MCP servers on Kubernetes, with session-aware routing. (Microsoft; open source)
- [Nebius Token Factory](https://modelmoot.com/tools/nebius-token-factory.md): A Nebius hosted inference service for open models with an OpenAI-compatible API, fine-tuning and dedicated endpoints. (Nebius; not open source, hosted service)
- [New API](https://modelmoot.com/tools/new-api.md): A self-hosted LLM gateway and asset management system with OpenAI-, Claude- and Gemini-compatible endpoints. (QuantumNous; open source)
- [Novita AI](https://modelmoot.com/tools/novita-ai.md): A hosted platform with an OpenAI-compatible LLM API, GPU instances and agent sandboxes. (Novita AI; not open source, hosted service)
- [Obot](https://modelmoot.com/tools/obot.md): An open-source platform for hosting, governing and exposing MCP servers inside an organisation. (Obot AI; open source)
- [One API](https://modelmoot.com/tools/one-api.md): A self-hosted system that distributes access to many model providers through one OpenAI-style API with key management. (songquanpeng; open source)
- [Open Responses](https://modelmoot.com/tools/open-responses.md): A specification for a model API interface based on the OpenAI Responses API that several providers can implement. (Open Responses project; open source)
- [OpenRouter](https://modelmoot.com/tools/openrouter.md): A hosted API that routes requests to models from many providers through one OpenAI-compatible endpoint. (OpenRouter; not open source, hosted service)
- [Plano](https://modelmoot.com/tools/plano.md): An AI-native proxy and data plane for agentic apps that handles model routing, orchestration and observability. (Katanemo; open source)
- [Portkey](https://modelmoot.com/tools/portkey.md): An AI gateway with routing, fallbacks, caching and guardrails, available as open source code and as a hosted control plane. (Portkey; open source, hosted service)
- [Replicate](https://modelmoot.com/tools/replicate.md): A hosted API for running open and custom models, with a catalogue of models and a Python and JavaScript client. (Replicate; not open source, hosted service)
- [Requesty](https://modelmoot.com/tools/requesty.md): A hosted LLM gateway with one API for many models, with routing, fallbacks, caching and cost controls. (Requesty; not open source, hosted service)
- [RouteLLM](https://modelmoot.com/tools/routellm.md): A framework for serving and evaluating routers that send each query to a stronger or cheaper model. (LMSYS; open source)
- [SambaNova Cloud](https://modelmoot.com/tools/sambanova.md): A hosted inference API for open models that runs on SambaNova hardware, with an OpenAI-compatible endpoint. (SambaNova; not open source, hosted service)
- [Supergateway](https://modelmoot.com/tools/supergateway.md): A command-line tool that runs stdio MCP servers over SSE or streamable HTTP, and the reverse. (Supercorp; open source)
- [Together AI](https://modelmoot.com/tools/together-ai.md): A hosted inference API for open models, with an OpenAI-compatible endpoint, fine-tuning and dedicated endpoints. (Together AI; not open source, hosted service)
- [ToolHive](https://modelmoot.com/tools/toolhive.md): An open-source platform for running and managing MCP servers in isolated containers. (Stacklok; open source)
- [Traceloop Hub](https://modelmoot.com/tools/traceloop-hub.md): A gateway written in Rust that routes to many model providers with OpenTelemetry-based observability. (Traceloop; open source)
- [Unla](https://modelmoot.com/tools/unla.md): A gateway that turns existing HTTP APIs and MCP servers into MCP endpoints through configuration only. (AmoyLab; open source)
- [Vercel AI Gateway](https://modelmoot.com/tools/vercel-ai-gateway.md): A Vercel service that gives one endpoint and key for models from many providers, with budgets and fallbacks. (Vercel; not open source, hosted service)
- [Vertex AI Model Garden](https://modelmoot.com/tools/vertex-ai-model-garden.md): A Google Cloud catalogue and serving service for Gemini, partner and open models behind the Vertex AI API. (Google Cloud; not open source, hosted service)
- [Wanaku](https://modelmoot.com/tools/wanaku.md): An open-source router and proxy that exposes enterprise integrations to AI agents as MCP tools. (Wanaku project; open source)

HTML: https://modelmoot.com/tools/c/gateways. All categories: https://modelmoot.com/tools.md
