Custom Providers

LLMGateway supports integrating custom OpenAI-compatible providers, allowing you to use any API that follows the OpenAI chat completions format. This feature is perfect for:

• Private or self-hosted LLM deployments
• Specialized AI providers not natively supported
• Internal AI services within your organization
• Testing against different model endpoints

Quick Setup

1. Add a Custom Provider Key

Navigate to your organization's provider settings and add a custom provider via the UI. Provide a lowercase name, OpenAI-compatible base URL, and API token for the custom provider.

2. Make Requests

Once configured, make requests using the format {customName}/{modelName}:

curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mycompany/custom-gpt-4",
    "messages": [
      {
        "role": "user",
        "content": "Hello from my custom provider!"
      }
    ]
  }'

Configuration Requirements

Custom Provider Name

• Format: Lowercase letters (a-z) with optional single hyphens between them
• Examples: mycompany, internal, my-company, eu-west
• Invalid: MyCompany, my_company, 123test, -mycompany, my-, my--company

Base URL

• Must be a valid URL pointing to your provider's base endpoint
• Use an https:// URL for hosted providers; an http:// URL is allowed when self-hosting (for example a service on your own network)
• LLMGateway will append /v1/chat/completions automatically
• Example: https://api.example.com → https://api.example.com/v1/chat/completions

API Token

• Provider-specific authentication token
• Used in the Authorization: Bearer {token} header

Supported Features

Custom providers inherit full LLMGateway functionality.

Custom Model Catalog

By default, requests through a custom provider are not billed and have no enforced limits — LLMGateway has no catalog entry for the model, so it cannot know its pricing, context window, or capabilities. The custom model catalog lets you define that information per custom provider key, so LLMGateway can attribute cost, enforce limits, and report usage just like a built-in model.

Custom catalog models are text-output models. Multi-modal input (images, audio) is still supported and can be priced via the input fields below; output generation (images, video, audio) is intentionally out of scope because it is too provider-specific to bill generically.

Defining a model

Open Organization → Custom Models, pick the custom provider key, and add a catalog entry. Every field except the model id is optional:

Prices are in USD per token and accept either decimal (0.000003) or exponent (3.0e-6) notation.

Cost attribution

When a request matches a catalog entry, LLMGateway bills it using the entry's token prices and records the cost on the activity log. Without a matching entry the request stays unbilled (zero/null cost).

Limit and capability enforcement

When an entry defines limits or capabilities, LLMGateway enforces them before forwarding the request:

• Context size / max output — requests that would exceed the configured window or output budget are rejected with 400.
• Capabilities — when a flag is explicitly set to disabled, requests that use that feature are rejected (e.g. images against a non-vision model, tools against a non-tools model, streaming against a non-streaming model). Flags left unset stay permissive and the upstream provider enforces them.

Restricting to the catalog

Each custom provider key has an Only allow catalog models switch. When enabled, requests through that provider must reference a defined catalog model — undefined models are rejected with 400. This guarantees that every request has known cost attribution and enforced limits. When disabled, undefined models still work but remain unbilled, as before.