> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roomote.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI-compatible

> Connect one or more OpenAI API endpoints as Roomote inference providers.

OpenAI-compatible is the generic option for any server that implements the
OpenAI `/v1` API. Use it for LiteLLM proxies, vLLM, local OpenAI servers, or
other OpenAI-shaped gateways when you do not need a provider-specific setup page.

You can connect **multiple** OpenAI-compatible endpoints. Each connection needs
a name so Roomote can keep their credentials, discovered models, and model ids
separate.

## Configure an OpenAI-compatible endpoint

In **Settings > Models**, add **OpenAI-compatible**, enter a connection name
(for example `company-proxy`), and provide the endpoint URL. An API key is
optional and only needed when your endpoint requires bearer auth.

Roomote stores each connection under namespaced deployment variables derived
from the connection name. For a connection named `company-proxy`:

```sh theme={null}
OPENAI_COMPATIBLE_COMPANY_PROXY_BASE_URL=https://proxy.example.com/v1
OPENAI_COMPATIBLE_COMPANY_PROXY_API_KEY=...
OPENAI_COMPATIBLE_COMPANY_PROXY_LABEL=company-proxy
```

The endpoint must be reachable from the Roomote deployment. For a Compose
deployment, use a service DNS name such as `http://litellm:4000/v1` when the
proxy runs on the same private network. Do not use `localhost` unless the
endpoint runs in the same network namespace as the Roomote service that proxies
inference requests.

After saving the provider, Roomote discovers models from `/v1/models`. Enable
the models you want and select a default coding model and any specialized roles.
Model IDs use the `openai-compatible-<connection>/<model-id>` form, where
`<connection>` is the slug from the connection name and `<model-id>` is the
name returned by your endpoint.

## When to use this versus LiteLLM, Ollama, or vLLM

Choose **OpenAI-compatible** when you want one clear setup path for any OpenAI
API endpoint, or when you need several custom endpoints side by side. Choose the
named LiteLLM, Ollama, or vLLM providers when you prefer those labels or their
dedicated env var names. Runtime discovery and chat completion behavior are the
same OpenAI `/v1` protocol for all of them.

## Secure the endpoint

Keep the endpoint private to Roomote whenever possible. Put it on an internal
network or behind a private ingress, require an API key when the server supports
it, and use TLS when traffic crosses an untrusted network. Store connection API
keys in your deployment secret manager or as encrypted Roomote deployment
variables, not in an environment's task variables or repository files.

Roomote proxies model traffic through its inference gateway, so task sandboxes
do not need direct network access to the endpoint or the API key.

## Verify setup

1. add OpenAI-compatible with a connection name, endpoint URL, and optional API key
2. confirm models appear in **Settings > Models** under that connection
3. enable one model and make it the default coding model
4. start a small Roomote task and confirm it completes through the endpoint
5. check the upstream server logs for the request

## Common issues

* **No models appear.** Confirm the endpoint URL includes the OpenAI `/v1` API
  path and that the API key can list models when auth is required.
* **Tasks cannot reach the endpoint.** Check DNS, container networking, firewall
  rules, and whether the endpoint is reachable from the Roomote deployment.
* **Tool calling fails.** Roomote needs models that support tool calling. Confirm
  the upstream model and gateway configuration expose tools over chat completions.
* **Two endpoints collide.** Give each OpenAI-compatible connection a distinct
  name so their env vars and `openai-compatible-<name>/...` model ids stay unique.