OpenAI-Compatible API Reference¶

SMG provides a fully OpenAI-compatible API, allowing you to use existing OpenAI client libraries with your self-hosted inference workers.

Base URL¶

http://localhost:30000/v1

Authentication¶

SMG supports optional API key authentication:

curl http://localhost:30000/v1/chat/completions \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '...'

Enable authentication with --api-key:

smg --worker-urls http://worker:8000 --api-key "your-api-key"

Endpoints¶

Chat Completions¶

Create a chat completion.

POST /v1/chat/completions

Request Body¶

Message Object¶

Example Request¶

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "max_tokens": 100,
    "temperature": 0.7
  }'

Response¶

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1705312345,
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 8,
    "total_tokens": 33
  }
}

Streaming Response¶

With "stream": true, responses are sent as Server-Sent Events:

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Response:

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"!"}}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Completions¶

Create a text completion (legacy API).

POST /v1/completions

Request Body¶

Example Request¶

curl http://localhost:30000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "prompt": "The quick brown fox",
    "max_tokens": 50
  }'

Response¶

{
  "id": "cmpl-abc123",
  "object": "text_completion",
  "created": 1705312345,
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "choices": [
    {
      "text": " jumps over the lazy dog.",
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 4,
    "completion_tokens": 7,
    "total_tokens": 11
  }
}

List Models¶

List available models.

GET /v1/models

Example Request¶

curl http://localhost:30000/v1/models

Response¶

{
  "object": "list",
  "data": [
    {
      "id": "meta-llama/Llama-3.1-8B-Instruct",
      "object": "model",
      "created": 0,
      "owned_by": "self_hosted"
    }
  ]
}

owned_by is self_hosted for locally hosted workers, or the provider name (for example openai, anthropic, xai, gemini) for upstream providers.

Audio Transcriptions¶

Transcribe an audio file (batch). Requires a worker serving an ASR model.

POST /v1/audio/transcriptions

Sent as multipart/form-data with fields file (the audio) and model, plus optional language, prompt, response_format, temperature, and stream.

curl http://localhost:30000/v1/audio/transcriptions \
  -F file=@audio.wav \
  -F model=Qwen/Qwen3-ASR-1.7B

Realtime API¶

SMG proxies the OpenAI Realtime API to a realtime-capable worker. Both the OpenAI router (to an upstream provider) and the HTTP router (to a local worker labeled realtime: "true") support it. SMG relays frames verbatim, so the worker must speak the OpenAI Realtime protocol — for local workers, for example vLLM serving an ASR model with the realtime task.

WebSocket example¶

# pip install websockets
import asyncio, websockets

async def main():
    url = "ws://localhost:30000/v1/realtime?model=Qwen/Qwen3-ASR-1.7B"
    headers = {"Authorization": "Bearer your-api-key"}
    async with websockets.connect(url, additional_headers=headers) as ws:
        # Send realtime events (session.update, input_audio_buffer.append, ...)
        # and receive transcription/response events from the worker.
        ...

asyncio.run(main())

Error Responses¶

Error Format¶

{
  "error": {
    "message": "Error description",
    "type": "error_type",
    "code": "error_code"
  }
}

Error Codes¶

Example Error Response¶

{
  "error": {
    "message": "Rate limit exceeded. Please retry later.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded"
  }
}

Client Libraries¶

Python (OpenAI SDK)¶

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="your-api-key"  # or "not-needed" if auth disabled
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)

print(response.choices[0].message.content)

JavaScript/TypeScript¶

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:30000/v1',
  apiKey: 'your-api-key'
});

const response = await client.chat.completions.create({
  model: 'meta-llama/Llama-3.1-8B-Instruct',
  messages: [
    { role: 'user', content: 'Hello!' }
  ]
});

console.log(response.choices[0].message.content);

cURL¶

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Request Headers¶

Rate Limiting¶

When rate limited, responses include: