Authentication

SMG supports multiple authentication methods for securing access to inference APIs and the control plane, including JWT/OIDC integration, API keys, and role-based access control.


Overview

Multiple Auth Methods

Support for JWT/OIDC, API keys, and worker authentication to fit your deployment model.

Role-Based Access

The admin role has full control plane access; the user role is read-only (GET and HEAD). Data plane APIs use API keys, not roles.

Enterprise SSO

Integrate with Keycloak, Auth0, Azure AD, Okta, and other OIDC providers.

Audit Logging

Track all control plane operations for security monitoring and compliance.


Authentication Methods

Method Use Case Configuration
Control plane JWT/OIDC Enterprise SSO integration with identity providers (admin routes) --jwt-issuer, --jwt-audience
Control plane API keys Service accounts and programmatic access (admin routes) --control-plane-api-keys
Data plane API key Shared bearer token gating data plane routes; also the default API key of startup and discovered workers --api-key
Data plane tenant keys Per-tenant bearer tokens for data plane routes, each resolving to its own tenant identity --tenant-api-key

When to Use Each Method

  • Control plane JWT/OIDC: Use for enterprise deployments with existing identity providers (Keycloak, Auth0, Azure AD, Okta). Provides centralized user management and SSO for control plane operations.
  • Control plane API keys: Use for service-to-service automation against admin endpoints (CI/CD pipelines, tooling). Simpler to set up but requires manual key management.
  • Data plane API key: Use when you want a single shared secret that clients present on data plane routes (chat, completions, responses, and so on), and that workers started with the same key accept.
  • Data plane tenant keys: Use when each team or application needs its own data plane key, so that tenant rate limits and priority scheduling can tell callers apart. Each --tenant-api-key tenant_id:key resolves to the tenant auth:<tenant_id>. Tenant keys never unlock control plane routes.

JWTs and control plane API keys are checked only on control plane routes. Data plane routes accept only --api-key and --tenant-api-key credentials, and are open when neither is set.


JWT/OIDC Authentication

JWT (JSON Web Token) authentication allows integration with OIDC-compliant identity providers for enterprise single sign-on on the control plane.

Configuration Options

Option Environment Variable Description
--jwt-issuer JWT_ISSUER OIDC issuer URL (required for JWT auth)
--jwt-audience JWT_AUDIENCE Expected audience claim (required for JWT auth)
--jwt-jwks-uri JWT_JWKS_URI Explicit JWKS URI (auto-discovered if not set)
--jwt-role-claim - Claim name containing roles (default: roles)
--jwt-role-mapping - Map IDP roles to gateway roles

The JWT_* environment variables belong to the Rust smg binary; the Python launcher (smg launch from pip, and the container image) takes the JWT settings as flags only. See Python Launcher Differences.

Basic Setup

Enable JWT authentication by providing the issuer and audience:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com/realms/myrealm" \
  --jwt-audience "smg-gateway"

--jwt-issuer and --jwt-audience are required together: SMG refuses to start with only one of them.

JWKS Discovery

By default, SMG discovers the JWKS (JSON Web Key Set) endpoint automatically via OIDC discovery (/.well-known/openid-configuration). You can override this with an explicit JWKS URI:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --jwt-jwks-uri "https://auth.example.com/.well-known/jwks.json"

Without --jwt-jwks-uri, SMG fetches the discovery document at startup. Either way, SMG fetches the key set on first use and caches it for an hour.

The discovery document's issuer must be identical to the configured --jwt-issuer: a document naming another issuer — even one differing only in a trailing slash — is refused, and JWT initialization fails.

Tokens must carry a kid header that names a key in the set. When a token names a key the cached set doesn't have, SMG fetches the set again, which is how a rotated key gets picked up. This refetch happens at most once a minute, and a failed fetch also counts against the minute. Until the minute has passed, tokens naming unknown keys are rejected without a request to the identity provider.

The discovery URL and the JWKS URI must use HTTPS (plain HTTP is allowed only for localhost, 127.0.0.1, and ::1), must not point at a private, loopback, link-local, or other internal IP address literal, and must not use a host name ending in .internal or .local. SMG does not follow redirects on these requests.

If SMG cannot set up JWT validation at startup (for example, OIDC discovery fails or a URL is rejected), control plane authentication stays required: SMG logs Failed to initialize JWT authentication: <error> and runs with JWT validation unavailable. Keys from --control-plane-api-keys keep working, while JWTs, the shared --api-key, and any other token get 401 on control plane routes. SMG does not retry JWT initialization while running; restart the gateway after correcting the configuration or restoring the identity provider.

Role Mapping

Map identity provider roles to SMG gateway roles:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user"

Role Mapping Format: idp_role=gateway_role

SMG refuses to start on a malformed mapping: one without =, or with a gateway role other than admin or user.

Gateway Role Permissions
admin Full access to all control plane APIs (workers, WASM modules, tokenizers)
user Read-only control plane access: GET and HEAD, except the admin-only GET /config; any other method answers 403

Supported Claims

SMG reads role values from the configured --jwt-role-claim (default: roles). Only when the token has no such claim does it fall back to the role, roles, groups, and group claims, collecting the values of all of them in that order. Each claim may be a string or an array of strings.

  • Without --jwt-role-mapping, the first value equal to admin or user (ignoring case) sets the role.
  • With --jwt-role-mapping, the first value that has a mapping sets the role; unmapped values are ignored.

If no role is found, the user defaults to the user role.

Supported Algorithms

SMG supports the following JWT signing algorithms:

  • RSA: RS256, RS384, RS512
  • ECDSA: ES256, ES384

Identity Provider Setup

Keycloak

1. Create a Client

  • Navigate to Clients > Create
  • Client ID: smg-gateway
  • Client Protocol: openid-connect
  • Access Type: confidential or public

2. Configure Mappers

  • Add a mapper of type "User Realm Role"
  • Token Claim Name: roles
  • Add to ID token: Yes
  • Add to access token: Yes
smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://keycloak.example.com/realms/myrealm" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "admin=admin" \
  --jwt-role-mapping "user=user"

Auth0

1. Create an API

  • Navigate to Applications > APIs > Create API
  • Name: SMG Gateway
  • Identifier: https://smg.example.com/api

2. Add Roles Action

exports.onExecutePostLogin = async (event, api) => {
  const namespace = 'https://smg.example.com';
  if (event.authorization) {
api.accessToken.setCustomClaim(
  `${namespace}/roles`,
  event.authorization.roles
);
  }
};
smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://your-tenant.auth0.com/" \
  --jwt-audience "https://smg.example.com/api" \
  --jwt-role-claim "https://smg.example.com/roles" \
  --jwt-role-mapping "smg-admin=admin" \
  --jwt-role-mapping "smg-user=user"

Azure AD / Entra ID

1. Register an Application

  • Navigate to Azure Portal > App registrations
  • Name: SMG Gateway
  • Configure app roles: Gateway.Admin, Gateway.User

2. Expose an API

  • Navigate to Expose an API
  • Set Application ID URI: api://smg-gateway
smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://login.microsoftonline.com/{tenant-id}/v2.0" \
  --jwt-audience "api://smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user"

Okta

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://your-org.okta.com/oauth2/default" \
  --jwt-audience "api://smg" \
  --jwt-role-mapping "smg_admins=admin" \
  --jwt-role-mapping "smg_users=user"

API Key Authentication

API keys provide a simpler authentication method for service accounts and programmatic access.

Control Plane API Keys

Configure API keys for control plane access:

smg \
  --worker-urls http://worker:8000 \
  --control-plane-api-keys "key1:Service Account:admin:sk-your-secret-key-here"

Format: id:name:role:key

Component Description
id Unique identifier for the key
name Human-readable name/description
role Gateway role (admin or user)
key The secret API key value

SMG refuses to start on a malformed entry, including one with a role other than admin or user. The error does not echo the entry, since it carries the key.

Multiple API Keys

smg \
  --worker-urls http://worker:8000 \
  --control-plane-api-keys "admin1:Admin Service:admin:sk-admin-key-12345" \
  --control-plane-api-keys "user1:Read Only Service:user:sk-readonly-key-67890"

A user key can read the control plane (GET and HEAD, except the admin-only GET /config); any other control plane method answers it with 403 (see Role-Based Access Control).

Environment Variable Configuration

For security, pass API keys via environment variable:

export CONTROL_PLANE_API_KEYS="admin1:Admin Service:admin:sk-admin-key-12345"
smg --worker-urls http://worker:8000

CONTROL_PLANE_API_KEYS holds one or more id:name:role:key entries, separated by commas or newlines, and both the Rust smg binary and the Python launcher (smg launch from pip, and the container image) read it. The flag beats the environment, and a variable that is set but holds no entry fails startup.

Key Files

Each key setting — the control plane keys, the shared data plane key, and the tenant keys — can also come from a file, such as a mounted Kubernetes Secret:

Setting Flag Environment File flag File's environment
Control plane keys --control-plane-api-keys CONTROL_PLANE_API_KEYS --control-plane-api-keys-file CONTROL_PLANE_API_KEYS_FILE
Shared data plane key --api-key SMG_API_KEY --api-key-file SMG_API_KEY_FILE
Tenant keys --tenant-api-key SMG_TENANT_API_KEYS --tenant-api-keys-file SMG_TENANT_API_KEYS_FILE

A key-list file holds one entry per line (id:name:role:key or tenant_id:key); blank lines and lines starting with # are skipped, and a file that holds no entry is refused — it was meant to close the routes its keys guard. --api-key-file holds the key alone on one line, surrounding whitespace dropped; an empty file or one with more than one line is refused. For each setting, the flag beats its environment variable, and a value together with a file is refused rather than ranked. A refusal names the setting and the path, never the key.

Reloading Keys Without a Restart

POST /secrets/reload reads every key file given at start again and puts the new sets in force in one swap; keys that came from a flag or an environment variable stay as the start read them. The route requires the admin role and exists only when control plane authentication is configured — without it the path answers 404. It answers:

  • 200 with a report of the files read and whether the serving and control plane key sets changed
  • 409 when no key came from a file, so there is nothing to reload
  • 422 when a file cannot be used — unreadable, malformed, or breaking a startup rule such as a duplicate key — with every set standing as it was

Requests in flight finish under the keys they were admitted with. With --secrets-auto-reload, SMG watches the key files and runs the same reload when one changes, so a mounted Secret rotated in place takes effect without a restart or an API call.

Using API Keys

Clients authenticate by including the API key in the Authorization header:

curl -H "Authorization: Bearer sk-admin-key-12345" \
  https://smg.example.com/workers

Security Features

  • Hashed Storage: Keys are SHA-256 hashed immediately; plaintext keys are never stored in memory
  • Constant-Time Comparison: Key verification uses constant-time comparison to prevent timing attacks
  • Role-Based Access: Each key is assigned a specific role limiting its permissions

Data Plane API Key (--api-key)

The --api-key option configures a single bearer token that does two things:

smg \
  --worker-urls http://worker:8000 \
  --api-key "shared-secret-key"
  1. Gates incoming data plane requests. The gateway requires every client request to data plane routes (/v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/rerank, /v1/messages, /v1/realtime/*, etc.) to present Authorization: Bearer <api-key> (or a --tenant-api-key key). Requests without a valid token receive 401 Unauthorized. Public routes such as /health and /v1/models stay open.
  2. Becomes the API key of startup and discovered workers. Workers from --worker-urls (and from --prefill and --decode) and workers that Kubernetes service discovery registers get the --api-key value as their worker API key. A worker added through POST /workers uses only the api_key in its own spec; SMG logs a warning when one arrives without a key while --api-key is set.

Like the other key settings, the key can come from the SMG_API_KEY environment variable or a file (--api-key-file), and tenant keys from SMG_TENANT_API_KEYS or --tenant-api-keys-file; see Key Files.

What reaches a worker depends on the path:

Path What the worker receives
HTTP regular router The client's Authorization header as sent, else the worker's API key as Authorization: Bearer <key>. A matched gateway key (--api-key or a tenant key) is removed at authentication, so the worker receives its own API key or, when it has no key, no Authorization header at all.
HTTP PD router The client's allowlisted headers on both the prefill and the decode request, with the same Authorization precedence as the regular router: the client's header when one reaches the router, else each worker's own API key as Authorization: Bearer <key>.
gRPC and ZMQ workers No API key and no Authorization metadata.
SMG's own calls to HTTP workers (health checks, metadata discovery, load polling) The worker's API key, as Authorization: Bearer <key>.
External providers See External Providers.

Service discovery registers each pod as a bare host:port, so SMG reaches it over plain HTTP (or plaintext gRPC), and a worker API key crosses the network unencrypted.

This is useful when:

  • Clients and workers share a common token
  • Workers require authentication (e.g., deployed with API key protection)
  • Using DP-aware scheduling that requires authenticated worker queries
  • Workers are behind an authentication proxy

Control plane routes (/workers, /wasm, /v1/tokenizers, etc.) use their own middleware stack. When --control-plane-api-keys or --jwt-* are configured they take over as the admin auth backend (with role-based access control). With this backend, the Rust binary enables audit logging by default; the Python launcher (pip and container image) requires --control-plane-audit-enabled. When neither auth option is set, admin routes fall back to the same --api-key bearer check that guards the data plane (tenant keys are never accepted there, and with only --tenant-api-key set, admin routes reject every request with 401). If you run with only --api-key, the same shared secret therefore gates both the data plane and the control plane — which is rarely what you want in production. See Control Plane Auth for configuring JWT/OIDC or dedicated control-plane keys.


Role-Based Access Control

SMG implements role-based access control (RBAC) with two primary roles:

Admin Role

Full access to all control plane APIs:

  • Worker management (/workers, /workers/{id})
  • WASM module management (/wasm/*)
  • Tokenizer configuration
  • System administration
  • Configuration view and key reload (GET /config, POST /secrets/reload) — two routes that exist only when control plane authentication is configured

User Role

Read-only control plane access: GET and HEAD on the control plane routes. Every other method answers a user credential with 403 Admin role required for this control plane operation. One read is kept to the admin role: GET /config, which names the store addresses, answers a user credential with the same 403.

Roles don't apply to data plane routes (chat completions, completions, embeddings, and the rest): those accept only --api-key and --tenant-api-key credentials, never JWTs or control plane API keys. /v1/models is public.

Role Assignment

Roles are assigned through:

  1. JWT Claims: Via --jwt-role-mapping configuration
  2. API Key Configuration: Via the role component in --control-plane-api-keys

If no role can be determined, the user defaults to user role for safety.


Audit Logging

SMG provides audit logging for control plane operations to support security monitoring and compliance.

Configuration

With control plane authentication configured (--jwt-issuer with --jwt-audience, or control plane API keys from any source), the Rust smg binary logs audit events by default. To disable:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --disable-audit-logging

The Python launcher (smg launch from pip, and the container image) works the other way around: audit logging is off unless you pass --control-plane-audit-enabled, and it has no --disable-audit-logging flag. Neither launcher audits admin requests checked by the --api-key fallback.

Audit Log Format

Each audit event is an INFO log record with target smg::audit; the layout of the record follows your log format (--log-json for JSON). A control-plane call writes up to two lines:

  • control_plane_audit — the admission line, written before the handler runs, for every request that reaches the control plane authentication check: successes, 403 role denials, and 401 authentication failures.
  • control_plane_mutation — the outcome line, written after the handler of an admitted request other than a GET or HEAD has answered. It records what the handler answered: the HTTP status, an outcome of success for a 2xx status and failed for any other, and the resource the handler acted on, when it named one.

Both lines of a mutation carry the same principal, auth_method, role, method, path, and request_id, so one request ID correlates them. A read (GET or HEAD) writes the admission line alone. A denied or unauthenticated request also writes only the admission line, recording the denial — no handler ran, so there is no outcome line. --disable-audit-logging silences both lines.

Audit Event Fields

Field Description
timestamp RFC 3339 timestamp of the event
principal JWT subject (or its email, then preferred_username, when sub is missing), the API key's id, or unauthenticated when authentication failed
auth_method Authentication method (jwt, api_key, or none when authentication failed)
role Role of the principal (admin, user)
method HTTP method (GET, POST, DELETE, etc.)
path Request path
resource What the handler acted on: a worker ID, a tokenizer ID, WASM module UUIDs (comma-joined for a batch), or secrets for a key-file reload (POST /secrets/reload). Only on a control_plane_mutation line whose handler named one; left out everywhere else
outcome On the admission line, success (admitted) or denied — decided before the handler runs; on the outcome line, success (a 2xx status) or failed (any other status)
status The HTTP status the handler answered. Only on the control_plane_mutation line; the admission line carries none
request_id Correlation ID for request tracing; a mutation's admission and outcome lines share it
details Why the request was denied or failed authentication

Viewing Audit Logs

# Filter for audit logs
RUST_LOG=smg::audit=info smg ...

# Or view in combined logs (both the admission and the outcome lines)
kubectl logs -n inference -l app=smg | grep -E "control_plane_(audit|mutation)"

Production Configuration

A production setup combining JWT and API key authentication:

smg \
  --worker-urls http://worker1:8000 http://worker2:8000 \
  --host 0.0.0.0 \
  --port 8443 \
  --tls-cert-path /etc/certs/server.crt \
  --tls-key-path /etc/certs/server.key \
  --jwt-issuer "https://auth.example.com/realms/production" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user" \
  --control-plane-api-keys "ci-cd:CI/CD Pipeline:admin:${CI_CD_API_KEY}" \
  --api-key "${DATA_PLANE_API_KEY}"

JWT and the control plane key protect the control plane routes. --api-key protects the data plane routes, which are open without it.

The example serves on 8443, not 443: the published image runs as uid 65532 and the Helm chart drops every capability, so a gateway in that image cannot bind a port below 1024. Put a Service, an ingress or a load balancer in front for 443, or grant the container CAP_NET_BIND_SERVICE.


Kubernetes Deployment

Secret for API Keys

apiVersion: v1
kind: Secret
metadata:
  name: smg-auth
  namespace: inference
type: Opaque
stringData:
  CONTROL_PLANE_API_KEYS: "admin1:Admin:admin:sk-secret-key"

ConfigMap for JWT Configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: smg-config
  namespace: inference
data:
  JWT_ISSUER: "https://auth.example.com/realms/production"
  JWT_AUDIENCE: "smg-gateway"

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: smg
  namespace: inference
spec:
  template:
    spec:
      containers:
        - name: smg
          image: lightseekorg/smg:latest
          envFrom:
            - configMapRef:
                name: smg-config
            - secretRef:
                name: smg-auth
          args:
            - --service-discovery
            - --selector
            - app=sglang-worker
            - --jwt-issuer
            - $(JWT_ISSUER)
            - --jwt-audience
            - $(JWT_AUDIENCE)
            - --jwt-role-mapping
            - "Gateway.Admin=admin"
            - --jwt-role-mapping
            - "Gateway.User=user"
            - --control-plane-audit-enabled

The container image runs the Python launcher, which does not read JWT_ISSUER or JWT_AUDIENCE itself, so the args pass them as flags; Kubernetes expands each $(VAR) reference from the container's environment. CONTROL_PLANE_API_KEYS needs no flag: the launcher reads the variable, with entries separated by commas or newlines. --control-plane-audit-enabled turns on audit logging, which the Python launcher leaves off by default. To rotate keys without a restart, mount the Secret as a file instead and pass --control-plane-api-keys-file with --secrets-auto-reload (see Key Files).


Troubleshooting

JWT Validation Failures

Symptom: Invalid JWT or Token validation failed errors

Solutions:

  1. Verify issuer URL matches exactly (including trailing slash):

    curl https://auth.example.com/.well-known/openid-configuration
  2. Verify audience claim matches your configuration:

    echo "YOUR_JWT" | cut -d. -f2 | base64 -d | jq .
  3. Check clock synchronization (JWT validation uses time-based claims and allows 30 seconds of clock skew)

  4. Verify JWKS endpoint is accessible from the SMG pod

API Key Not Working

Symptom: Invalid authentication token errors

Solutions:

  1. Verify key format is correct: id:name:role:key
  2. Check for special characters that may need escaping
  3. Ensure the Authorization header format is correct: Bearer <key>

Role Mapping Issues

Symptom: Users getting wrong permissions

Solutions:

  1. Check which claim contains roles in your JWT
  2. Verify role mapping syntax: idp_role=gateway_role
  3. Check if role claim name needs to be specified with --jwt-role-claim

Security Best Practices

Use HTTPS

Always enable TLS for the gateway in production.

Rotate Keys

Regularly rotate API keys and use short-lived JWT tokens.

Least Privilege

Assign user role by default, admin only when needed.

Enable Auditing

Keep audit logs for security monitoring and compliance.


What's Next?

Rate Limiting

Protect against overload and abuse.

Rate Limiting →

High Availability

Deploy SMG in a highly available configuration.

High Availability →

Metrics Reference

Count 401 and 403 responses by path with smg_http_responses_total.

Metrics Reference →