/realtime

Use this to loadbalance across Azure + OpenAI + xAI and more.

Supported Providers:

• OpenAI
• Azure
• xAI (see full docs)
• Google AI Studio (Gemini)
• Vertex AI
• Bedrock

Proxy Usage

Add model to config

OpenAI

model_list:
  - model_name: openai-gpt-4o-realtime-audio
    litellm_params:
      model: openai/gpt-4o-realtime-preview-2024-10-01
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: realtime

OpenAI + Azure

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/gpt-4o-realtime-preview
      api_key: os.environ/AZURE_SWEDEN_API_KEY
      api_base: os.environ/AZURE_SWEDEN_API_BASE

  - model_name: openai-gpt-4o-realtime-audio
    litellm_params:
      model: openai/gpt-4o-realtime-preview-2024-10-01
      api_key: os.environ/OPENAI_API_KEY

xAI Grok Voice Agent

Connect to xAI's Grok Voice Agent for real-time voice conversations:

// test_xai.js
const WebSocket = require("ws");

const url = "wss://api.haimaker.ai/v1/realtime?model=xai/grok-4-1-fast-non-reasoning";

const ws = new WebSocket(url, {
    headers: {
        "Authorization": "Bearer YOUR_API_KEY",
    },
});

ws.on("open", function open() {
    console.log("Connected to xAI Grok Voice Agent via haimaker");
    ws.send(JSON.stringify({
        type: "conversation.item.create",
        item: {
            type: "message",
            role: "user",
            content: [{
                type: "input_text",
                text: "Hello! How are you?"
            }]
        }
    }));
    ws.send(JSON.stringify({
        type: "response.create",
        response: {
            modalities: ["text"],
            instructions: "Please assist the user.",
        }
    }));
});

ws.on("message", function incoming(message) {
    const data = JSON.parse(message.toString());
    console.log(`Event: ${data.type}`);
    if (data.type === 'response.done') {
        ws.close();
    }
});

ws.on("error", function handleError(error) {
    console.error("Error: ", error);
});

See full xAI Realtime documentation

Start proxy

litellm --config /path/to/config.yaml 

# RUNNING on http://0.0.0.0:8000

Test

Run this script using node - node test.js

// test.js
const WebSocket = require("ws");

const url = "ws://0.0.0.0:4000/v1/realtime?model=openai-gpt-4o-realtime-audio";
// const url = "wss://my-endpoint-sweden-berri992.openai.azure.com/openai/realtime?api-version=2024-10-01-preview&deployment=gpt-4o-realtime-preview";
const ws = new WebSocket(url, {
    headers: {
        "api-key": `f28ab7b695af4154bc53498e5bdccb07`,
        "OpenAI-Beta": "realtime=v1",
    },
});

ws.on("open", function open() {
    console.log("Connected to server.");
    ws.send(JSON.stringify({
        type: "response.create",
        response: {
            modalities: ["text"],
            instructions: "Please assist the user.",
        }
    }));
});

ws.on("message", function incoming(message) {
    console.log(JSON.parse(message.toString()));
});

ws.on("error", function handleError(error) {
    console.error("Error: ", error);
});

Logging

To prevent requests from being dropped, by default haimaker just logs these event types:

• session.created
• response.create
• response.done

You can override this by setting the logged_real_time_event_types parameter in the config. For example:

litellm_settings:
  logged_real_time_event_types: "*" # Log all events
  ## OR ## 
  logged_real_time_event_types: ["session.created", "response.create", "response.done"] # Log only these event types