Prompt Caching

Prompt caching support is model-specific. Check supported_openai_params in GET /public/model_hub before using explicit cache_control blocks. Providers such as OpenAI may also cache eligible prompts automatically.

Cache prices

Haimaker exposes published cache prices in the public model catalog, alongside ordinary input and output prices. Fetch it without an API key:

curl -sS https://api.haimaker.ai/public/model_hub \
  | jq '.[] | {model_group, mode, input_cost_per_token, output_cost_per_token,
               cache_read_input_token_cost, cache_creation_input_token_cost}'

Field	Meaning
`cache_read_input_token_cost`	USD per input token read from the prompt cache.
`cache_creation_input_token_cost`	USD per input token written to the prompt cache, at the default cache TTL.

Both fields are nullable numbers on catalog entries for Chat Completions and Responses models, documented in the public GET /openapi.json schema. Multiply a rate by 1,000,000 to display USD per million tokens; for example, 0.0000001 means $0.10 per million tokens.

The dashboard's Model Hub shows the same prices: the cache-read price under each input price in the table, and both prices in each model's details.

null means no rate is published. It does not mean free, or that caching is unsupported. A model also publishes null when requests for it can be served at more than one cache price, because no single price applies.
0 is an explicitly published zero rate. Keep it distinct from null.
A cache price equal to the model's input price means those tokens are billed like ordinary input on that model.
These are default-tier reference prices, not a guaranteed quote for every request. Context length, cache TTL, service tier, routing, and promotions can affect the final charge. Existing usage and billing rules remain authoritative.
When time_based_pricing is present, the top-level rates reflect its current window; time_based_pricing.schedule includes cache-read and cache-write rates for each window. The promo object is separate; cache rates do not imply that a promotional price and a cache discount stack.

A published rate does not enable caching or guarantee a cache hit. Eligible prompt length, cache markers, retention, and model support still apply. Check the response's reported usage to see whether that particular request hit the cache. Do not use the presence of a price as a capability flag.

Cache usage

haimaker follows the OpenAI prompt caching usage object format:

"usage": {
  "prompt_tokens": 2006,
  "completion_tokens": 300,
  "total_tokens": 2306,
  "prompt_tokens_details": {
    "cached_tokens": 1920
  },
  "completion_tokens_details": {
    "reasoning_tokens": 0
  },
  "cache_creation_input_tokens": 0
}

prompt_tokens: All prompt tokens including cache-miss and cache-hit input tokens.
completion_tokens: Output tokens generated by the model.
total_tokens: Sum of prompt_tokens + completion_tokens.
prompt_tokens_details.cached_tokens: Tokens that were a cache-hit for that call.
cache_creation_input_tokens: Tokens written to the cache, on models that report cache writes this way (for example Anthropic).
prompt_tokens_details.created_cache_tokens: Tokens written to the cache, on models served with vLLM that report cache writes this way.

Cache writes are billed at the cache-write price whichever field reports them; see Cache prices.

Quick Start

Note: OpenAI caching is only available for prompts containing 1024 tokens or more.

Python

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.haimaker.ai/v1"
)

for _ in range(2):
    response = client.chat.completions.create(
        model="openai/gpt-4o",
        messages=[
            {
                "role": "system",
                "content": [
                    {
                        "type": "text",
                        "text": "Here is the full text of a complex legal agreement" * 400,
                    }
                ],
            },
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What are the key terms and conditions in this agreement?",
                    }
                ],
            },
            {
                "role": "assistant",
                "content": "Certainly! the key terms and conditions are the following: the contract is 1 year long for $10/mo",
            },
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What are the key terms and conditions in this agreement?",
                    }
                ],
            },
        ],
        temperature=0.2,
        max_tokens=10,
    )

print("response=", response)
print("response.usage=", response.usage)

# On second call, cached_tokens should be > 0
assert response.usage.prompt_tokens_details.cached_tokens > 0

Anthropic Example

Anthropic charges for cache writes.

Specify the content to cache with "cache_control": {"type": "ephemeral"}.

If you pass that in for any other LLM provider, it will be ignored.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.haimaker.ai/v1"
)

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-6",
    messages=[
        {
            "role": "system",
            "content": [
                {
                    "type": "text",
                    "text": "You are an AI assistant tasked with analyzing legal documents.",
                },
                {
                    "type": "text",
                    "text": "Here is the full text of a complex legal agreement" * 400,
                    "cache_control": {"type": "ephemeral"},
                },
            ],
        },
        {
            "role": "user",
            "content": "what are the key terms and conditions in this agreement?",
        },
    ]
)

print(response.usage)

Deepseek Example

Works the same as OpenAI.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.haimaker.ai/v1"
)

messages_1 = [
    {
        "role": "system",
        "content": "You are a history expert. The user will provide a series of questions, and your answers should be concise and start with `Answer:`",
    },
    {
        "role": "user",
        "content": "In what year did Qin Shi Huang unify the six states?",
    },
    {"role": "assistant", "content": "Answer: 221 BC"},
    {"role": "user", "content": "Who was the founder of the Han Dynasty?"},
    {"role": "assistant", "content": "Answer: Liu Bang"},
    {"role": "user", "content": "Who was the last emperor of the Tang Dynasty?"},
    {"role": "assistant", "content": "Answer: Li Zhu"},
    {
        "role": "user",
        "content": "Who was the founding emperor of the Ming Dynasty?",
    },
    {"role": "assistant", "content": "Answer: Zhu Yuanzhang"},
    {
        "role": "user",
        "content": "Who was the founding emperor of the Qing Dynasty?",
    },
]

messages_2 = [
    {
        "role": "system",
        "content": "You are a history expert. The user will provide a series of questions, and your answers should be concise and start with `Answer:`",
    },
    {
        "role": "user",
        "content": "In what year did Qin Shi Huang unify the six states?",
    },
    {"role": "assistant", "content": "Answer: 221 BC"},
    {"role": "user", "content": "Who was the founder of the Han Dynasty?"},
    {"role": "assistant", "content": "Answer: Liu Bang"},
    {"role": "user", "content": "Who was the last emperor of the Tang Dynasty?"},
    {"role": "assistant", "content": "Answer: Li Zhu"},
    {
        "role": "user",
        "content": "Who was the founding emperor of the Ming Dynasty?",
    },
    {"role": "assistant", "content": "Answer: Zhu Yuanzhang"},
    {"role": "user", "content": "When did the Shang Dynasty fall?"},
]

response_1 = client.chat.completions.create(model="deepseek/deepseek-chat", messages=messages_1)
response_2 = client.chat.completions.create(model="deepseek/deepseek-chat", messages=messages_2)

print(response_2.usage)

Cache prices​

Cache usage​

Quick Start​

Python​

Anthropic Example​

Deepseek Example​