API Compatibility

Your OpenAI code. Your Anthropic code. Now with EU sovereign inference.

Point your OpenAI or Anthropic SDK at Infercom. Function calling, structured output and streaming work as before, and the inference runs in Munich.

Switch in 5 Minutes

Change one line of code. Keep everything else.

01

Sign up & get API key

Create your account at cloud.infercom.ai.

02

Change base URL

Point your existing OpenAI or Anthropic SDK to api.infercom.ai

03

Choose your model

Pick a model: gpt-oss-120b for reasoning and tools, Gemma 4 31B for text and images.

OpenAI SDK
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-api-key"
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="")
Responses API
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-api-key"
)

# Responses API - built for agentic workflows
response = client.responses.create(
    model="gpt-oss-120b",
    input="Hello",
    reasoning={"effort": "high"}
)

print(response.output_text)
Anthropic SDK
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.infercom.ai",
    api_key="your-api-key"
)

response = client.messages.create(
    model="gpt-oss-120b",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}]
)

print(response.content[0].text)
curl
curl https://api.infercom.ai/v1/chat/completions \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Key features

Function calling

Auto, required or a named tool, also while streaming.

Structured output

JSON mode and JSON schema with strict.

Reasoning

Set reasoning_effort on gpt-oss-120b. The reasoning comes back in its own field.

Vision

Images on Gemma 4 31B, sent as base64.

Embeddings and speech-to-text

E5-Mistral 7B and Whisper Large v3 on the same API key.

Full compatibility reference

Choose Your SDK

Full compatibility with OpenAI and Anthropic client libraries

OpenAI SDK

OpenAI SDK Compatible

Drop-in replacement for OpenAI. Works with your existing code, tools, and frameworks.

  • Chat Completions and Responses API
  • Works with LangChain, LlamaIndex and LiteLLM
  • Python, JavaScript, TypeScript, REST
  • Streaming, function calling, JSON mode

View OpenAI compatibility docs

Anthropic SDK

Anthropic Messages API

The same endpoint speaks the Anthropic Messages API. Set base_url to https://api.infercom.ai and keep your messages, system prompts, streaming and tool use. gpt-oss-120b returns its reasoning as thinking blocks. Gemma 4 31B reads images.

  • Standard /v1/messages endpoint
  • Works with existing Anthropic code
  • Same authentication patterns

View Anthropic compatibility docs

More Examples

Works with your favorite languages and frameworks

JavaScript / TypeScript
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.infercom.ai/v1',
  apiKey: 'your-api-key',
});

const response = await client.chat.completions.create({
  model: 'gpt-oss-120b',
  messages: [{ role: 'user', content: 'Hello' }],
});

console.log(response.choices[0].message.content);
LangChain
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-api-key",
    model="gpt-oss-120b"
)

response = llm.invoke("Explain quantum computing")
print(response.content)
LiteLLM
from litellm import completion

response = completion(
    model="openai/gpt-oss-120b",
    api_base="https://api.infercom.ai/v1",
    api_key="your-api-key",
    messages=[{"role": "user", "content": "Hello"}]
)

print(response.choices[0].message.content)
LlamaIndex
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="gpt-oss-120b",
    api_base="https://api.infercom.ai/v1",
    api_key="your-api-key",
    is_chat_model=True
)

response = llm.complete("Explain quantum computing")
print(response)

View full quickstart guide

Up to 10x faster than GPU inference

SambaNova's dataflow architecture delivers up to 10x faster inference than GPU-based providers. Every response reports its time to first token and tokens per second.

713tok/s · gpt-oss-120b
Up to 10xfaster than GPU inference

Output Throughput, p50, 10K input / 1K output, single request. Last measured: July 2026.

See performance benchmarks

Supported Endpoints

  • /v1/chat/completions

    Chat completions with streaming support

  • /v1/responses

    Responses API for agentic workflows - tools, reasoning, streaming

  • /v1/messages

    Anthropic Messages API format

  • /v1/models

    List available models and metadata

  • /v1/embeddings

    Text embeddings (selected models)

  • /v1/audio/transcriptions

    Speech-to-text with Whisper Large v3

Full API reference

EU Sovereignty Included

OpenAI-compatible API, European jurisdiction.

  • EU sovereign models run in a certified data center in Munich
  • No US CLOUD Act exposure
  • GDPR compliant, ISO 27001 certified
  • No persistent storage of prompts or outputs

Learn about EU sovereignty

ISO 27001Certified
GDPRCompliant
AI ActWe support your work
MunichDatacenter

FAQ

Questions before you switch

Do I have to change my code?

No. Set the base URL to https://api.infercom.ai/v1 (OpenAI SDK) or https://api.infercom.ai (Anthropic SDK), use your Infercom API key and pick a model. The rest of your code stays the same.

Does function calling work?

Yes, on gpt-oss-120b and Gemma 4 31B: tool_choice auto, required or a named function, with and without streaming. Send the tool result back as a tool message, as you do with OpenAI.

Can I get JSON that matches my schema?

Yes. Use response_format with json_schema and strict: true, or json_object for free-form JSON. To constrain a single field, add a pattern to it in the schema.

Where do parameters such as top_k go?

top_k and repetition_penalty are not part of the OpenAI SDK. In Python, send them in extra_body. On the wire they sit in the normal JSON body.

View OpenAI compatibility docs

gpt-oss-120b sometimes returns no text. Why?

gpt-oss-120b reasons before it answers, and the reasoning counts inside max_tokens. If the budget runs out during reasoning, finish_reason is "length" and the answer is empty. Give it enough max_tokens and check finish_reason. reasoning_effort sets how much it reasons.

Does the Anthropic SDK work?

Yes. Messages, system prompts, streaming, tool use and token counting work on the same endpoint. gpt-oss-120b returns its reasoning as thinking blocks, and Gemma 4 31B reads images.

View Anthropic compatibility docs

Can I send images?

Yes, to Gemma 4 31B, as base64 JPEG, PNG or WebP, through both SDKs. Each image uses a fixed budget of about 260 tokens. For fine detail, split a large image into parts and send them as several images.

Does it work with LangChain, LlamaIndex or LiteLLM?

Yes. Use ChatOpenAI in LangChain, OpenAILike in LlamaIndex and the openai/ model prefix in LiteLLM, each with our base URL. Tools that call models through LiteLLM, such as CrewAI, work the same way.

What should I check when I switch?

Most code runs unchanged. Three things are worth a look: set temperature on every request (the recommended value per model is in the docs), use repetition_penalty instead of frequency_penalty or presence_penalty, and send images as base64. The Responses API is stateless, so send the conversation with each request.

View OpenAI compatibility docs

I get a 429. What does it mean?

You reached the rate limit of your tier, or the queue is full at a peak. The limits per tier are in the docs. If the message talks about "deployments", it comes from a router in front of us, not from our API.

View rate limits in the docs

Do you store my prompts?

No. We store only metadata such as model, token counts and timing. Prompts and outputs are never stored persistently.

Visit the Trust Center

Ready to Build the Future of AI in Europe?

Join forward-thinking organizations deploying sovereign AI with world-class performance