What an agent needs from a free API
| Need | Why | What FreeLLMAPI does |
|---|---|---|
| Tool calling | The agent selects and runs tools | Passes OpenAI-style tools to each provider. Converts plain-text tool calls into real tool_calls. |
| Many requests | One task can take 20 to 100 steps | Counts each key's quota and moves to the next key or provider on 429 |
| Large context | Tool results and history grow each step | Routes to models from your chain. You can make a long-context profile. |
| Stable behaviour | A change of model in the middle of a task can confuse the agent | Keeps a session on one model for 30 minutes. Optional context handoff when a change happens. |
| Visibility | You must know which model failed | X-Routed-Via and X-Fallback-Trail headers on each response |
Free tiers have limits. Failover spreads the load over your keys, but the total is still the sum of the free limits. Add keys from four or more providers for agent work.
| Model | Free on | Context | Free limits (best offer) | Tools | Vision |
|---|---|---|---|---|---|
| GLM 5.3 | UnoRouter | 1M | 1 req/min 1 req/min per model | Yes | No |
| GLM 5.3 Flash | NVIDIA NIM, UnoRouter | 1M | 40 req/min 40 RPM | Yes | Yes |
| Gemini 3.8 Flash | Google AI Studio | 1M | Recurring provider quota | Yes | Yes |
| Qwen3.8 Flash Next | AMD Radeon Cloud | 262K | Recurring daily account allowance | Yes | No |
| Qwen3.8 Flash | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| GPT-5.6 Luna | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Dots3 Note Preview | Kilo Gateway, OpenRouter | 512K | 20 req/min 50 req/day 50 req/day account-wide (1,000 after a $10 purchase) | Yes | Yes |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 req/min 50 req/day Shared rate limit | Yes | No |
| Qwen3.8 27B | ElectronHub, Groq, OVH AI Endpoints | 262K | 30 req/min 250 req/day 70,000 tokens/min ~6M | Yes | Yes |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| Qwen 3.8 27B | Cloudflare Workers AI | 262K | Shared 10k neurons/day | Yes | Yes |
Showing 12 of 109 matching free models. See all models.
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
A complete agent loop in Python
The loop below uses the official OpenAI SDK. It sends the tools, runs each tool call, sends the result back and stops when the model gives a final answer. It also stops after a maximum number of steps.
agent.py
import json
import os
import time
import openai
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:3001/v1",
api_key=os.environ["FREELLMAPI_API_KEY"], # <unified-key>
max_retries=3,
)
def get_weather(city: str) -> dict:
return {"city": city, "temp_c": 18, "sky": "cloudy"}
TOOLS = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
FUNCTIONS = {"get_weather": get_weather}
def call_model(messages, session_id):
for attempt in range(5):
try:
return client.chat.completions.create(
model="auto:smart",
messages=messages,
tools=TOOLS,
extra_headers={"X-Session-Id": session_id},
)
except openai.RateLimitError as err:
header = err.response.headers.get("retry-after", "")
wait = int(header) if header.isdigit() else 2 ** attempt
time.sleep(min(wait, 60))
raise RuntimeError("All free providers are at their limit")
def run(task: str, session_id: str, max_steps: int = 10) -> str:
messages = [
{"role": "system", "content": "Use the tools when they help. Be brief."},
{"role": "user", "content": task},
]
for _ in range(max_steps):
msg = call_model(messages, session_id).choices[0].message
messages.append(msg)
if not msg.tool_calls:
return msg.content
for call in msg.tool_calls:
fn = FUNCTIONS.get(call.function.name)
try:
args = json.loads(call.function.arguments or "{}")
result = fn(**args) if fn else {"error": "unknown tool"}
except Exception as exc: # return tool errors to the model
result = {"error": str(exc)}
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(result),
})
return "Stopped: too many steps."
print(run("Should I take an umbrella in Dublin today?", session_id="task-001"))- X-Session-Id: keeps all steps of one task on the same model while that model has quota.
- Retry-After: when every key is at its limit, the gateway returns 429 with this header. The loop waits and tries again.
- Tool errors: the loop sends errors back to the model as tool results. The model can then correct its call.
Route each step to the correct model
Agent steps are not equal. A planning step needs a strong model. A step that extracts one field needs a fast model. Use a different model string for each step type. You do not need to change the dashboard.
| Model string | Ranks models by | Agent use |
|---|---|---|
auto:smart | Intelligence | Planning, tool selection, final answers |
auto:fast | Measured speed and time to first byte | Extraction, classification, short summaries of tool output |
auto:reliable | Recent success rate | Scheduled agents that must finish |
auto:balanced | Reliability first, then speed and intelligence | General default |
auto:<profile> | Your named fallback chain | A chain with only tool-calling models |
The auto:* strategies rank every enabled model and ignore your chain order. A request with tools still needs a model with tool calling. If some enabled models do not support tools well, make a profile with only tool-calling models. Then use auto:<profile> for tool steps. An unknown profile name returns error 400.
The header X-FreeLLM-Task-Type: code moves the routing weight from speed to intelligence for one request. It is a soft preference. It does not change the capability checks or the fallback chain.
Long agent sessions and model changes
During a long task, a model can reach its daily limit. The router then moves to another model. The new model gets the full message history, but it does not know that another model did the earlier steps.
Turn on context handoff to add one short system message when the model changes. The message tells the new model to continue the task and to keep the earlier tool results.
FreeLLMAPI .env
FREELLMAPI_CONTEXT_HANDOFF=on_model_switch- The handoff message is added only when the selected model changes for a session.
- Sessions are kept in memory for 3 hours. Nothing is written to disk.
- The session key is the
X-Session-Idheader. Without the header, it is a hash of the first user message.
Use the router as an MCP server
FreeLLMAPI is also an MCP server on /mcp. An agent that speaks MCP can send a text task to a FreeLLMAPI model. It can also read the gateway state during a session: usable models and their parameters, provider health, usage and cache statistics, and the routing strategy.
The MCP surface is off on fresh installs. Turn it on in the dashboard on the Keys page, under Agent compatibility. While it is off, /mcp returns 403.
Add the MCP server to Claude Code
claude mcp add --transport http freellmapi http://localhost:3001/mcp \
--header "Authorization: Bearer <unified-key>"Any MCP client with Streamable HTTP works the same way. Point it at /mcp and send the unified key as a Bearer token. Useful tools include healthcheck, list_models and ask_freellmapi. The ask_freellmapi tool runs a task through the same chat route as the API, with routing and fallback.
For coding agents, FreeLLMAPI has setup commands that write the client configuration for you. Examples are npx freellmapi setup-claude, setup-codex, setup-cline, setup-openclaw and setup-hermes. Each command supports --dry-run and makes a backup first.
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| The model answers in text and calls no tool | The model has weak tool calling | Use auto:<profile> with tool-calling models only |
| The agent restarts the task in the middle | A limit forced a change of model | Send X-Session-Id and turn on context handoff |
| Error 429 after many steps | All keys in the chain are at their limit | Add providers, use auto:fast for small steps and wait for Retry-After |
Error 400 with auto:name | The profile does not exist | Check the profile name on the dashboard |
/mcp returns 403 | The MCP surface is off | Turn it on under Agent compatibility on the Keys page |
| Context length errors late in a task | The history is larger than the model's window | Summarize old tool results or use a long-context profile |
Frequently asked questions
Is there a free LLM API for AI agents?
Yes. Several providers have free tiers with tool calling. FreeLLMAPI combines them behind one OpenAI-compatible key and moves to the next provider when a limit stops a request.
Which free LLM is best for tool calling?
Use a model with tool calling and a context window of 128K tokens or more. The table on this page shows the strongest free models that meet both rules today.
Can I use FreeLLMAPI as an MCP server?
Yes. Turn on the MCP surface on the Keys page under Agent compatibility. Then connect your MCP client to /mcp with the unified key as a Bearer token.
How do I stop an agent from failing on rate limits?
Add keys from more providers, send small steps to auto:fast, retry on 429 with the Retry-After header and set a maximum number of steps.
What happens when the model changes in the middle of an agent task?
The new model gets the full message history. With context handoff on, it also gets a short note that tells it to continue the task.
Related guides
- Free LLM API with function calling
- Free LLM API in Python
- Free LLM API for LangChain
- Free LLM API for n8n
- How to use Claude Code for free
- Free LLM API rate limits
Sources
- FreeLLMAPI docs: REST API (tool calling, routing strategies, response headers) (checked 20 November 2026)
- FreeLLMAPI docs: clients, MCP server and context handoff (checked 20 November 2026)
- OpenAI Python SDK: errors and retries (checked 20 November 2026)
- Claude Code docs: MCP (checked 20 November 2026)