How tool calling works through FreeLLMAPI
- You send the messages and a list of
tools. Each tool has a name, a description and a JSON Schema for its parameters. - The model answers with
tool_callsandfinish_reason: "tool_calls". It does not run anything itself. - Your code runs the function and adds the result as a message with
role: "tool". - You send the conversation again. The model uses the result to write the final answer.
OpenAI-compatible providers, such as Groq, Cerebras, Mistral, OpenRouter and Cloudflare, get the request as it is. For Gemini, FreeLLMAPI translates the tools into Google’s functionDeclarations format and translates the answer back. Your code is the same for every provider.
Some models write a tool call as plain text and not as a real tool call. FreeLLMAPI detects these and changes them into real tool_calls.
Free models with tool calling today
The table shows the strongest free models with tool calling. The data comes from the live catalog. For agents, also check the context size, because tool definitions and results use many tokens.
| Model | Free on | Context | Free limits (best offer) | Tools | Vision |
|---|---|---|---|---|---|
| GLM 5.3 | UnoRouter | 1M | 1 req/min 1 req/min per model | Yes | No |
| Qwen3.8 Max | ACLIDE | - | €20/month shared internal usage credits · no card | Yes | No |
| GLM 5.3 Flash | NVIDIA NIM, UnoRouter | 1M | 40 req/min 40 RPM | Yes | Yes |
| Gemini 3.8 Flash | Google AI Studio | 1M | Recurring provider quota | Yes | Yes |
| Gemini Flash (latest alias) | Google AI Studio | - | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| Qwen3.8 Flash Next | AMD Radeon Cloud | 262K | Recurring daily account allowance | Yes | No |
| Qwen3.8 Flash | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| GPT-5.6 Luna | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Dots3 Note Preview | Kilo Gateway, OpenRouter | 512K | 20 req/min 50 req/day 50 req/day account-wide (1,000 after a $10 purchase) | Yes | Yes |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 req/min 50 req/day Shared rate limit | Yes | No |
| Qwen3.8 27B | ElectronHub, Groq, OVH AI Endpoints | 262K | 30 req/min 250 req/day 70,000 tokens/min ~6M | Yes | Yes |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| Qwen 3.8 27B | Cloudflare Workers AI | 262K | Shared 10k neurons/day | Yes | Yes |
| Gemini 3.5 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
Showing 15 of 146 matching free models. See all models.
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
Step 1: Install FreeLLMAPI and add keys
FreeLLMAPI is open source. It runs on your computer in Docker. Your provider keys stay on your computer.
Install (macOS or Linux)
curl -fsSL https://freellmapi.co/install.sh | bashOpen the dashboard at http://localhost:3001. Add keys from three or more providers on the Keys page. A tool loop sends several requests for one task, so more keys give more room. Copy your unified key from the same page.
Step 2: Run a complete tool loop
curl: the first request
curl http://localhost:3001/v1/chat/completions \
-H "Authorization: Bearer <unified-key>" \
-H "Content-Type: application/json" \
-d '{
"model": "auto",
"messages": [{"role": "user", "content": "What is the weather in Karachi?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}'Python: loop until the model gives a final answer
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3001/v1", api_key="<unified-key>")
def get_weather(city: str) -> dict:
return {"city": city, "temp_c": 32, "condition": "sunny"} # call a real API here
FUNCS = {"get_weather": get_weather}
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
messages = [{"role": "user", "content": "What is the weather in Karachi and Lahore?"}]
for _ in range(5): # stop after 5 rounds
msg = client.chat.completions.create(model="auto", messages=messages, tools=tools).choices[0].message
if not msg.tool_calls:
print(msg.content)
break
messages.append(msg)
for call in msg.tool_calls:
try:
args = json.loads(call.function.arguments or "{}")
result = FUNCS[call.function.name](**args)
except Exception as exc: # bad JSON, unknown tool or wrong arguments
result = {"error": str(exc)}
messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})Tool calling also works with stream=True. You get delta.tool_calls chunks, then a final chunk with finish_reason: "tool_calls". Join the argument chunks before you parse the JSON.
Use a tool to get structured data
Tool calling is also a good way to get data in a fixed shape. Define one tool whose parameters are the fields that you want. Set tool_choice to "required", so that the model must call a tool. Then read the arguments. You do not need to run any function.
Python: extract fields with a required tool call
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3001/v1", api_key="<unified-key>")
schema = {
"type": "function",
"function": {
"name": "save_contact",
"description": "Save one contact found in the text.",
"parameters": {
"type": "object",
"properties": {"name": {"type": "string"}, "email": {"type": "string"}},
"required": ["name", "email"],
},
},
}
resp = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "Reach Sara Khan at [email protected]."}],
tools=[schema],
tool_choice="required",
)
print(json.loads(resp.choices[0].message.tool_calls[0].function.arguments))Validate the arguments with your own code. A model can leave out a required field or send a wrong type. If the check fails, send the error back and ask again.
Pitfalls
- Arguments are a string:
function.argumentsis a JSON string. Parse it, and catch errors. Small models sometimes send JSON that is not valid. - Match the ids: each tool message needs the
tool_call_idof its call. Add the assistant message with thetool_callsbefore the tool messages. - Several calls at once: a model can return more than one tool call in one answer. Run all of them, then send all results.
- Token cost: each tool definition is sent with every request. Twenty large tools can use thousands of tokens and reach a low tokens-per-minute limit fast. Send only the tools that the task needs.
- Clear descriptions: the model chooses a tool from its name and description. Write one short sentence that says what the tool does and when to use it.
- Stop the loop: set a maximum number of rounds. A weak model can call the same tool again and again.
- Models without tools: some free providers do not support tools. For example, AI Horde drops the tools and answers as plain chat.
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| The model answers in text and calls no tool | The model is weak at tools, or the description is not clear | Pin a stronger model from the table, or improve the tool description |
json.JSONDecodeError on arguments | The model sent invalid JSON | Catch the error and send it back as the tool result, so the model can try again |
| Error 400 about a tool message | The tool_call_id does not match, or the assistant message is missing | Add the assistant message first, then one tool message for each call |
| Error 429 during an agent task | The loop sends many large requests | Add more provider keys, and send fewer tools |
| The loop does not stop | The model calls the same tool again | Set a round limit, and return clear results |
Frequently asked questions
Which free LLM APIs support function calling?
Gemini, Groq, Cerebras, Mistral, OpenRouter, Cloudflare and other providers have free models with tool calling. The FreeLLMAPI catalog has 190 free model offers with tools today.
Is tool calling the same as function calling?
Yes. OpenAI first used the name function calling. The current API uses tools and tool_calls. Both names mean that the model asks your code to run a function.
Does Gemini tool calling work with the OpenAI format?
Yes, through FreeLLMAPI. The gateway translates OpenAI tools into Gemini functionDeclarations and translates the answer back into OpenAI tool_calls.
Can I stream tool calls?
Yes. With stream=True you get delta.tool_calls chunks and a final finish_reason of tool_calls. Join the argument chunks before you parse them.
Related guides
- Free LLM API for AI agents
- Free web search API for AI agents
- Best free coding model APIs
- Free LLM API for LangChain
- Free LLM API in Python
Sources
- FreeLLMAPI docs: Tool calling (checked 20 November 2026)
- FreeLLMAPI README: features (checked 20 November 2026)
- OpenAI API reference: Chat Completions tools (checked 20 November 2026)