Guide

Free LLM API for LangChain

LangChain connects to OpenAI-compatible APIs through ChatOpenAI and OpenAIEmbeddings. You change the base URL and the key. This guide points both classes at FreeLLMAPI, which pools 45 free provider tiers behind one key. The code below works in Python and JavaScript and handles rate limits.

Set up the gateway and the packages

Install FreeLLMAPI and add free keys from two or more providers on the dashboard Keys page. Copy the unified key. Your code uses only this key. The provider keys stay in FreeLLMAPI.

Install FreeLLMAPI and the LangChain packages

curl -fsSL https://freellmapi.co/install.sh | bash

# Python
pip install -U langchain langchain-openai

# JavaScript
npm install @langchain/openai @langchain/core zod

Environment variables

export FREELLMAPI_BASE_URL=http://localhost:3001/v1
export FREELLMAPI_API_KEY=<unified-key>

Use model="auto" to let the router select a model. You can also use a routing strategy, for example auto:fast or auto:smart. A named profile such as auto:coding and a fixed model ID from the catalog also work.

To see which models your keys can call now, request GET /v1/models?execution_status=ready. The full list also contains models without a key. A model with the status exhausted has keys, but all of them are at a limit at this moment. That status clears when the cooldowns end.

List the models that can answer now

curl "http://localhost:3001/v1/models?execution_status=ready" \
  -H "Authorization: Bearer <unified-key>"

Chat with ChatOpenAI

In Python, set base_url and api_key. The max_retries value goes to the OpenAI client, which retries 429 and 5xx errors with exponential backoff.

Python: chat and streaming

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="auto",
    base_url=os.environ["FREELLMAPI_BASE_URL"],
    api_key=os.environ["FREELLMAPI_API_KEY"],
    temperature=0.2,
    max_retries=3,
    timeout=60,
    include_response_headers=True,
)

msg = llm.invoke("Explain retrieval-augmented generation in two sentences.")
print(msg.content)
print("Served by:", msg.response_metadata["headers"].get("x-routed-via"))

for chunk in llm.stream("Write a haiku about SQLite."):
    print(chunk.content, end="", flush=True)

In JavaScript, put the base URL in configuration.baseURL. The configuration object goes to the official OpenAI SDK client.

JavaScript: chat and streaming

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "auto",
  apiKey: process.env.FREELLMAPI_API_KEY,
  temperature: 0.2,
  maxRetries: 3,
  configuration: { baseURL: process.env.FREELLMAPI_BASE_URL },
});

const msg = await llm.invoke("Explain retrieval-augmented generation in two sentences.");
console.log(msg.content);

for await (const chunk of await llm.stream("Write a haiku about SQLite.")) {
  process.stdout.write(String(chunk.content));
}

The x-routed-via header shows the provider and model that answered. If the router changed provider, x-fallback-trail shows each failed attempt and the reason.

Tools and agents

FreeLLMAPI passes OpenAI-style tools to each provider and returns tool_calls in OpenAI format. For Gemini, it translates the request into Google's format and back. Use bind_tools for one call, or create_agent for a loop that runs the tools.

Python: bind_tools and create_agent

from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    return f"It is 18 C and cloudy in {city}."

# One call: the model returns the tool call, your code runs it
msg = llm.bind_tools([get_weather]).invoke("What is the weather in Dublin?")
print(msg.tool_calls)

# Agent loop: LangChain runs the tool and sends the result back
agent = create_agent(
    model=llm,
    tools=[get_weather],
    system_prompt="You answer weather questions. Use the tool.",
)
result = agent.invoke({"messages": [{"role": "user", "content": "Weather in Dublin?"}]})
print(result["messages"][-1].content)

JavaScript: bindTools

import { tool } from "@langchain/core/tools";
import { z } from "zod";

const getWeather = tool(
  async ({ city }) => `It is 18 C and cloudy in ${city}.`,
  {
    name: "get_weather",
    description: "Return the current weather for a city.",
    schema: z.object({ city: z.string() }),
  },
);

const reply = await llm.bindTools([getWeather]).invoke("What is the weather in Dublin?");
console.log(reply.tool_calls);

Only some free models support tool calling. With auto, keep tool-calling models in your fallback chain, or make a profile for agents. These free models support tools today:

Free models with tool calling and 128K+ context
ModelFree onContextFree limits (best offer)ToolsVision
GLM 5.3UnoRouter1M1 req/min
1 req/min per model
YesNo
GLM 5.3 FlashNVIDIA NIM, UnoRouter1M40 req/min
40 RPM
YesYes
Gemini 3.8 FlashGoogle AI Studio1MRecurring provider quotaYesYes
Qwen3.8 Flash NextAMD Radeon Cloud262KRecurring daily account allowanceYesNo
Qwen3.8 FlashElectronHub1M$0.25/week shared recurring creditYesYes
Gemini 3.7 FlashGoogle AI Studio1M10 req/min
20 req/day
250,000 tokens/min
~3M
YesYes
GPT-5.6 LunaElectronHub1M$0.25/week shared recurring creditYesYes
Dots3 Note PreviewKilo Gateway, OpenRouter512K20 req/min
50 req/day
50 req/day account-wide (1,000 after a $10 purchase)
YesYes
DeepSeek V4 FlashAMD Radeon Cloud, BazaarLink, ElectronHub1M10 req/min
50 req/day
Shared rate limit
YesNo
Qwen3.8 27BElectronHub, Groq, OVH AI Endpoints262K30 req/min
250 req/day
70,000 tokens/min
~6M
YesYes

Showing 10 of 109 matching free models. See all models.

Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.

Embeddings with OpenAIEmbeddings

By default, OpenAIEmbeddings in Python splits text with an OpenAI tokenizer and sends tokens. Most other providers need raw text. Set check_embedding_ctx_length=False to send text.

Python: embeddings

from langchain_openai import OpenAIEmbeddings

emb = OpenAIEmbeddings(
    model="bge-m3",  # one embedding family, 1024 dimensions
    base_url=os.environ["FREELLMAPI_BASE_URL"],
    api_key=os.environ["FREELLMAPI_API_KEY"],
    check_embedding_ctx_length=False,
)
vectors = emb.embed_documents(["FreeLLMAPI pools free tiers.", "LangChain builds LLM apps."])
print(len(vectors), len(vectors[0]))

JavaScript: embeddings

import { OpenAIEmbeddings } from "@langchain/openai";

const emb = new OpenAIEmbeddings({
  model: "bge-m3",
  apiKey: process.env.FREELLMAPI_API_KEY,
  configuration: { baseURL: process.env.FREELLMAPI_BASE_URL },
});
const vector = await emb.embedQuery("What does FreeLLMAPI do?");
console.log(vector.length);

Embedding failover in FreeLLMAPI never changes the model. It only moves between providers that serve the same family. This keeps all vectors in one vector store compatible. Select a family and keep it. The free embedding models today are:

ModelProviderDimensionsMax inputFree limit
Gemini Embedding
gemini-embedding-001
Google AI Studio30722048100 rpm · 1K req/day
Nemotron Embed VL 1B
nvidia/llama-nemotron-embed-vl-1b-v2
NVIDIA NIM20488192~40 rpm
Nemotron 3 Embed 1B (OR free)
nvidia/nemotron-3-embed-1b:free
OpenRouter20483276820 rpm · shared free daily cap
SEA-LION E5 Embedding 600M
aisingapore/SEA-LION-E5-Embedding-600M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 300M
aisingapore/SEA-LION-ModernBERT-Embedding-300M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 600M
aisingapore/SEA-LION-ModernBERT-Embedding-600M
SEA-LION10248192free · 10 rpm
BGE-M3
@cf/baai/bge-m3
Cloudflare Workers AI1024819210K neurons/day (shared)
EmbeddingGemma 300M
@cf/google/embeddinggemma-300m
Cloudflare Workers AI768204810K neurons/day (shared)
Qwen3 Embedding 0.6B
@cf/qwen/qwen3-embedding-0.6b
Cloudflare Workers AI1024819210K neurons/day (shared)
Cohere Embed v4
embed-v4.0
Cohere15361280001K calls/mo (shared w/ chat)
Gemini Embedding 2
gemini-embedding-2
Google AI Studio30728192free-tier rate limited
Nemotron 3 Embed 1B
nvidia/nemotron-3-embed-1b
NVIDIA NIM20484096~40 rpm
PLaMo Embedding 1B (CF)
@cf/pfnet/plamo-embedding-1b
Cloudflare Workers AI2048409610K neurons/day (shared)
BGE Small EN v1.5 (CF)
@cf/baai/bge-small-en-v1.5
Cloudflare Workers AI38451210K neurons/day (shared)
BGE Base EN v1.5 (CF)
@cf/baai/bge-base-en-v1.5
Cloudflare Workers AI76851210K neurons/day (shared)
BGE Large EN v1.5 (CF)
@cf/baai/bge-large-en-v1.5
Cloudflare Workers AI102451210K neurons/day (shared)
Mistral Embed
mistral-embed
Mistral10248192free mode · monthly cap (shared)
Codestral Embed
codestral-embed
Mistral15368192free mode · monthly cap (shared)
Llama Text Embed v2 (Pinecone)
llama-text-embed-v2
Pinecone102420485M tokens/mo per model
Multilingual E5 Large (Pinecone)
multilingual-e5-large
Pinecone10245075M tokens/mo per model
Nemotron 3 Embed 1B (Speka)
nvidia/nemotron-3-embed-1b
Speka20488192$1/month shared across Speka models
OpenAI: Text Embedding 3 Small (ElectronHub)
text-embedding-3-small
ElectronHub15368192$0.25/week shared free credits
OpenAI: Text Embedding 3 Large (ElectronHub)
text-embedding-3-large
ElectronHub30728192$0.25/week shared free credits
Nemotron 3 Embed 8B (Inferbase)
nvidia-nemotron-3-embed-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 0.6B (Inferbase)
qwen3-embedding-0-6b
Inferbase102481925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 4B (Inferbase)
qwen3-embedding-4b
Inferbase256081925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 8B (Inferbase)
qwen3-embedding-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
LFM2.5 Embedding 350M (LLMTR)
liquid/lfm-2.5-embedding-350m-free
LLMTR1024512Ongoing zero-price route; provider terms apply
BGE-M3 (SEA-LION)
BAAI/bge-m3
SEA-LION10248192free · 10 rpm
Cohere Embed English v3
embed-english-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Qwen3 Embedding 8B (OVH)
Qwen3-Embedding-8B
OVH AI Endpoints409632768free · anonymous 2 req/min per IP
BGE-M3 (OVH)
bge-m3
OVH AI Endpoints10248192free · anonymous 2 req/min per IP
BGE Multilingual Gemma2 (OVH)
bge-multilingual-gemma2
OVH AI Endpoints35848192free · anonymous 2 req/min per IP
Mistral: Codestral Embed (ElectronHub)
codestral-embed
ElectronHub15368192$0.25/week shared free credits
Mistral: Mistral Embed (ElectronHub)
mistral-embed
ElectronHub10248192$0.25/week shared free credits
Google: Gemini Embedding 2 (ElectronHub)
gemini-embedding-2
ElectronHub30728192$0.25/week shared free credits
Google: Gemini Embedding 001 (ElectronHub)
gemini-embedding-001
ElectronHub30722048$0.25/week shared free credits
OpenAI: Text Embedding Ada 002 (ElectronHub)
text-embedding-ada-002
ElectronHub15368192$0.25/week shared free credits
Cohere Embed English Light v3
embed-english-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)
Cohere Embed Multilingual v3
embed-multilingual-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Cohere Embed Multilingual Light v3
embed-multilingual-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)

Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.

Structured output, sessions and model choice

with_structured_output returns a Pydantic object in place of text. By default, LangChain uses function calling for this. That method works on every free model that supports tools.

Python: structured output

from pydantic import BaseModel, Field

class Ticket(BaseModel):
    category: str = Field(description="billing, bug or other")
    urgent: bool

classifier = llm.with_structured_output(Ticket)
ticket = classifier.invoke("The app crashes when I pay. Please fix it today.")
print(ticket.category, ticket.urgent)

Do not use method="json_schema" with auto. Not every free provider supports strict JSON schema output, and the router can select a different provider for each request.

FreeLLMAPI keeps a conversation on one model for 30 minutes. It finds the conversation from the X-Session-Id header. If there is no header, it uses a hash of the first user message. For a chat app with many users, send one session ID per user:

Python: one session per user

def llm_for(user_id: str) -> ChatOpenAI:
    return ChatOpenAI(
        model="auto",
        base_url=os.environ["FREELLMAPI_BASE_URL"],
        api_key=os.environ["FREELLMAPI_API_KEY"],
        default_headers={"X-Session-Id": f"user-{user_id}"},
        max_retries=3,
    )

Select the model string for each part of your app:

Task in your appModel stringReason
Chat answers for usersautoFollows the chain that you set on the dashboard
Agent with toolsauto:<profile>A profile that has only tool-calling models
Classification, routing, short summariesauto:fastRanks models by measured speed
Hard reasoning stepsauto:smartRanks models by intelligence
Tests that must be repeatableA fixed model IDSame model each time, but it stops at that model's limit

Free tiers have no SLA. Latency changes during the day, and the strongest models reach their daily limits first. Test your chains with the models that your chain will actually use.

Rate limits and retries

FreeLLMAPI tries the next provider when one returns 429. Your code sees a 429 only when every key in the chain is at its limit. Then the response has a Retry-After header. Many daily limits reset at midnight UTC.

For batch jobs, add a second retry layer with with_retry. It retries the whole runnable with exponential backoff and jitter.

Python: retry a chain on rate limits

import openai
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "Summarize the text in one sentence."),
    ("user", "{text}"),
])

chain = (prompt | llm).with_retry(
    retry_if_exception_type=(openai.RateLimitError, openai.APITimeoutError),
    wait_exponential_jitter=True,
    stop_after_attempt=5,
)

# batch() runs requests in parallel; max_concurrency keeps the rate low
results = chain.batch(
    [{"text": t} for t in ["First document.", "Second document."]],
    config={"max_concurrency": 4},
)
ProblemCauseFix
openai.AuthenticationErrorWrong unified keyCopy the key again from the Keys page
openai.NotFoundError on every callThe base URL has no /v1Use http://localhost:3001/v1
openai.RateLimitErrorAll keys in the chain are at their limitAdd providers, reduce max_concurrency or wait for Retry-After
Embedding error about tokens or input typeLangChain sends tokens, not textSet check_embedding_ctx_length=False
tool_calls is emptyThe model does not support toolsUse a model from the tools table or a tools profile
Reasoning text is missingChatOpenAI keeps only standard OpenAI fieldsRead the answer in content; extra provider fields are not kept
Use every one of these through one key. FreeLLMAPI is an open-source, self-hosted router that puts all these free tiers behind a single OpenAI-compatible endpoint and fails over when one is rate-limited. Browse the catalog or go live.

Frequently asked questions

Can I use a free LLM API with LangChain?

Yes. Any OpenAI-compatible API works with ChatOpenAI. Set base_url to http://localhost:3001/v1 and api_key to your FreeLLMAPI unified key. The router uses free provider tiers.

How do I set a custom base URL in LangChain ChatOpenAI?

In Python, pass base_url to ChatOpenAI. In JavaScript, pass configuration: { baseURL } to the constructor. The OPENAI_BASE_URL environment variable also works in Python.

Do LangChain agents work with free models?

Yes, if the model supports tool calling. Use create_agent with a ChatOpenAI model and keep tool-calling models in your FreeLLMAPI chain.

Why does OpenAIEmbeddings fail with a non-OpenAI provider?

By default it sends token IDs. Most other providers need raw text. Set check_embedding_ctx_length=False in Python.

How do I handle 429 errors in LangChain?

Set max_retries on ChatOpenAI and add with_retry to the chain for batch jobs. FreeLLMAPI already moves to the next provider before your code sees a 429.

Sources

All guides · Browse all free models

Keep reading

All posts →
Guide

Best free LLM APIs 2026

Ten genuinely free LLM APIs worth using — ranked, with real limits.

Guide

Unlimited free AI tokens?

No free AI API is unlimited. The closest real thing: 10.1 billion free tokens a month behind one key.

Guide

How to get a free LLM API key

Get a working key in minutes — no credit card required.