Install the packages
Install FreeLLMAPI and add free keys from two or more providers on the dashboard Keys page. Copy the unified key. Then install the LlamaIndex packages.
Install
curl -fsSL https://freellmapi.co/install.sh | bash
pip install -U llama-index-core \
llama-index-llms-openai-like \
llama-index-embeddings-openai-like
export FREELLMAPI_BASE_URL=http://localhost:3001/v1
export FREELLMAPI_API_KEY=<unified-key>Do not use the plain OpenAI class from llama-index-llms-openai with model names that OpenAI does not know. That class checks model names and context sizes against a list of OpenAI models. OpenAILike does not, so you set these values yourself.
Configure the LLM and the embedding model
settings.py
import os
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
BASE = os.environ["FREELLMAPI_BASE_URL"]
KEY = os.environ["FREELLMAPI_API_KEY"]
Settings.llm = OpenAILike(
model="auto",
api_base=BASE,
api_key=KEY,
is_chat_model=True, # use /v1/chat/completions
is_function_calling_model=True, # needed for FunctionAgent
context_window=32000, # the smallest window in your chain
max_retries=5,
timeout=90,
)
Settings.embed_model = OpenAILikeEmbedding(
model_name="bge-m3", # one family, 1024 dimensions
api_base=BASE,
api_key=KEY,
embed_batch_size=16,
max_retries=10,
)- is_chat_model=True: without it, LlamaIndex sends requests to the legacy completions endpoint.
- context_window: LlamaIndex uses this value to decide how many retrieved chunks fit in the prompt. With
auto, the router can select different models. Use the smallest context window in your chain. - is_function_calling_model=True: set it only when your chain has tool-calling models.
- embed_batch_size: each batch is one request. Larger batches use fewer requests per minute.
A RAG prompt carries several retrieved chunks, so the chat model needs room for them. Make a fallback chain profile with long-context models for RAG. Then set model="auto:<profile>" and raise context_window to the smallest window in that profile. These free models have the largest context windows today:
| Model | Free on | Context | Free limits (best offer) | Tools | Vision |
|---|---|---|---|---|---|
| OpenAI: GPT-6 Luna | ElectronHub | 1.1M | $0.25/week shared free credits | No | No |
| Gemini 3.8 Flash | APINex, Google AI Studio | 1M | 5 req/min 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | Yes | Yes |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 req/min 50 req/day Shared rate limit | Yes | No |
| DeepSeek V4 Flash 0731 | ElectronHub, Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | No | No |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| GLM-5.2 | ModelScope, Router9 | 1M | 100 req/day Daily login: 200 credits + 50 for linked Alibaba Cloud; 24h expiry; shared; noncommercial experience | No | No |
| Gemini 3.5 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
Showing 8 of 115 matching free models. See all models.
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
To keep one user on one model, send an X-Session-Id header. OpenAILike accepts default_headers, for example default_headers={"X-Session-Id": "user-42"}. FreeLLMAPI keeps a session on the same model for 30 minutes. This gives answers in the same style across one chat.
Build and query a RAG index
The script reads files from a folder, makes embeddings, saves the index and answers a question. Run it once to build the index. Later runs load the saved index and do not embed the files again.
rag.py
import os
import settings # noqa: F401 (sets Settings.llm and Settings.embed_model)
from llama_index.core import (
SimpleDirectoryReader,
StorageContext,
VectorStoreIndex,
load_index_from_storage,
)
PERSIST = "./storage"
def load_index() -> VectorStoreIndex:
if os.path.isdir(PERSIST):
return load_index_from_storage(StorageContext.from_defaults(persist_dir=PERSIST))
docs = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(docs, show_progress=True)
index.storage_context.persist(persist_dir=PERSIST)
return index
if __name__ == "__main__":
query_engine = load_index().as_query_engine(similarity_top_k=4, streaming=True)
response = query_engine.query("How do I rotate the API keys?")
response.print_response_stream()
for node in response.source_nodes:
print("\nSource:", node.metadata.get("file_name"), round(node.score or 0, 3))./storage and build the index again.These embedding models are free in today's catalog. The dimensions column shows the vector size of each family.
| Model | Provider | Dimensions | Max input | Free limit |
|---|---|---|---|---|
Gemini Embeddinggemini-embedding-001 | Google AI Studio | 3072 | 2048 | 100 rpm · 1K req/day |
Nemotron Embed VL 1Bnvidia/llama-nemotron-embed-vl-1b-v2 | NVIDIA NIM | 2048 | 8192 | ~40 rpm |
Nemotron 3 Embed 1B (OR free)nvidia/nemotron-3-embed-1b:free | OpenRouter | 2048 | 32768 | 20 rpm · shared free daily cap |
SEA-LION E5 Embedding 600Maisingapore/SEA-LION-E5-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 300Maisingapore/SEA-LION-ModernBERT-Embedding-300M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 600Maisingapore/SEA-LION-ModernBERT-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
BGE-M3@cf/baai/bge-m3 | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
EmbeddingGemma 300M@cf/google/embeddinggemma-300m | Cloudflare Workers AI | 768 | 2048 | 10K neurons/day (shared) |
Qwen3 Embedding 0.6B@cf/qwen/qwen3-embedding-0.6b | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
Cohere Embed v4embed-v4.0 | Cohere | 1536 | 128000 | 1K calls/mo (shared w/ chat) |
Gemini Embedding 2gemini-embedding-2 | Google AI Studio | 3072 | 8192 | free-tier rate limited |
Nemotron 3 Embed 1Bnvidia/nemotron-3-embed-1b | NVIDIA NIM | 2048 | 4096 | ~40 rpm |
PLaMo Embedding 1B (CF)@cf/pfnet/plamo-embedding-1b | Cloudflare Workers AI | 2048 | 4096 | 10K neurons/day (shared) |
BGE Small EN v1.5 (CF)@cf/baai/bge-small-en-v1.5 | Cloudflare Workers AI | 384 | 512 | 10K neurons/day (shared) |
BGE Base EN v1.5 (CF)@cf/baai/bge-base-en-v1.5 | Cloudflare Workers AI | 768 | 512 | 10K neurons/day (shared) |
BGE Large EN v1.5 (CF)@cf/baai/bge-large-en-v1.5 | Cloudflare Workers AI | 1024 | 512 | 10K neurons/day (shared) |
Mistral Embedmistral-embed | Mistral | 1024 | 8192 | free mode · monthly cap (shared) |
Codestral Embedcodestral-embed | Mistral | 1536 | 8192 | free mode · monthly cap (shared) |
Llama Text Embed v2 (Pinecone)llama-text-embed-v2 | Pinecone | 1024 | 2048 | 5M tokens/mo per model |
Multilingual E5 Large (Pinecone)multilingual-e5-large | Pinecone | 1024 | 507 | 5M tokens/mo per model |
Nemotron 3 Embed 1B (Speka)nvidia/nemotron-3-embed-1b | Speka | 2048 | 8192 | $1/month shared across Speka models |
OpenAI: Text Embedding 3 Small (ElectronHub)text-embedding-3-small | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
OpenAI: Text Embedding 3 Large (ElectronHub)text-embedding-3-large | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Nemotron 3 Embed 8B (Inferbase)nvidia-nemotron-3-embed-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 0.6B (Inferbase)qwen3-embedding-0-6b | Inferbase | 1024 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 4B (Inferbase)qwen3-embedding-4b | Inferbase | 2560 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 8B (Inferbase)qwen3-embedding-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
LFM2.5 Embedding 350M (LLMTR)liquid/lfm-2.5-embedding-350m-free | LLMTR | 1024 | 512 | Ongoing zero-price route; provider terms apply |
BGE-M3 (SEA-LION)BAAI/bge-m3 | SEA-LION | 1024 | 8192 | free · 10 rpm |
Cohere Embed English v3embed-english-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Qwen3 Embedding 8B (OVH)Qwen3-Embedding-8B | OVH AI Endpoints | 4096 | 32768 | free · anonymous 2 req/min per IP |
BGE-M3 (OVH)bge-m3 | OVH AI Endpoints | 1024 | 8192 | free · anonymous 2 req/min per IP |
BGE Multilingual Gemma2 (OVH)bge-multilingual-gemma2 | OVH AI Endpoints | 3584 | 8192 | free · anonymous 2 req/min per IP |
Mistral: Codestral Embed (ElectronHub)codestral-embed | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Mistral: Mistral Embed (ElectronHub)mistral-embed | ElectronHub | 1024 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 2 (ElectronHub)gemini-embedding-2 | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 001 (ElectronHub)gemini-embedding-001 | ElectronHub | 3072 | 2048 | $0.25/week shared free credits |
OpenAI: Text Embedding Ada 002 (ElectronHub)text-embedding-ada-002 | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Cohere Embed English Light v3embed-english-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual v3embed-multilingual-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual Light v3embed-multilingual-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
Add an agent with FunctionAgent
FunctionAgent uses tool calling. You can give it the query engine as a tool and add your own Python functions.
agent.py
import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool
from llama_index.core import Settings
from rag import load_index
docs_tool = QueryEngineTool.from_defaults(
query_engine=load_index().as_query_engine(similarity_top_k=4),
name="project_docs",
description="Answers questions about the project documentation.",
)
def open_tickets(team: str) -> int:
"""Return the number of open tickets for a team."""
return {"billing": 4, "platform": 11}.get(team, 0)
agent = FunctionAgent(
tools=[docs_tool, open_tickets],
llm=Settings.llm,
system_prompt="Use the tools. Cite the document name in each answer.",
)
async def main():
result = await agent.run("How many platform tickets are open, and how do we rotate keys?")
print(result)
asyncio.run(main())For agent work, select models with tool calling. Make a fallback chain profile with only these models, and use auto:<profile> as the model name.
| Model | Free on | Context | Free limits (best offer) | Tools | Vision |
|---|---|---|---|---|---|
| GLM 5.3 | UnoRouter | 1M | 1 req/min 1 req/min per model | Yes | No |
| GLM 5.3 Flash | NVIDIA NIM, UnoRouter | 1M | 40 req/min 40 RPM | Yes | Yes |
| Gemini 3.8 Flash | Google AI Studio | 1M | Recurring provider quota | Yes | Yes |
| Qwen3.8 Flash Next | AMD Radeon Cloud | 262K | Recurring daily account allowance | Yes | No |
| Qwen3.8 Flash | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| GPT-5.6 Luna | ElectronHub | 1M | $0.25/week shared recurring credit | Yes | Yes |
| Dots3 Note Preview | Kilo Gateway, OpenRouter | 512K | 20 req/min 50 req/day 50 req/day account-wide (1,000 after a $10 purchase) | Yes | Yes |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 req/min 50 req/day Shared rate limit | Yes | No |
| Qwen3.8 27B | ElectronHub, Groq, OVH AI Endpoints | 262K | 30 req/min 250 req/day 70,000 tokens/min ~6M | Yes | Yes |
Showing 10 of 109 matching free models. See all models.
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
Keep ingestion inside free limits
Ingestion sends many embedding requests in a short time. Free embedding tiers have low limits. For example, some allow 10 requests per minute and others count calls per month. Plan the first ingestion before you start it.
- Use large chunks. Fewer chunks mean fewer vectors and fewer requests. A chunk size near 1,024 tokens is a good start.
- Use batches. Set
embed_batch_sizeto 16 or more. One request then carries many chunks. - Save the index. Persist the index after the first run. Do not embed the same files again.
- Keep retries on.
OpenAILikeEmbeddingretries up to 10 times by default. Keep this value for long ingestion jobs. - Ingest in parts. For thousands of files, ingest one folder at a time.
Queries are cheaper than ingestion. Each query uses one embedding request and one chat request. A chat request with four chunks of 1,024 tokens sends about 5,000 tokens. Free tiers with low tokens-per-minute limits can reject this. FreeLLMAPI then tries the next provider.
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
Unknown model 'auto' | You used the OpenAI class | Use OpenAILike |
Requests go to /v1/completions | is_chat_model is not set | Set is_chat_model=True |
| The agent does not call tools | is_function_calling_model is off, or the model has no tool calling | Set it to True and use a tools profile |
| Error 429 during ingestion | The embedding family is at its limit on all providers | Use larger batches, wait for Retry-After and run again |
| Bad answers after a change of embedding model | Vectors from two families in one index | Delete the saved index and build it again |
| Context length error | context_window is larger than the model that answered | Set a smaller value or use a long-context profile |
To see which provider answered a request, use the FreeLLMAPI dashboard. The Analytics page shows requests, latency and errors for each provider and model. Each response also has an X-Routed-Via header. If the router changed provider, the X-Fallback-Trail header names each failed attempt and the reason.
Frequently asked questions
Can I use LlamaIndex with a free LLM?
Yes. Use OpenAILike with api_base set to an OpenAI-compatible URL. FreeLLMAPI gives one URL for many free provider tiers. Each tier has its own limits.
What is the difference between OpenAI and OpenAILike in LlamaIndex?
The OpenAI class checks model names against OpenAI models. OpenAILike accepts any model name. You set the context window and the chat and function-calling flags yourself.
Which free embedding model is best for LlamaIndex?
Select a family with enough free requests for your ingestion and keep it for the whole index. The bge-m3 family is free on several providers, so failover can move between them.
Does FunctionAgent work with free models?
Yes, if the model supports tool calling and you set is_function_calling_model=True. Use a fallback chain profile that has only tool-calling models.
How do I avoid rate limits when I build a LlamaIndex index?
Use large chunks, set embed_batch_size to 16 or more, persist the index and ingest large folders in parts.
Related guides
- Free LLM API for RAG
- Free embeddings API
- Free LLM API for LangChain
- Free LLM API in Python
- Free LLM API with 1 million token context
Sources
- LlamaIndex: llama-index-llms-openai-like README (checked 20 November 2026)
- LlamaIndex: llama-index-embeddings-openai-like source (checked 20 November 2026)
- LlamaIndex: FunctionAgent (agent workflow) source (checked 20 November 2026)
- FreeLLMAPI docs: REST API (embeddings families, routing strategies) (checked 20 November 2026)