The RAG pipeline on free tiers
| Step | API | Free-tier concern |
|---|---|---|
| 1. Split documents into chunks | None (local code) | None |
| 2. Embed each chunk | /v1/embeddings | Many requests at the start. Some free embedding tiers allow 10 requests per minute. |
| 3. Embed the question | /v1/embeddings | One request per question |
| 4. Find the top chunks | None (local vector search) | None |
| 5. Rerank the chunks (optional) | /v1/chat/completions | One extra chat request per question |
| 6. Write the answer | /v1/chat/completions | Large prompts. Tokens-per-minute limits apply. |
Step 2 is the expensive step. You do it once for each document. Save the vectors, and embed only new or changed documents later.
Install FreeLLMAPI and the Python packages
curl -fsSL https://freellmapi.co/install.sh | bash
python3 -m pip install -U openai numpy
export FREELLMAPI_API_KEY=<unified-key>Choose a free embedding model
Select one embedding family and keep it for the whole index. Vectors from two models are not compatible. FreeLLMAPI protects you from mixed vectors: embedding failover never changes the model. It only moves between providers that serve the same family.
| Model | Provider | Dimensions | Max input | Free limit |
|---|---|---|---|---|
Gemini Embeddinggemini-embedding-001 | Google AI Studio | 3072 | 2048 | 100 rpm · 1K req/day |
Nemotron Embed VL 1Bnvidia/llama-nemotron-embed-vl-1b-v2 | NVIDIA NIM | 2048 | 8192 | ~40 rpm |
Nemotron 3 Embed 1B (OR free)nvidia/nemotron-3-embed-1b:free | OpenRouter | 2048 | 32768 | 20 rpm · shared free daily cap |
SEA-LION E5 Embedding 600Maisingapore/SEA-LION-E5-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 300Maisingapore/SEA-LION-ModernBERT-Embedding-300M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 600Maisingapore/SEA-LION-ModernBERT-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
BGE-M3@cf/baai/bge-m3 | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
EmbeddingGemma 300M@cf/google/embeddinggemma-300m | Cloudflare Workers AI | 768 | 2048 | 10K neurons/day (shared) |
Qwen3 Embedding 0.6B@cf/qwen/qwen3-embedding-0.6b | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
Cohere Embed v4embed-v4.0 | Cohere | 1536 | 128000 | 1K calls/mo (shared w/ chat) |
Gemini Embedding 2gemini-embedding-2 | Google AI Studio | 3072 | 8192 | free-tier rate limited |
Nemotron 3 Embed 1Bnvidia/nemotron-3-embed-1b | NVIDIA NIM | 2048 | 4096 | ~40 rpm |
PLaMo Embedding 1B (CF)@cf/pfnet/plamo-embedding-1b | Cloudflare Workers AI | 2048 | 4096 | 10K neurons/day (shared) |
BGE Small EN v1.5 (CF)@cf/baai/bge-small-en-v1.5 | Cloudflare Workers AI | 384 | 512 | 10K neurons/day (shared) |
BGE Base EN v1.5 (CF)@cf/baai/bge-base-en-v1.5 | Cloudflare Workers AI | 768 | 512 | 10K neurons/day (shared) |
BGE Large EN v1.5 (CF)@cf/baai/bge-large-en-v1.5 | Cloudflare Workers AI | 1024 | 512 | 10K neurons/day (shared) |
Mistral Embedmistral-embed | Mistral | 1024 | 8192 | free mode · monthly cap (shared) |
Codestral Embedcodestral-embed | Mistral | 1536 | 8192 | free mode · monthly cap (shared) |
Llama Text Embed v2 (Pinecone)llama-text-embed-v2 | Pinecone | 1024 | 2048 | 5M tokens/mo per model |
Multilingual E5 Large (Pinecone)multilingual-e5-large | Pinecone | 1024 | 507 | 5M tokens/mo per model |
Nemotron 3 Embed 1B (Speka)nvidia/nemotron-3-embed-1b | Speka | 2048 | 8192 | $1/month shared across Speka models |
OpenAI: Text Embedding 3 Small (ElectronHub)text-embedding-3-small | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
OpenAI: Text Embedding 3 Large (ElectronHub)text-embedding-3-large | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Nemotron 3 Embed 8B (Inferbase)nvidia-nemotron-3-embed-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 0.6B (Inferbase)qwen3-embedding-0-6b | Inferbase | 1024 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 4B (Inferbase)qwen3-embedding-4b | Inferbase | 2560 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 8B (Inferbase)qwen3-embedding-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
LFM2.5 Embedding 350M (LLMTR)liquid/lfm-2.5-embedding-350m-free | LLMTR | 1024 | 512 | Ongoing zero-price route; provider terms apply |
BGE-M3 (SEA-LION)BAAI/bge-m3 | SEA-LION | 1024 | 8192 | free · 10 rpm |
Cohere Embed English v3embed-english-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Qwen3 Embedding 8B (OVH)Qwen3-Embedding-8B | OVH AI Endpoints | 4096 | 32768 | free · anonymous 2 req/min per IP |
BGE-M3 (OVH)bge-m3 | OVH AI Endpoints | 1024 | 8192 | free · anonymous 2 req/min per IP |
BGE Multilingual Gemma2 (OVH)bge-multilingual-gemma2 | OVH AI Endpoints | 3584 | 8192 | free · anonymous 2 req/min per IP |
Mistral: Codestral Embed (ElectronHub)codestral-embed | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Mistral: Mistral Embed (ElectronHub)mistral-embed | ElectronHub | 1024 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 2 (ElectronHub)gemini-embedding-2 | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 001 (ElectronHub)gemini-embedding-001 | ElectronHub | 3072 | 2048 | $0.25/week shared free credits |
OpenAI: Text Embedding Ada 002 (ElectronHub)text-embedding-ada-002 | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Cohere Embed English Light v3embed-english-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual v3embed-multilingual-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual Light v3embed-multilingual-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
- Several providers for one family: a family such as
bge-m3is free on more than one provider. Failover then has somewhere to go. - Max input: chunks longer than the maximum input are cut or rejected. Make chunks smaller than this value.
- Languages: for text that is not English, use a multilingual family such as
bge-m3. The families with-en-in the name are for English. - Dimensions: larger vectors use more storage. For a small index, this is not a problem.
Complete RAG code in Python
The script splits text files into chunks, embeds them in batches and saves the vectors with NumPy. Then it answers a question with the top chunks and gives the sources. It retries when a limit stops a request.
rag.py
import json
import os
import pathlib
import time
import numpy as np
import openai
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:3001/v1",
api_key=os.environ["FREELLMAPI_API_KEY"],
max_retries=3,
timeout=90,
)
EMBED_MODEL = "bge-m3" # keep one family for the whole index
CHAT_MODEL = "auto" # or a long-context profile: auto:<profile>
INDEX = pathlib.Path("index")
def with_retry(fn, *args, **kwargs):
for attempt in range(6):
try:
return fn(*args, **kwargs)
except openai.RateLimitError as err:
header = err.response.headers.get("retry-after", "")
time.sleep(min(int(header) if header.isdigit() else 2 ** attempt, 120))
raise RuntimeError("Rate limited on every provider")
def chunk(text: str, size: int = 1500, overlap: int = 200) -> list[str]:
step = size - overlap
return [text[i:i + size] for i in range(0, max(len(text), 1), step)]
def embed(texts: list[str], batch: int = 32) -> np.ndarray:
out = []
for i in range(0, len(texts), batch):
resp = with_retry(client.embeddings.create, model=EMBED_MODEL, input=texts[i:i + batch])
out.extend(d.embedding for d in resp.data)
vecs = np.array(out, dtype=np.float32)
return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)
def build(folder: str) -> None:
chunks = []
for path in sorted(pathlib.Path(folder).rglob("*.md")):
for part in chunk(path.read_text(encoding="utf-8")):
if part.strip():
chunks.append({"source": str(path), "text": part})
INDEX.mkdir(exist_ok=True)
np.save(INDEX / "vectors.npy", embed([c["text"] for c in chunks]))
(INDEX / "chunks.json").write_text(json.dumps(chunks))
def answer(question: str, k: int = 5) -> str:
vecs = np.load(INDEX / "vectors.npy")
chunks = json.loads((INDEX / "chunks.json").read_text())
q = embed([question])[0]
top = np.argsort(vecs @ q)[::-1][:k]
context = "\n\n".join(f"[{n + 1}] ({chunks[i]['source']})\n{chunks[i]['text']}" for n, i in enumerate(top))
resp = with_retry(
client.chat.completions.create,
model=CHAT_MODEL,
temperature=0,
messages=[
{"role": "system", "content": "Answer only from the sources. Cite them as [1], [2]. If the sources do not answer, say so."},
{"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"},
],
)
return resp.choices[0].message.content
if __name__ == "__main__":
if not (INDEX / "vectors.npy").exists():
build("docs")
print(answer("How do I rotate the API keys?"))For more than a few thousand chunks, replace the NumPy search with a vector database. The embedding and chat calls stay the same.
Users often ask the same question again. FreeLLMAPI has an opt-in response cache for identical requests. A cache hit uses no provider quota. You can turn the cache on or off for one request with the X-FreeLLM-Cache: on or off header. Use temperature=0, so that the same question gives the same request.
Add a rerank step
Vector search finds similar text, but the top chunk is not always the best answer. A rerank step sorts the candidates again. The FreeLLMAPI docs do not describe a dedicated rerank endpoint. You can use a fast chat model as the reranker. Retrieve 15 candidates and keep the best 5.
LLM rerank with auto:fast
def rerank(question: str, candidates: list[str], keep: int = 5) -> list[int]:
listing = "\n".join(f"{n}: {c[:400]}" for n, c in enumerate(candidates))
resp = with_retry(
client.chat.completions.create,
model="auto:fast",
temperature=0,
messages=[{
"role": "user",
"content": (
f"Question: {question}\n\nPassages:\n{listing}\n\n"
f"Return the numbers of the {keep} most useful passages, best first, "
"as a JSON list of integers. Return only the list."
),
}],
)
try:
ids = json.loads(resp.choices[0].message.content.strip())
return [i for i in ids if isinstance(i, int) and 0 <= i < len(candidates)][:keep]
except (json.JSONDecodeError, AttributeError):
return list(range(min(keep, len(candidates)))) # fall back to vector order
# Use it in answer(): take 15 candidates, keep the best 5
# top = np.argsort(vecs @ q)[::-1][:15]
# best = rerank(question, [chunks[i]["text"] for i in top])
# top = [top[i] for i in best]The rerank step adds one chat request per question. If your free limits are tight, skip it. Better chunking often helps more than a rerank step.
Long context: when you do not need retrieval
Today, 56 free model offers have a context window of 1 million tokens or more. If your documents are small, you can send all of them in the prompt and skip the vector index. This is simpler, but each question then sends many tokens.
| Model | Free on | Context | Free limits (best offer) | Tools | Vision |
|---|---|---|---|---|---|
| OpenAI: GPT-6 Luna | ElectronHub | 1.1M | $0.25/week shared free credits | No | No |
| Gemini 3.8 Flash | APINex, Google AI Studio | 1M | 5 req/min 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | Yes | Yes |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 req/min 50 req/day Shared rate limit | Yes | No |
| DeepSeek V4 Flash 0731 | ElectronHub, Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | No | No |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| GLM 5.2 | Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | No | No |
| Gemini 3.5 Flash | Google AI Studio | 1M | 10 req/min 20 req/day 250,000 tokens/min ~3M | Yes | Yes |
| DeepSeek V4 Pro | APINex, Router9 | 1M | 5 req/min 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | No | No |
| Nemotron 3 Ultra 550B | Gizmo, Kilo Gateway, LLMTR +3 more | 1M | 200 req/day 200 rpd | Yes | No |
Showing 10 of 35 matching free models. See all models.
Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.
- Use long context for a few documents, for questions that need the full text, or for a single report.
- Use retrieval for many documents, for frequent questions, or when free tokens-per-minute limits are low.
- Combine them: retrieve 20 or more chunks and send them to a long-context model.
A large prompt can fail on a provider with a low tokens-per-minute limit. FreeLLMAPI then tries the next provider. To control which models get large prompts, make a fallback chain profile with long-context models only. Use it as auto:<profile>.
Limits and troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| Ingestion stops with 429 | The embedding family is at its limit on all providers | Use larger batches, wait for Retry-After, ingest in parts |
| Bad results after a change | The index has vectors from two families | Embed all chunks again with one family |
| Error about input length | A chunk is longer than the family's max input | Use smaller chunks |
| The answer ignores the sources | The model answers from its own knowledge | Use temperature=0 and a strict system prompt |
| Context length error on the answer | Too many or too long chunks | Lower k or use a long-context profile |
| Answers change in style | The router selected a different model | Send an X-Session-Id header or pin a model |
Free tiers have no SLA. The strongest models reach their daily limits first, and many limits reset at midnight UTC. Test your pipeline with the models that your chain will actually use.
Frequently asked questions
Is there a free API for RAG?
Yes. Several providers have free embedding and chat tiers. FreeLLMAPI serves both through one OpenAI-compatible key, with failover between providers.
Which free embedding model is best for RAG?
Select a family that is free on several providers and fits your languages. For multilingual text, bge-m3 is a common choice. Keep one family for the whole index.
Is there a free rerank API?
The FreeLLMAPI docs do not describe a rerank endpoint. You can rerank with a fast chat model, as shown on this page, or skip the rerank step.
Do I need RAG if a free model has 1 million tokens of context?
Not for a few small documents. For many documents or frequent questions, retrieval sends fewer tokens per request and stays inside free limits.
Can I change the embedding model later?
Yes, but you must embed all chunks again. Vectors from two models are not compatible, so do not mix them in one index.
Related guides
- Free embeddings API
- Free LLM API with 1 million token context
- Free LLM for LlamaIndex
- Free LLM API for LangChain
- Free LLM API in Python
Sources
- FreeLLMAPI docs: embeddings (families and failover rules) (checked 20 November 2026)
- FreeLLMAPI docs: routing strategies and response headers (checked 20 November 2026)
- OpenAI Python SDK: embeddings, errors and retries (checked 20 November 2026)