Guide

Free LLM API for RAG

Retrieval-augmented generation (RAG) needs two APIs: an embedding model to find the relevant text, and a chat model to write the answer. Both are available on free tiers. FreeLLMAPI gives one OpenAI-compatible key for 41 free embedding models and 487 free chat models, with failover between providers. This guide shows the free models for each step and a complete Python pipeline.

The RAG pipeline on free tiers

StepAPIFree-tier concern
1. Split documents into chunksNone (local code)None
2. Embed each chunk/v1/embeddingsMany requests at the start. Some free embedding tiers allow 10 requests per minute.
3. Embed the question/v1/embeddingsOne request per question
4. Find the top chunksNone (local vector search)None
5. Rerank the chunks (optional)/v1/chat/completionsOne extra chat request per question
6. Write the answer/v1/chat/completionsLarge prompts. Tokens-per-minute limits apply.

Step 2 is the expensive step. You do it once for each document. Save the vectors, and embed only new or changed documents later.

Install FreeLLMAPI and the Python packages

curl -fsSL https://freellmapi.co/install.sh | bash

python3 -m pip install -U openai numpy
export FREELLMAPI_API_KEY=<unified-key>

Choose a free embedding model

Select one embedding family and keep it for the whole index. Vectors from two models are not compatible. FreeLLMAPI protects you from mixed vectors: embedding failover never changes the model. It only moves between providers that serve the same family.

ModelProviderDimensionsMax inputFree limit
Gemini Embedding
gemini-embedding-001
Google AI Studio30722048100 rpm · 1K req/day
Nemotron Embed VL 1B
nvidia/llama-nemotron-embed-vl-1b-v2
NVIDIA NIM20488192~40 rpm
Nemotron 3 Embed 1B (OR free)
nvidia/nemotron-3-embed-1b:free
OpenRouter20483276820 rpm · shared free daily cap
SEA-LION E5 Embedding 600M
aisingapore/SEA-LION-E5-Embedding-600M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 300M
aisingapore/SEA-LION-ModernBERT-Embedding-300M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 600M
aisingapore/SEA-LION-ModernBERT-Embedding-600M
SEA-LION10248192free · 10 rpm
BGE-M3
@cf/baai/bge-m3
Cloudflare Workers AI1024819210K neurons/day (shared)
EmbeddingGemma 300M
@cf/google/embeddinggemma-300m
Cloudflare Workers AI768204810K neurons/day (shared)
Qwen3 Embedding 0.6B
@cf/qwen/qwen3-embedding-0.6b
Cloudflare Workers AI1024819210K neurons/day (shared)
Cohere Embed v4
embed-v4.0
Cohere15361280001K calls/mo (shared w/ chat)
Gemini Embedding 2
gemini-embedding-2
Google AI Studio30728192free-tier rate limited
Nemotron 3 Embed 1B
nvidia/nemotron-3-embed-1b
NVIDIA NIM20484096~40 rpm
PLaMo Embedding 1B (CF)
@cf/pfnet/plamo-embedding-1b
Cloudflare Workers AI2048409610K neurons/day (shared)
BGE Small EN v1.5 (CF)
@cf/baai/bge-small-en-v1.5
Cloudflare Workers AI38451210K neurons/day (shared)
BGE Base EN v1.5 (CF)
@cf/baai/bge-base-en-v1.5
Cloudflare Workers AI76851210K neurons/day (shared)
BGE Large EN v1.5 (CF)
@cf/baai/bge-large-en-v1.5
Cloudflare Workers AI102451210K neurons/day (shared)
Mistral Embed
mistral-embed
Mistral10248192free mode · monthly cap (shared)
Codestral Embed
codestral-embed
Mistral15368192free mode · monthly cap (shared)
Llama Text Embed v2 (Pinecone)
llama-text-embed-v2
Pinecone102420485M tokens/mo per model
Multilingual E5 Large (Pinecone)
multilingual-e5-large
Pinecone10245075M tokens/mo per model
Nemotron 3 Embed 1B (Speka)
nvidia/nemotron-3-embed-1b
Speka20488192$1/month shared across Speka models
OpenAI: Text Embedding 3 Small (ElectronHub)
text-embedding-3-small
ElectronHub15368192$0.25/week shared free credits
OpenAI: Text Embedding 3 Large (ElectronHub)
text-embedding-3-large
ElectronHub30728192$0.25/week shared free credits
Nemotron 3 Embed 8B (Inferbase)
nvidia-nemotron-3-embed-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 0.6B (Inferbase)
qwen3-embedding-0-6b
Inferbase102481925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 4B (Inferbase)
qwen3-embedding-4b
Inferbase256081925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 8B (Inferbase)
qwen3-embedding-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
LFM2.5 Embedding 350M (LLMTR)
liquid/lfm-2.5-embedding-350m-free
LLMTR1024512Ongoing zero-price route; provider terms apply
BGE-M3 (SEA-LION)
BAAI/bge-m3
SEA-LION10248192free · 10 rpm
Cohere Embed English v3
embed-english-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Qwen3 Embedding 8B (OVH)
Qwen3-Embedding-8B
OVH AI Endpoints409632768free · anonymous 2 req/min per IP
BGE-M3 (OVH)
bge-m3
OVH AI Endpoints10248192free · anonymous 2 req/min per IP
BGE Multilingual Gemma2 (OVH)
bge-multilingual-gemma2
OVH AI Endpoints35848192free · anonymous 2 req/min per IP
Mistral: Codestral Embed (ElectronHub)
codestral-embed
ElectronHub15368192$0.25/week shared free credits
Mistral: Mistral Embed (ElectronHub)
mistral-embed
ElectronHub10248192$0.25/week shared free credits
Google: Gemini Embedding 2 (ElectronHub)
gemini-embedding-2
ElectronHub30728192$0.25/week shared free credits
Google: Gemini Embedding 001 (ElectronHub)
gemini-embedding-001
ElectronHub30722048$0.25/week shared free credits
OpenAI: Text Embedding Ada 002 (ElectronHub)
text-embedding-ada-002
ElectronHub15368192$0.25/week shared free credits
Cohere Embed English Light v3
embed-english-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)
Cohere Embed Multilingual v3
embed-multilingual-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Cohere Embed Multilingual Light v3
embed-multilingual-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)

Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.

  • Several providers for one family: a family such as bge-m3 is free on more than one provider. Failover then has somewhere to go.
  • Max input: chunks longer than the maximum input are cut or rejected. Make chunks smaller than this value.
  • Languages: for text that is not English, use a multilingual family such as bge-m3. The families with -en- in the name are for English.
  • Dimensions: larger vectors use more storage. For a small index, this is not a problem.

Complete RAG code in Python

The script splits text files into chunks, embeds them in batches and saves the vectors with NumPy. Then it answers a question with the top chunks and gives the sources. It retries when a limit stops a request.

rag.py

import json
import os
import pathlib
import time

import numpy as np
import openai
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key=os.environ["FREELLMAPI_API_KEY"],
    max_retries=3,
    timeout=90,
)
EMBED_MODEL = "bge-m3"   # keep one family for the whole index
CHAT_MODEL = "auto"      # or a long-context profile: auto:<profile>
INDEX = pathlib.Path("index")

def with_retry(fn, *args, **kwargs):
    for attempt in range(6):
        try:
            return fn(*args, **kwargs)
        except openai.RateLimitError as err:
            header = err.response.headers.get("retry-after", "")
            time.sleep(min(int(header) if header.isdigit() else 2 ** attempt, 120))
    raise RuntimeError("Rate limited on every provider")

def chunk(text: str, size: int = 1500, overlap: int = 200) -> list[str]:
    step = size - overlap
    return [text[i:i + size] for i in range(0, max(len(text), 1), step)]

def embed(texts: list[str], batch: int = 32) -> np.ndarray:
    out = []
    for i in range(0, len(texts), batch):
        resp = with_retry(client.embeddings.create, model=EMBED_MODEL, input=texts[i:i + batch])
        out.extend(d.embedding for d in resp.data)
    vecs = np.array(out, dtype=np.float32)
    return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)

def build(folder: str) -> None:
    chunks = []
    for path in sorted(pathlib.Path(folder).rglob("*.md")):
        for part in chunk(path.read_text(encoding="utf-8")):
            if part.strip():
                chunks.append({"source": str(path), "text": part})
    INDEX.mkdir(exist_ok=True)
    np.save(INDEX / "vectors.npy", embed([c["text"] for c in chunks]))
    (INDEX / "chunks.json").write_text(json.dumps(chunks))

def answer(question: str, k: int = 5) -> str:
    vecs = np.load(INDEX / "vectors.npy")
    chunks = json.loads((INDEX / "chunks.json").read_text())
    q = embed([question])[0]
    top = np.argsort(vecs @ q)[::-1][:k]
    context = "\n\n".join(f"[{n + 1}] ({chunks[i]['source']})\n{chunks[i]['text']}" for n, i in enumerate(top))
    resp = with_retry(
        client.chat.completions.create,
        model=CHAT_MODEL,
        temperature=0,
        messages=[
            {"role": "system", "content": "Answer only from the sources. Cite them as [1], [2]. If the sources do not answer, say so."},
            {"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content

if __name__ == "__main__":
    if not (INDEX / "vectors.npy").exists():
        build("docs")
    print(answer("How do I rotate the API keys?"))

For more than a few thousand chunks, replace the NumPy search with a vector database. The embedding and chat calls stay the same.

Users often ask the same question again. FreeLLMAPI has an opt-in response cache for identical requests. A cache hit uses no provider quota. You can turn the cache on or off for one request with the X-FreeLLM-Cache: on or off header. Use temperature=0, so that the same question gives the same request.

Add a rerank step

Vector search finds similar text, but the top chunk is not always the best answer. A rerank step sorts the candidates again. The FreeLLMAPI docs do not describe a dedicated rerank endpoint. You can use a fast chat model as the reranker. Retrieve 15 candidates and keep the best 5.

LLM rerank with auto:fast

def rerank(question: str, candidates: list[str], keep: int = 5) -> list[int]:
    listing = "\n".join(f"{n}: {c[:400]}" for n, c in enumerate(candidates))
    resp = with_retry(
        client.chat.completions.create,
        model="auto:fast",
        temperature=0,
        messages=[{
            "role": "user",
            "content": (
                f"Question: {question}\n\nPassages:\n{listing}\n\n"
                f"Return the numbers of the {keep} most useful passages, best first, "
                "as a JSON list of integers. Return only the list."
            ),
        }],
    )
    try:
        ids = json.loads(resp.choices[0].message.content.strip())
        return [i for i in ids if isinstance(i, int) and 0 <= i < len(candidates)][:keep]
    except (json.JSONDecodeError, AttributeError):
        return list(range(min(keep, len(candidates))))  # fall back to vector order


# Use it in answer(): take 15 candidates, keep the best 5
# top = np.argsort(vecs @ q)[::-1][:15]
# best = rerank(question, [chunks[i]["text"] for i in top])
# top = [top[i] for i in best]

The rerank step adds one chat request per question. If your free limits are tight, skip it. Better chunking often helps more than a rerank step.

Long context: when you do not need retrieval

Today, 56 free model offers have a context window of 1 million tokens or more. If your documents are small, you can send all of them in the prompt and skip the vector index. This is simpler, but each question then sends many tokens.

Free models with the largest context windows
ModelFree onContextFree limits (best offer)ToolsVision
OpenAI: GPT-6 LunaElectronHub1.1M$0.25/week shared free creditsNoNo
Gemini 3.8 FlashAPINex, Google AI Studio1M5 req/min
10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account
YesYes
Gemini 3.7 FlashGoogle AI Studio1M10 req/min
20 req/day
250,000 tokens/min
~3M
YesYes
DeepSeek V4 FlashAMD Radeon Cloud, BazaarLink, ElectronHub1M10 req/min
50 req/day
Shared rate limit
YesNo
DeepSeek V4 Flash 0731ElectronHub, Router91M10,000 credits/month (~$1), shared; account-anniversary resetNoNo
Gemini 3.6 FlashGoogle AI Studio1M10 req/min
20 req/day
250,000 tokens/min
~3M
YesYes
GLM 5.2Router91M10,000 credits/month (~$1), shared; account-anniversary resetNoNo
Gemini 3.5 FlashGoogle AI Studio1M10 req/min
20 req/day
250,000 tokens/min
~3M
YesYes
DeepSeek V4 ProAPINex, Router91M5 req/min
10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account
NoNo
Nemotron 3 Ultra 550BGizmo, Kilo Gateway, LLMTR +3 more1M200 req/day
200 rpd
YesNo

Showing 10 of 35 matching free models. See all models.

Live data from the FreeLLMAPI catalog (version 2026.11.19, checked 19 November 2026). Free limits change often. The catalog is checked every day.

  • Use long context for a few documents, for questions that need the full text, or for a single report.
  • Use retrieval for many documents, for frequent questions, or when free tokens-per-minute limits are low.
  • Combine them: retrieve 20 or more chunks and send them to a long-context model.

A large prompt can fail on a provider with a low tokens-per-minute limit. FreeLLMAPI then tries the next provider. To control which models get large prompts, make a fallback chain profile with long-context models only. Use it as auto:<profile>.

Limits and troubleshooting

ProblemCauseFix
Ingestion stops with 429The embedding family is at its limit on all providersUse larger batches, wait for Retry-After, ingest in parts
Bad results after a changeThe index has vectors from two familiesEmbed all chunks again with one family
Error about input lengthA chunk is longer than the family's max inputUse smaller chunks
The answer ignores the sourcesThe model answers from its own knowledgeUse temperature=0 and a strict system prompt
Context length error on the answerToo many or too long chunksLower k or use a long-context profile
Answers change in styleThe router selected a different modelSend an X-Session-Id header or pin a model

Free tiers have no SLA. The strongest models reach their daily limits first, and many limits reset at midnight UTC. Test your pipeline with the models that your chain will actually use.

Use every one of these through one key. FreeLLMAPI is an open-source, self-hosted router that puts all these free tiers behind a single OpenAI-compatible endpoint and fails over when one is rate-limited. Browse the catalog or go live.

Frequently asked questions

Is there a free API for RAG?

Yes. Several providers have free embedding and chat tiers. FreeLLMAPI serves both through one OpenAI-compatible key, with failover between providers.

Which free embedding model is best for RAG?

Select a family that is free on several providers and fits your languages. For multilingual text, bge-m3 is a common choice. Keep one family for the whole index.

Is there a free rerank API?

The FreeLLMAPI docs do not describe a rerank endpoint. You can rerank with a fast chat model, as shown on this page, or skip the rerank step.

Do I need RAG if a free model has 1 million tokens of context?

Not for a few small documents. For many documents or frequent questions, retrieval sends fewer tokens per request and stays inside free limits.

Can I change the embedding model later?

Yes, but you must embed all chunks again. Vectors from two models are not compatible, so do not mix them in one index.

Sources

All guides · Browse all free models

Keep reading

All posts →
Guide

Best free LLM APIs 2026

Ten genuinely free LLM APIs worth using — ranked, with real limits.

Guide

Unlimited free AI tokens?

No free AI API is unlimited. The closest real thing: 10.1 billion free tokens a month behind one key.

Guide

How to get a free LLM API key

Get a working key in minutes — no credit card required.