API ya LLM bure kwa RAG
Retrieval-augmented generation (RAG) inahitaji API mbili: modeli ya embedding ya kupata maandishi husika, na modeli ya gumzo ya kuandika jibu. Zote mbili zinapatikana kwenye viwango bure. FreeLLMAPI inatoa ufunguo mmoja unaoendana na OpenAI kwa modeli 41 za embedding bure na modeli 487 za gumzo bure, pamoja na akiba kati ya watoa huduma. Mwongozo huu unaonyesha modeli bure kwa kila hatua na mnyororo kamili wa Python.
Msimbo, majina ya modeli na lebo za katalogi ya moja kwa moja zinaonyeshwa kwa Kiingereza.
Mnyororo wa RAG kwenye viwango bure
| Hatua | API | Jambo la kiwango bure |
|---|---|---|
| 1. Gawanya hati kuwa vipande | Hakuna (msimbo wa ndani) | Hakuna |
| 2. Weka embedding kwa kila kipande | /v1/embeddings | Maombi mengi mwanzoni. Baadhi ya viwango bure vya embedding vinaruhusu maombi 10 kwa dakika. |
| 3. Weka embedding kwa swali | /v1/embeddings | Ombi moja kwa kila swali |
| 4. Tafuta vipande vya juu | Hakuna (utafutaji wa vekta wa ndani) | Hakuna |
| 5. Panga upya vipande (hiari) | /v1/chat/completions | Ombi moja la ziada la gumzo kwa kila swali |
| 6. Andika jibu | /v1/chat/completions | Maelekezo makubwa. Mipaka ya tokeni kwa dakika inatumika. |
Hatua ya 2 ndiyo yenye gharama kubwa. Unaifanya mara moja kwa kila hati. Hifadhi vekta, na baadaye weka embedding kwa hati mpya au zilizobadilika pekee.
Sakinisha FreeLLMAPI na pakiti za Python
curl -fsSL https://freellmapi.co/install.sh | bash
python3 -m pip install -U openai numpy
export FREELLMAPI_API_KEY=<unified-key>Chagua modeli bure ya embedding
Chagua familia moja ya embedding na uitumie kwa faharasa nzima. Vekta kutoka modeli mbili hazioani. FreeLLMAPI inakulinda dhidi ya vekta zilizochanganyika: akiba ya embedding haibadilishi modeli kamwe. Inahamia tu kati ya watoa huduma wanaotoa familia hiyo hiyo.
| Modeli | Mtoa huduma | Vipimo | Ingizo la juu zaidi | Kikomo cha bure |
|---|---|---|---|---|
Gemini Embeddinggemini-embedding-001 | Google AI Studio | 3072 | 2048 | 100 rpm · 1K req/day |
Nemotron Embed VL 1Bnvidia/llama-nemotron-embed-vl-1b-v2 | NVIDIA NIM | 2048 | 8192 | ~40 rpm |
Nemotron 3 Embed 1B (OR free)nvidia/nemotron-3-embed-1b:free | OpenRouter | 2048 | 32768 | 20 rpm · shared free daily cap |
SEA-LION E5 Embedding 600Maisingapore/SEA-LION-E5-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 300Maisingapore/SEA-LION-ModernBERT-Embedding-300M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 600Maisingapore/SEA-LION-ModernBERT-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
BGE-M3@cf/baai/bge-m3 | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
EmbeddingGemma 300M@cf/google/embeddinggemma-300m | Cloudflare Workers AI | 768 | 2048 | 10K neurons/day (shared) |
Qwen3 Embedding 0.6B@cf/qwen/qwen3-embedding-0.6b | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
Cohere Embed v4embed-v4.0 | Cohere | 1536 | 128000 | 1K calls/mo (shared w/ chat) |
Gemini Embedding 2gemini-embedding-2 | Google AI Studio | 3072 | 8192 | free-tier rate limited |
Nemotron 3 Embed 1Bnvidia/nemotron-3-embed-1b | NVIDIA NIM | 2048 | 4096 | ~40 rpm |
PLaMo Embedding 1B (CF)@cf/pfnet/plamo-embedding-1b | Cloudflare Workers AI | 2048 | 4096 | 10K neurons/day (shared) |
BGE Small EN v1.5 (CF)@cf/baai/bge-small-en-v1.5 | Cloudflare Workers AI | 384 | 512 | 10K neurons/day (shared) |
BGE Base EN v1.5 (CF)@cf/baai/bge-base-en-v1.5 | Cloudflare Workers AI | 768 | 512 | 10K neurons/day (shared) |
BGE Large EN v1.5 (CF)@cf/baai/bge-large-en-v1.5 | Cloudflare Workers AI | 1024 | 512 | 10K neurons/day (shared) |
Mistral Embedmistral-embed | Mistral | 1024 | 8192 | free mode · monthly cap (shared) |
Codestral Embedcodestral-embed | Mistral | 1536 | 8192 | free mode · monthly cap (shared) |
Llama Text Embed v2 (Pinecone)llama-text-embed-v2 | Pinecone | 1024 | 2048 | 5M tokens/mo per model |
Multilingual E5 Large (Pinecone)multilingual-e5-large | Pinecone | 1024 | 507 | 5M tokens/mo per model |
Nemotron 3 Embed 1B (Speka)nvidia/nemotron-3-embed-1b | Speka | 2048 | 8192 | $1/month shared across Speka models |
OpenAI: Text Embedding 3 Small (ElectronHub)text-embedding-3-small | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
OpenAI: Text Embedding 3 Large (ElectronHub)text-embedding-3-large | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Nemotron 3 Embed 8B (Inferbase)nvidia-nemotron-3-embed-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 0.6B (Inferbase)qwen3-embedding-0-6b | Inferbase | 1024 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 4B (Inferbase)qwen3-embedding-4b | Inferbase | 2560 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 8B (Inferbase)qwen3-embedding-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
LFM2.5 Embedding 350M (LLMTR)liquid/lfm-2.5-embedding-350m-free | LLMTR | 1024 | 512 | Ongoing zero-price route; provider terms apply |
BGE-M3 (SEA-LION)BAAI/bge-m3 | SEA-LION | 1024 | 8192 | free · 10 rpm |
Cohere Embed English v3embed-english-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Qwen3 Embedding 8B (OVH)Qwen3-Embedding-8B | OVH AI Endpoints | 4096 | 32768 | free · anonymous 2 req/min per IP |
BGE-M3 (OVH)bge-m3 | OVH AI Endpoints | 1024 | 8192 | free · anonymous 2 req/min per IP |
BGE Multilingual Gemma2 (OVH)bge-multilingual-gemma2 | OVH AI Endpoints | 3584 | 8192 | free · anonymous 2 req/min per IP |
Mistral: Codestral Embed (ElectronHub)codestral-embed | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Mistral: Mistral Embed (ElectronHub)mistral-embed | ElectronHub | 1024 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 2 (ElectronHub)gemini-embedding-2 | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 001 (ElectronHub)gemini-embedding-001 | ElectronHub | 3072 | 2048 | $0.25/week shared free credits |
OpenAI: Text Embedding Ada 002 (ElectronHub)text-embedding-ada-002 | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Cohere Embed English Light v3embed-english-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual v3embed-multilingual-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual Light v3embed-multilingual-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.
- Watoa huduma kadhaa kwa familia moja: familia kama
bge-m3ni bure kwa zaidi ya mtoa huduma mmoja. Kwa hiyo akiba ina mahali pa kwenda. - Ingizo la juu zaidi: vipande virefu kuliko ingizo la juu zaidi hukatwa au hukataliwa. Fanya vipande viwe vidogo kuliko thamani hii.
- Lugha: kwa maandishi yasiyo ya Kiingereza, tumia familia ya lugha nyingi kama
bge-m3. Familia zenye-en-kwenye jina ni za Kiingereza. - Vipimo: vekta kubwa zinatumia hifadhi zaidi. Kwa faharasa ndogo, hili si tatizo.
Msimbo kamili wa RAG kwa Python
Hati hugawanya faili za maandishi kuwa vipande, huweka embedding kwa makundi na kuhifadhi vekta kwa NumPy. Kisha hujibu swali kwa vipande vya juu na kutoa vyanzo. Hujaribu tena kikomo kinapozuia ombi.
rag.py
import json
import os
import pathlib
import time
import numpy as np
import openai
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:3001/v1",
api_key=os.environ["FREELLMAPI_API_KEY"],
max_retries=3,
timeout=90,
)
EMBED_MODEL = "bge-m3" # keep one family for the whole index
CHAT_MODEL = "auto" # or a long-context profile: auto:<profile>
INDEX = pathlib.Path("index")
def with_retry(fn, *args, **kwargs):
for attempt in range(6):
try:
return fn(*args, **kwargs)
except openai.RateLimitError as err:
header = err.response.headers.get("retry-after", "")
time.sleep(min(int(header) if header.isdigit() else 2 ** attempt, 120))
raise RuntimeError("Rate limited on every provider")
def chunk(text: str, size: int = 1500, overlap: int = 200) -> list[str]:
step = size - overlap
return [text[i:i + size] for i in range(0, max(len(text), 1), step)]
def embed(texts: list[str], batch: int = 32) -> np.ndarray:
out = []
for i in range(0, len(texts), batch):
resp = with_retry(client.embeddings.create, model=EMBED_MODEL, input=texts[i:i + batch])
out.extend(d.embedding for d in resp.data)
vecs = np.array(out, dtype=np.float32)
return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)
def build(folder: str) -> None:
chunks = []
for path in sorted(pathlib.Path(folder).rglob("*.md")):
for part in chunk(path.read_text(encoding="utf-8")):
if part.strip():
chunks.append({"source": str(path), "text": part})
INDEX.mkdir(exist_ok=True)
np.save(INDEX / "vectors.npy", embed([c["text"] for c in chunks]))
(INDEX / "chunks.json").write_text(json.dumps(chunks))
def answer(question: str, k: int = 5) -> str:
vecs = np.load(INDEX / "vectors.npy")
chunks = json.loads((INDEX / "chunks.json").read_text())
q = embed([question])[0]
top = np.argsort(vecs @ q)[::-1][:k]
context = "\n\n".join(f"[{n + 1}] ({chunks[i]['source']})\n{chunks[i]['text']}" for n, i in enumerate(top))
resp = with_retry(
client.chat.completions.create,
model=CHAT_MODEL,
temperature=0,
messages=[
{"role": "system", "content": "Answer only from the sources. Cite them as [1], [2]. If the sources do not answer, say so."},
{"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"},
],
)
return resp.choices[0].message.content
if __name__ == "__main__":
if not (INDEX / "vectors.npy").exists():
build("docs")
print(answer("How do I rotate the API keys?"))Kwa vipande zaidi ya elfu chache, badilisha utafutaji wa NumPy na hifadhidata ya vekta. Wito wa embedding na gumzo unabaki vilevile.
Watumiaji mara nyingi huuliza swali lilelile tena. FreeLLMAPI ina akiba ya majibu ya hiari kwa maombi yanayofanana. Jibu kutoka akiba halitumii mgao wa mtoa huduma. Unaweza kuwasha au kuzima akiba kwa ombi moja kwa kichwa cha X-FreeLLM-Cache: on au off. Tumia temperature=0, ili swali lilelile lipate ombi lilelile.
Ongeza hatua ya rerank
Utafutaji wa vekta unapata maandishi yanayofanana, lakini kipande cha juu si mara zote jibu bora. Hatua ya rerank hupanga upya wagombea. Nyaraka za FreeLLMAPI hazielezi sehemu maalum ya mwisho ya rerank. Unaweza kutumia modeli ya gumzo ya haraka kama kipanga upya. Pata wagombea 15 na uhifadhi bora 5.
LLM rerank kwa auto:fast
def rerank(question: str, candidates: list[str], keep: int = 5) -> list[int]:
listing = "\n".join(f"{n}: {c[:400]}" for n, c in enumerate(candidates))
resp = with_retry(
client.chat.completions.create,
model="auto:fast",
temperature=0,
messages=[{
"role": "user",
"content": (
f"Question: {question}\n\nPassages:\n{listing}\n\n"
f"Return the numbers of the {keep} most useful passages, best first, "
"as a JSON list of integers. Return only the list."
),
}],
)
try:
ids = json.loads(resp.choices[0].message.content.strip())
return [i for i in ids if isinstance(i, int) and 0 <= i < len(candidates)][:keep]
except (json.JSONDecodeError, AttributeError):
return list(range(min(keep, len(candidates)))) # fall back to vector order
# Use it in answer(): take 15 candidates, keep the best 5
# top = np.argsort(vecs @ q)[::-1][:15]
# best = rerank(question, [chunks[i]["text"] for i in top])
# top = [top[i] for i in best]Hatua ya rerank inaongeza ombi moja la gumzo kwa kila swali. Ikiwa mipaka yako bure ni midogo, ruka hatua hii. Ugawaji bora wa vipande mara nyingi unasaidia zaidi kuliko hatua ya rerank.
Muktadha mrefu: wakati hauhitaji utafutaji
Leo, ofa 56 za modeli bure zina dirisha la muktadha la tokeni milioni 1 au zaidi. Ikiwa hati zako ni ndogo, unaweza kutuma zote kwenye maelekezo na kuruka faharasa ya vekta. Hii ni rahisi zaidi, lakini kila swali linatuma tokeni nyingi.
| Modeli | Bure kwenye | Muktadha | Vikomo vya bure (ofa bora) | Zana | Uoni |
|---|---|---|---|---|---|
| OpenAI: GPT-6 Luna | ElectronHub | 1.1M | $0.25/week shared free credits | Hapana | Hapana |
| Gemini 3.8 Flash | APINex, Google AI Studio | 1M | 5 maombi/dakika 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | Ndiyo | Ndiyo |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 maombi/dakika 50 maombi/siku Shared rate limit | Ndiyo | Hapana |
| DeepSeek V4 Flash 0731 | ElectronHub, Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | Hapana | Hapana |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| GLM 5.2 | Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | Hapana | Hapana |
| Gemini 3.5 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| DeepSeek V4 Pro | APINex, Router9 | 1M | 5 maombi/dakika 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | Hapana | Hapana |
| Nemotron 3 Ultra 550B | Gizmo, Kilo Gateway, LLMTR +3 zaidi | 1M | 200 maombi/siku 200 rpd | Ndiyo | Hapana |
Inaonyesha 10 kati ya 35 modeli za bure zinazolingana. Tazama modeli zote.
Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.
- Tumia muktadha mrefu kwa hati chache, kwa maswali yanayohitaji maandishi yote, au kwa ripoti moja.
- Tumia utafutaji kwa hati nyingi, kwa maswali ya mara kwa mara, au pale mipaka ya tokeni kwa dakika bure ni midogo.
- Changanya: pata vipande 20 au zaidi na uvitume kwa modeli ya muktadha mrefu.
Maelekezo makubwa yanaweza kushindwa kwa mtoa huduma mwenye mipaka midogo ya tokeni kwa dakika. FreeLLMAPI kisha inajaribu mtoa huduma anayefuata. Ili kudhibiti modeli zipi zipokee maelekezo makubwa, tengeneza wasifu wa mnyororo wa akiba wenye modeli za muktadha mrefu pekee. Uitumie kama auto:<profile>.
Mipaka na kutatua matatizo
| Tatizo | Chanzo | Suluhisho |
|---|---|---|
| Uingizaji unasimama kwa 429 | Familia ya embedding imefikia kikomo kwa watoa huduma wote | Tumia makundi makubwa zaidi, subiri Retry-After, ingiza kwa sehemu |
| Matokeo mabaya baada ya mabadiliko | Faharasa ina vekta kutoka familia mbili | Weka embedding kwa vipande vyote tena kwa familia moja |
| Hitilafu kuhusu urefu wa ingizo | Kipande ni kirefu kuliko ingizo la juu la familia | Tumia vipande vidogo zaidi |
| Jibu halizingatii vyanzo | Modeli inajibu kutokana na maarifa yake yenyewe | Tumia temperature=0 na maelekezo madhubuti ya mfumo |
| Hitilafu ya urefu wa muktadha kwenye jibu | Vipande vingi mno au virefu mno | Punguza k au tumia wasifu wa muktadha mrefu |
| Majibu yanabadilika kwa mtindo | Kipanga njia kimechagua modeli nyingine | Tuma kichwa cha X-Session-Id au bandika modeli |
Viwango bure havina SLA. Modeli zenye nguvu zaidi hufikia mipaka yake ya kila siku kwanza, na mipaka mingi huwekwa upya saa sita usiku UTC. Jaribu mnyororo wako kwa modeli ambazo mnyororo wako utazitumia kweli.
Maswali yanayoulizwa mara kwa mara
Kuna API bure ya RAG?
Ndiyo. Watoa huduma kadhaa wana viwango bure vya embedding na gumzo. FreeLLMAPI inahudumia zote mbili kupitia ufunguo mmoja unaoendana na OpenAI, pamoja na akiba kati ya watoa huduma.
Modeli bure ipi ya embedding ni bora kwa RAG?
Chagua familia ambayo ni bure kwa watoa huduma kadhaa na inalingana na lugha zako. Kwa maandishi ya lugha nyingi, bge-m3 ni chaguo la kawaida. Tumia familia moja kwa faharasa nzima.
Kuna API bure ya rerank?
Nyaraka za FreeLLMAPI hazielezi sehemu ya mwisho ya rerank. Unaweza kupanga upya kwa modeli ya gumzo ya haraka, kama inavyoonyeshwa kwenye ukurasa huu, au kuruka hatua ya rerank.
Je, nahitaji RAG ikiwa modeli bure ina muktadha wa tokeni milioni 1?
Si kwa hati chache ndogo. Kwa hati nyingi au maswali ya mara kwa mara, utafutaji unatuma tokeni chache kwa kila ombi na unabaki ndani ya mipaka bure.
Naweza kubadilisha modeli ya embedding baadaye?
Ndiyo, lakini lazima uweke embedding kwa vipande vyote tena. Vekta kutoka modeli mbili hazioani, kwa hiyo usizichanganye kwenye faharasa moja.
Miongozo inayohusiana
- API ya bure ya embeddings
- API ya LLM bure yenye muktadha wa tokeni milioni 1
- LLM ya bure kwa LlamaIndex
- API ya LLM ya bure kwa LangChain
- LLM API ya bure kwa Python
Vyanzo
- FreeLLMAPI docs: embeddings (families and failover rules) (imekaguliwa 20 November 2026)
- FreeLLMAPI docs: routing strategies and response headers (imekaguliwa 20 November 2026)
- OpenAI Python SDK: embeddings, errors and retries (imekaguliwa 20 November 2026)