Nyumbani / Miongozo / API ya LLM bure kwa RAG

API ya LLM bure kwa RAG

Retrieval-augmented generation (RAG) inahitaji API mbili: modeli ya embedding ya kupata maandishi husika, na modeli ya gumzo ya kuandika jibu. Zote mbili zinapatikana kwenye viwango bure. FreeLLMAPI inatoa ufunguo mmoja unaoendana na OpenAI kwa modeli 41 za embedding bure na modeli 487 za gumzo bure, pamoja na akiba kati ya watoa huduma. Mwongozo huu unaonyesha modeli bure kwa kila hatua na mnyororo kamili wa Python.

Msimbo, majina ya modeli na lebo za katalogi ya moja kwa moja zinaonyeshwa kwa Kiingereza.

Mnyororo wa RAG kwenye viwango bure

HatuaAPIJambo la kiwango bure
1. Gawanya hati kuwa vipandeHakuna (msimbo wa ndani)Hakuna
2. Weka embedding kwa kila kipande/v1/embeddingsMaombi mengi mwanzoni. Baadhi ya viwango bure vya embedding vinaruhusu maombi 10 kwa dakika.
3. Weka embedding kwa swali/v1/embeddingsOmbi moja kwa kila swali
4. Tafuta vipande vya juuHakuna (utafutaji wa vekta wa ndani)Hakuna
5. Panga upya vipande (hiari)/v1/chat/completionsOmbi moja la ziada la gumzo kwa kila swali
6. Andika jibu/v1/chat/completionsMaelekezo makubwa. Mipaka ya tokeni kwa dakika inatumika.

Hatua ya 2 ndiyo yenye gharama kubwa. Unaifanya mara moja kwa kila hati. Hifadhi vekta, na baadaye weka embedding kwa hati mpya au zilizobadilika pekee.

Sakinisha FreeLLMAPI na pakiti za Python

curl -fsSL https://freellmapi.co/install.sh | bash

python3 -m pip install -U openai numpy
export FREELLMAPI_API_KEY=<unified-key>

Chagua modeli bure ya embedding

Chagua familia moja ya embedding na uitumie kwa faharasa nzima. Vekta kutoka modeli mbili hazioani. FreeLLMAPI inakulinda dhidi ya vekta zilizochanganyika: akiba ya embedding haibadilishi modeli kamwe. Inahamia tu kati ya watoa huduma wanaotoa familia hiyo hiyo.

ModeliMtoa hudumaVipimoIngizo la juu zaidiKikomo cha bure
Gemini Embedding
gemini-embedding-001
Google AI Studio30722048100 rpm · 1K req/day
Nemotron Embed VL 1B
nvidia/llama-nemotron-embed-vl-1b-v2
NVIDIA NIM20488192~40 rpm
Nemotron 3 Embed 1B (OR free)
nvidia/nemotron-3-embed-1b:free
OpenRouter20483276820 rpm · shared free daily cap
SEA-LION E5 Embedding 600M
aisingapore/SEA-LION-E5-Embedding-600M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 300M
aisingapore/SEA-LION-ModernBERT-Embedding-300M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 600M
aisingapore/SEA-LION-ModernBERT-Embedding-600M
SEA-LION10248192free · 10 rpm
BGE-M3
@cf/baai/bge-m3
Cloudflare Workers AI1024819210K neurons/day (shared)
EmbeddingGemma 300M
@cf/google/embeddinggemma-300m
Cloudflare Workers AI768204810K neurons/day (shared)
Qwen3 Embedding 0.6B
@cf/qwen/qwen3-embedding-0.6b
Cloudflare Workers AI1024819210K neurons/day (shared)
Cohere Embed v4
embed-v4.0
Cohere15361280001K calls/mo (shared w/ chat)
Gemini Embedding 2
gemini-embedding-2
Google AI Studio30728192free-tier rate limited
Nemotron 3 Embed 1B
nvidia/nemotron-3-embed-1b
NVIDIA NIM20484096~40 rpm
PLaMo Embedding 1B (CF)
@cf/pfnet/plamo-embedding-1b
Cloudflare Workers AI2048409610K neurons/day (shared)
BGE Small EN v1.5 (CF)
@cf/baai/bge-small-en-v1.5
Cloudflare Workers AI38451210K neurons/day (shared)
BGE Base EN v1.5 (CF)
@cf/baai/bge-base-en-v1.5
Cloudflare Workers AI76851210K neurons/day (shared)
BGE Large EN v1.5 (CF)
@cf/baai/bge-large-en-v1.5
Cloudflare Workers AI102451210K neurons/day (shared)
Mistral Embed
mistral-embed
Mistral10248192free mode · monthly cap (shared)
Codestral Embed
codestral-embed
Mistral15368192free mode · monthly cap (shared)
Llama Text Embed v2 (Pinecone)
llama-text-embed-v2
Pinecone102420485M tokens/mo per model
Multilingual E5 Large (Pinecone)
multilingual-e5-large
Pinecone10245075M tokens/mo per model
Nemotron 3 Embed 1B (Speka)
nvidia/nemotron-3-embed-1b
Speka20488192$1/month shared across Speka models
OpenAI: Text Embedding 3 Small (ElectronHub)
text-embedding-3-small
ElectronHub15368192$0.25/week shared free credits
OpenAI: Text Embedding 3 Large (ElectronHub)
text-embedding-3-large
ElectronHub30728192$0.25/week shared free credits
Nemotron 3 Embed 8B (Inferbase)
nvidia-nemotron-3-embed-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 0.6B (Inferbase)
qwen3-embedding-0-6b
Inferbase102481925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 4B (Inferbase)
qwen3-embedding-4b
Inferbase256081925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 8B (Inferbase)
qwen3-embedding-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
LFM2.5 Embedding 350M (LLMTR)
liquid/lfm-2.5-embedding-350m-free
LLMTR1024512Ongoing zero-price route; provider terms apply
BGE-M3 (SEA-LION)
BAAI/bge-m3
SEA-LION10248192free · 10 rpm
Cohere Embed English v3
embed-english-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Qwen3 Embedding 8B (OVH)
Qwen3-Embedding-8B
OVH AI Endpoints409632768free · anonymous 2 req/min per IP
BGE-M3 (OVH)
bge-m3
OVH AI Endpoints10248192free · anonymous 2 req/min per IP
BGE Multilingual Gemma2 (OVH)
bge-multilingual-gemma2
OVH AI Endpoints35848192free · anonymous 2 req/min per IP
Mistral: Codestral Embed (ElectronHub)
codestral-embed
ElectronHub15368192$0.25/week shared free credits
Mistral: Mistral Embed (ElectronHub)
mistral-embed
ElectronHub10248192$0.25/week shared free credits
Google: Gemini Embedding 2 (ElectronHub)
gemini-embedding-2
ElectronHub30728192$0.25/week shared free credits
Google: Gemini Embedding 001 (ElectronHub)
gemini-embedding-001
ElectronHub30722048$0.25/week shared free credits
OpenAI: Text Embedding Ada 002 (ElectronHub)
text-embedding-ada-002
ElectronHub15368192$0.25/week shared free credits
Cohere Embed English Light v3
embed-english-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)
Cohere Embed Multilingual v3
embed-multilingual-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Cohere Embed Multilingual Light v3
embed-multilingual-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)

Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.

  • Watoa huduma kadhaa kwa familia moja: familia kama bge-m3 ni bure kwa zaidi ya mtoa huduma mmoja. Kwa hiyo akiba ina mahali pa kwenda.
  • Ingizo la juu zaidi: vipande virefu kuliko ingizo la juu zaidi hukatwa au hukataliwa. Fanya vipande viwe vidogo kuliko thamani hii.
  • Lugha: kwa maandishi yasiyo ya Kiingereza, tumia familia ya lugha nyingi kama bge-m3. Familia zenye -en- kwenye jina ni za Kiingereza.
  • Vipimo: vekta kubwa zinatumia hifadhi zaidi. Kwa faharasa ndogo, hili si tatizo.

Msimbo kamili wa RAG kwa Python

Hati hugawanya faili za maandishi kuwa vipande, huweka embedding kwa makundi na kuhifadhi vekta kwa NumPy. Kisha hujibu swali kwa vipande vya juu na kutoa vyanzo. Hujaribu tena kikomo kinapozuia ombi.

rag.py

import json
import os
import pathlib
import time

import numpy as np
import openai
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key=os.environ["FREELLMAPI_API_KEY"],
    max_retries=3,
    timeout=90,
)
EMBED_MODEL = "bge-m3"   # keep one family for the whole index
CHAT_MODEL = "auto"      # or a long-context profile: auto:<profile>
INDEX = pathlib.Path("index")

def with_retry(fn, *args, **kwargs):
    for attempt in range(6):
        try:
            return fn(*args, **kwargs)
        except openai.RateLimitError as err:
            header = err.response.headers.get("retry-after", "")
            time.sleep(min(int(header) if header.isdigit() else 2 ** attempt, 120))
    raise RuntimeError("Rate limited on every provider")

def chunk(text: str, size: int = 1500, overlap: int = 200) -> list[str]:
    step = size - overlap
    return [text[i:i + size] for i in range(0, max(len(text), 1), step)]

def embed(texts: list[str], batch: int = 32) -> np.ndarray:
    out = []
    for i in range(0, len(texts), batch):
        resp = with_retry(client.embeddings.create, model=EMBED_MODEL, input=texts[i:i + batch])
        out.extend(d.embedding for d in resp.data)
    vecs = np.array(out, dtype=np.float32)
    return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)

def build(folder: str) -> None:
    chunks = []
    for path in sorted(pathlib.Path(folder).rglob("*.md")):
        for part in chunk(path.read_text(encoding="utf-8")):
            if part.strip():
                chunks.append({"source": str(path), "text": part})
    INDEX.mkdir(exist_ok=True)
    np.save(INDEX / "vectors.npy", embed([c["text"] for c in chunks]))
    (INDEX / "chunks.json").write_text(json.dumps(chunks))

def answer(question: str, k: int = 5) -> str:
    vecs = np.load(INDEX / "vectors.npy")
    chunks = json.loads((INDEX / "chunks.json").read_text())
    q = embed([question])[0]
    top = np.argsort(vecs @ q)[::-1][:k]
    context = "\n\n".join(f"[{n + 1}] ({chunks[i]['source']})\n{chunks[i]['text']}" for n, i in enumerate(top))
    resp = with_retry(
        client.chat.completions.create,
        model=CHAT_MODEL,
        temperature=0,
        messages=[
            {"role": "system", "content": "Answer only from the sources. Cite them as [1], [2]. If the sources do not answer, say so."},
            {"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content

if __name__ == "__main__":
    if not (INDEX / "vectors.npy").exists():
        build("docs")
    print(answer("How do I rotate the API keys?"))

Kwa vipande zaidi ya elfu chache, badilisha utafutaji wa NumPy na hifadhidata ya vekta. Wito wa embedding na gumzo unabaki vilevile.

Watumiaji mara nyingi huuliza swali lilelile tena. FreeLLMAPI ina akiba ya majibu ya hiari kwa maombi yanayofanana. Jibu kutoka akiba halitumii mgao wa mtoa huduma. Unaweza kuwasha au kuzima akiba kwa ombi moja kwa kichwa cha X-FreeLLM-Cache: on au off. Tumia temperature=0, ili swali lilelile lipate ombi lilelile.

Ongeza hatua ya rerank

Utafutaji wa vekta unapata maandishi yanayofanana, lakini kipande cha juu si mara zote jibu bora. Hatua ya rerank hupanga upya wagombea. Nyaraka za FreeLLMAPI hazielezi sehemu maalum ya mwisho ya rerank. Unaweza kutumia modeli ya gumzo ya haraka kama kipanga upya. Pata wagombea 15 na uhifadhi bora 5.

LLM rerank kwa auto:fast

def rerank(question: str, candidates: list[str], keep: int = 5) -> list[int]:
    listing = "\n".join(f"{n}: {c[:400]}" for n, c in enumerate(candidates))
    resp = with_retry(
        client.chat.completions.create,
        model="auto:fast",
        temperature=0,
        messages=[{
            "role": "user",
            "content": (
                f"Question: {question}\n\nPassages:\n{listing}\n\n"
                f"Return the numbers of the {keep} most useful passages, best first, "
                "as a JSON list of integers. Return only the list."
            ),
        }],
    )
    try:
        ids = json.loads(resp.choices[0].message.content.strip())
        return [i for i in ids if isinstance(i, int) and 0 <= i < len(candidates)][:keep]
    except (json.JSONDecodeError, AttributeError):
        return list(range(min(keep, len(candidates))))  # fall back to vector order


# Use it in answer(): take 15 candidates, keep the best 5
# top = np.argsort(vecs @ q)[::-1][:15]
# best = rerank(question, [chunks[i]["text"] for i in top])
# top = [top[i] for i in best]

Hatua ya rerank inaongeza ombi moja la gumzo kwa kila swali. Ikiwa mipaka yako bure ni midogo, ruka hatua hii. Ugawaji bora wa vipande mara nyingi unasaidia zaidi kuliko hatua ya rerank.

Muktadha mrefu: wakati hauhitaji utafutaji

Leo, ofa 56 za modeli bure zina dirisha la muktadha la tokeni milioni 1 au zaidi. Ikiwa hati zako ni ndogo, unaweza kutuma zote kwenye maelekezo na kuruka faharasa ya vekta. Hii ni rahisi zaidi, lakini kila swali linatuma tokeni nyingi.

Modeli bure zenye madirisha makubwa zaidi ya muktadha
ModeliBure kwenyeMuktadhaVikomo vya bure (ofa bora)ZanaUoni
OpenAI: GPT-6 LunaElectronHub1.1M$0.25/week shared free creditsHapanaHapana
Gemini 3.8 FlashAPINex, Google AI Studio1M5 maombi/dakika
10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account
NdiyoNdiyo
Gemini 3.7 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
DeepSeek V4 FlashAMD Radeon Cloud, BazaarLink, ElectronHub1M10 maombi/dakika
50 maombi/siku
Shared rate limit
NdiyoHapana
DeepSeek V4 Flash 0731ElectronHub, Router91M10,000 credits/month (~$1), shared; account-anniversary resetHapanaHapana
Gemini 3.6 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
GLM 5.2Router91M10,000 credits/month (~$1), shared; account-anniversary resetHapanaHapana
Gemini 3.5 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
DeepSeek V4 ProAPINex, Router91M5 maombi/dakika
10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account
HapanaHapana
Nemotron 3 Ultra 550BGizmo, Kilo Gateway, LLMTR +3 zaidi1M200 maombi/siku
200 rpd
NdiyoHapana

Inaonyesha 10 kati ya 35 modeli za bure zinazolingana. Tazama modeli zote.

Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.

  • Tumia muktadha mrefu kwa hati chache, kwa maswali yanayohitaji maandishi yote, au kwa ripoti moja.
  • Tumia utafutaji kwa hati nyingi, kwa maswali ya mara kwa mara, au pale mipaka ya tokeni kwa dakika bure ni midogo.
  • Changanya: pata vipande 20 au zaidi na uvitume kwa modeli ya muktadha mrefu.

Maelekezo makubwa yanaweza kushindwa kwa mtoa huduma mwenye mipaka midogo ya tokeni kwa dakika. FreeLLMAPI kisha inajaribu mtoa huduma anayefuata. Ili kudhibiti modeli zipi zipokee maelekezo makubwa, tengeneza wasifu wa mnyororo wa akiba wenye modeli za muktadha mrefu pekee. Uitumie kama auto:<profile>.

Mipaka na kutatua matatizo

TatizoChanzoSuluhisho
Uingizaji unasimama kwa 429Familia ya embedding imefikia kikomo kwa watoa huduma woteTumia makundi makubwa zaidi, subiri Retry-After, ingiza kwa sehemu
Matokeo mabaya baada ya mabadilikoFaharasa ina vekta kutoka familia mbiliWeka embedding kwa vipande vyote tena kwa familia moja
Hitilafu kuhusu urefu wa ingizoKipande ni kirefu kuliko ingizo la juu la familiaTumia vipande vidogo zaidi
Jibu halizingatii vyanzoModeli inajibu kutokana na maarifa yake yenyeweTumia temperature=0 na maelekezo madhubuti ya mfumo
Hitilafu ya urefu wa muktadha kwenye jibuVipande vingi mno au virefu mnoPunguza k au tumia wasifu wa muktadha mrefu
Majibu yanabadilika kwa mtindoKipanga njia kimechagua modeli nyingineTuma kichwa cha X-Session-Id au bandika modeli

Viwango bure havina SLA. Modeli zenye nguvu zaidi hufikia mipaka yake ya kila siku kwanza, na mipaka mingi huwekwa upya saa sita usiku UTC. Jaribu mnyororo wako kwa modeli ambazo mnyororo wako utazitumia kweli.

Tumia kila moja ya hizi kwa ufunguo mmoja. FreeLLMAPI ni kipanga njia huria, unachoendesha mwenyewe. Kinaweka tabaka hizi zote za bure nyuma ya kituo kimoja cha OpenAI kinachooana, na kinahamia kwenye nyingine pale kikomo kinapofikiwa. Vinjari katalogi au anza sasa.

Maswali yanayoulizwa mara kwa mara

Kuna API bure ya RAG?

Ndiyo. Watoa huduma kadhaa wana viwango bure vya embedding na gumzo. FreeLLMAPI inahudumia zote mbili kupitia ufunguo mmoja unaoendana na OpenAI, pamoja na akiba kati ya watoa huduma.

Modeli bure ipi ya embedding ni bora kwa RAG?

Chagua familia ambayo ni bure kwa watoa huduma kadhaa na inalingana na lugha zako. Kwa maandishi ya lugha nyingi, bge-m3 ni chaguo la kawaida. Tumia familia moja kwa faharasa nzima.

Kuna API bure ya rerank?

Nyaraka za FreeLLMAPI hazielezi sehemu ya mwisho ya rerank. Unaweza kupanga upya kwa modeli ya gumzo ya haraka, kama inavyoonyeshwa kwenye ukurasa huu, au kuruka hatua ya rerank.

Je, nahitaji RAG ikiwa modeli bure ina muktadha wa tokeni milioni 1?

Si kwa hati chache ndogo. Kwa hati nyingi au maswali ya mara kwa mara, utafutaji unatuma tokeni chache kwa kila ombi na unabaki ndani ya mipaka bure.

Naweza kubadilisha modeli ya embedding baadaye?

Ndiyo, lakini lazima uweke embedding kwa vipande vyote tena. Vekta kutoka modeli mbili hazioani, kwa hiyo usizichanganye kwenye faharasa moja.

Miongozo inayohusiana

Vyanzo

Miongozo yote · Tazama modeli zote za bure