Nyumbani / Miongozo / LLM ya bure kwa LlamaIndex

LLM ya bure kwa LlamaIndex

LlamaIndex ina madarasa mawili ya API zinazoendana na OpenAI: OpenAILike kwa gumzo na OpenAILikeEmbedding kwa embeddings. Mwongozo huu unaelekeza yote mawili kwa FreeLLMAPI, ambayo inakusanya tabaka 45 za bure za watoa huduma nyuma ya ufunguo mmoja. Utajenga faharasa ya retrieval-augmented generation (RAG), kuiuliza na kuongeza wakala. Msimbo unashughulikia vikomo vya kasi.

Msimbo, majina ya modeli na lebo za katalogi ya moja kwa moja zinaonyeshwa kwa Kiingereza.

Sakinisha pakiti

Sakinisha FreeLLMAPI kisha ongeza funguo za bure kutoka kwa watoa huduma wawili au zaidi kwenye ukurasa wa Keys wa dashibodi. Nakili ufunguo wa umoja. Kisha sakinisha pakiti za LlamaIndex.

Sakinisha

curl -fsSL https://freellmapi.co/install.sh | bash

pip install -U llama-index-core \
  llama-index-llms-openai-like \
  llama-index-embeddings-openai-like

export FREELLMAPI_BASE_URL=http://localhost:3001/v1
export FREELLMAPI_API_KEY=<unified-key>

Usitumie darasa la kawaida la OpenAI kutoka llama-index-llms-openai na majina ya modeli ambayo OpenAI haiyajui. Darasa hilo huangalia majina ya modeli na ukubwa wa muktadha dhidi ya orodha ya modeli za OpenAI. OpenAILike haifanyi hivyo, kwa hiyo unaweka thamani hizi mwenyewe.

Sanidi LLM na modeli ya embedding

settings.py

import os
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding

BASE = os.environ["FREELLMAPI_BASE_URL"]
KEY = os.environ["FREELLMAPI_API_KEY"]

Settings.llm = OpenAILike(
    model="auto",
    api_base=BASE,
    api_key=KEY,
    is_chat_model=True,              # use /v1/chat/completions
    is_function_calling_model=True,  # needed for FunctionAgent
    context_window=32000,            # the smallest window in your chain
    max_retries=5,
    timeout=90,
)

Settings.embed_model = OpenAILikeEmbedding(
    model_name="bge-m3",  # one family, 1024 dimensions
    api_base=BASE,
    api_key=KEY,
    embed_batch_size=16,
    max_retries=10,
)
  • is_chat_model=True: bila hii, LlamaIndex hutuma maombi kwenye endpoint ya zamani ya completions.
  • context_window: LlamaIndex hutumia thamani hii kuamua vipande vingapi vilivyorejeshwa vinatoshea kwenye prompt. Kwa auto, kipanga njia kinaweza kuchagua modeli tofauti. Tumia dirisha dogo zaidi la muktadha kwenye mnyororo wako.
  • is_function_calling_model=True: iweke tu wakati mnyororo wako una modeli zenye tool calling.
  • embed_batch_size: kila kundi ni ombi moja. Makundi makubwa hutumia maombi machache kwa dakika.

Prompt ya RAG hubeba vipande kadhaa vilivyorejeshwa, kwa hiyo modeli ya gumzo inahitaji nafasi kwa ajili yake. Tengeneza wasifu wa mnyororo wa akiba wenye modeli za muktadha mrefu kwa RAG. Kisha weka model="auto:<profile>" na upandishe context_window hadi dirisha dogo zaidi kwenye wasifu huo. Modeli hizi za bure zina madirisha makubwa zaidi ya muktadha leo:

Modeli za bure zenye madirisha makubwa zaidi ya muktadha
ModeliBure kwenyeMuktadhaVikomo vya bure (ofa bora)ZanaUoni
OpenAI: GPT-6 LunaElectronHub1.1M$0.25/week shared free creditsHapanaHapana
Gemini 3.8 FlashAPINex, Google AI Studio1M5 maombi/dakika
10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account
NdiyoNdiyo
Gemini 3.7 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
DeepSeek V4 FlashAMD Radeon Cloud, BazaarLink, ElectronHub1M10 maombi/dakika
50 maombi/siku
Shared rate limit
NdiyoHapana
DeepSeek V4 Flash 0731ElectronHub, Router91M10,000 credits/month (~$1), shared; account-anniversary resetHapanaHapana
Gemini 3.6 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
GLM-5.2ModelScope, Router91M100 maombi/siku
Daily login: 200 credits + 50 for linked Alibaba Cloud; 24h expiry; shared; noncommercial experience
HapanaHapana
Gemini 3.5 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo

Inaonyesha 8 kati ya 115 modeli za bure zinazolingana. Tazama modeli zote.

Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.

Ili kuweka mtumiaji mmoja kwenye modeli moja, tuma kichwa cha X-Session-Id. OpenAILike inakubali default_headers, kwa mfano default_headers={"X-Session-Id": "user-42"}. FreeLLMAPI huweka kikao kwenye modeli ile ile kwa dakika 30. Hii hutoa majibu ya mtindo mmoja katika gumzo moja.

Jenga na uulize faharasa ya RAG

Hati hii husoma faili kutoka kwenye folda, hutengeneza embeddings, huhifadhi faharasa na kujibu swali. Iendeshe mara moja kujenga faharasa. Mara zinazofuata, hupakia faharasa iliyohifadhiwa na haiweki embeddings za faili tena.

rag.py

import os
import settings  # noqa: F401  (sets Settings.llm and Settings.embed_model)
from llama_index.core import (
    SimpleDirectoryReader,
    StorageContext,
    VectorStoreIndex,
    load_index_from_storage,
)

PERSIST = "./storage"


def load_index() -> VectorStoreIndex:
    if os.path.isdir(PERSIST):
        return load_index_from_storage(StorageContext.from_defaults(persist_dir=PERSIST))
    docs = SimpleDirectoryReader("./docs").load_data()
    index = VectorStoreIndex.from_documents(docs, show_progress=True)
    index.storage_context.persist(persist_dir=PERSIST)
    return index


if __name__ == "__main__":
    query_engine = load_index().as_query_engine(similarity_top_k=4, streaming=True)
    response = query_engine.query("How do I rotate the API keys?")
    response.print_response_stream()
    for node in response.source_nodes:
        print("\nSource:", node.metadata.get("file_name"), round(node.score or 0, 3))
Muhimu: weka familia moja ya embedding kwa faharasa nzima. FreeLLMAPI haibadilishi modeli ya embedding wakati wa failover. Inahama tu kati ya watoa huduma wanaotoa familia ile ile. Ukibadilisha familia, futa ./storage kisha ujenge faharasa upya.

Modeli hizi za embedding ni za bure katika katalogi ya leo. Safu ya dimensions inaonyesha ukubwa wa vekta wa kila familia.

ModeliMtoa hudumaVipimoIngizo la juu zaidiKikomo cha bure
Gemini Embedding
gemini-embedding-001
Google AI Studio30722048100 rpm · 1K req/day
Nemotron Embed VL 1B
nvidia/llama-nemotron-embed-vl-1b-v2
NVIDIA NIM20488192~40 rpm
Nemotron 3 Embed 1B (OR free)
nvidia/nemotron-3-embed-1b:free
OpenRouter20483276820 rpm · shared free daily cap
SEA-LION E5 Embedding 600M
aisingapore/SEA-LION-E5-Embedding-600M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 300M
aisingapore/SEA-LION-ModernBERT-Embedding-300M
SEA-LION10248192free · 10 rpm
SEA-LION ModernBERT Embedding 600M
aisingapore/SEA-LION-ModernBERT-Embedding-600M
SEA-LION10248192free · 10 rpm
BGE-M3
@cf/baai/bge-m3
Cloudflare Workers AI1024819210K neurons/day (shared)
EmbeddingGemma 300M
@cf/google/embeddinggemma-300m
Cloudflare Workers AI768204810K neurons/day (shared)
Qwen3 Embedding 0.6B
@cf/qwen/qwen3-embedding-0.6b
Cloudflare Workers AI1024819210K neurons/day (shared)
Cohere Embed v4
embed-v4.0
Cohere15361280001K calls/mo (shared w/ chat)
Gemini Embedding 2
gemini-embedding-2
Google AI Studio30728192free-tier rate limited
Nemotron 3 Embed 1B
nvidia/nemotron-3-embed-1b
NVIDIA NIM20484096~40 rpm
PLaMo Embedding 1B (CF)
@cf/pfnet/plamo-embedding-1b
Cloudflare Workers AI2048409610K neurons/day (shared)
BGE Small EN v1.5 (CF)
@cf/baai/bge-small-en-v1.5
Cloudflare Workers AI38451210K neurons/day (shared)
BGE Base EN v1.5 (CF)
@cf/baai/bge-base-en-v1.5
Cloudflare Workers AI76851210K neurons/day (shared)
BGE Large EN v1.5 (CF)
@cf/baai/bge-large-en-v1.5
Cloudflare Workers AI102451210K neurons/day (shared)
Mistral Embed
mistral-embed
Mistral10248192free mode · monthly cap (shared)
Codestral Embed
codestral-embed
Mistral15368192free mode · monthly cap (shared)
Llama Text Embed v2 (Pinecone)
llama-text-embed-v2
Pinecone102420485M tokens/mo per model
Multilingual E5 Large (Pinecone)
multilingual-e5-large
Pinecone10245075M tokens/mo per model
Nemotron 3 Embed 1B (Speka)
nvidia/nemotron-3-embed-1b
Speka20488192$1/month shared across Speka models
OpenAI: Text Embedding 3 Small (ElectronHub)
text-embedding-3-small
ElectronHub15368192$0.25/week shared free credits
OpenAI: Text Embedding 3 Large (ElectronHub)
text-embedding-3-large
ElectronHub30728192$0.25/week shared free credits
Nemotron 3 Embed 8B (Inferbase)
nvidia-nemotron-3-embed-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 0.6B (Inferbase)
qwen3-embedding-0-6b
Inferbase102481925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 4B (Inferbase)
qwen3-embedding-4b
Inferbase256081925,000 requests/month; 100 requests and $0.50 usage/day shared
Qwen 3 Embedding 8B (Inferbase)
qwen3-embedding-8b
Inferbase409681925,000 requests/month; 100 requests and $0.50 usage/day shared
LFM2.5 Embedding 350M (LLMTR)
liquid/lfm-2.5-embedding-350m-free
LLMTR1024512Ongoing zero-price route; provider terms apply
BGE-M3 (SEA-LION)
BAAI/bge-m3
SEA-LION10248192free · 10 rpm
Cohere Embed English v3
embed-english-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Qwen3 Embedding 8B (OVH)
Qwen3-Embedding-8B
OVH AI Endpoints409632768free · anonymous 2 req/min per IP
BGE-M3 (OVH)
bge-m3
OVH AI Endpoints10248192free · anonymous 2 req/min per IP
BGE Multilingual Gemma2 (OVH)
bge-multilingual-gemma2
OVH AI Endpoints35848192free · anonymous 2 req/min per IP
Mistral: Codestral Embed (ElectronHub)
codestral-embed
ElectronHub15368192$0.25/week shared free credits
Mistral: Mistral Embed (ElectronHub)
mistral-embed
ElectronHub10248192$0.25/week shared free credits
Google: Gemini Embedding 2 (ElectronHub)
gemini-embedding-2
ElectronHub30728192$0.25/week shared free credits
Google: Gemini Embedding 001 (ElectronHub)
gemini-embedding-001
ElectronHub30722048$0.25/week shared free credits
OpenAI: Text Embedding Ada 002 (ElectronHub)
text-embedding-ada-002
ElectronHub15368192$0.25/week shared free credits
Cohere Embed English Light v3
embed-english-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)
Cohere Embed Multilingual v3
embed-multilingual-v3.0
Cohere10245121K calls/mo (shared w/ chat)
Cohere Embed Multilingual Light v3
embed-multilingual-light-v3.0
Cohere3845121K calls/mo (shared w/ chat)

Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.

Ongeza wakala kwa FunctionAgent

FunctionAgent hutumia tool calling. Unaweza kumpa injini ya maswali kama zana na kuongeza kazi zako za Python.

agent.py

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool
from llama_index.core import Settings
from rag import load_index

docs_tool = QueryEngineTool.from_defaults(
    query_engine=load_index().as_query_engine(similarity_top_k=4),
    name="project_docs",
    description="Answers questions about the project documentation.",
)

def open_tickets(team: str) -> int:
    """Return the number of open tickets for a team."""
    return {"billing": 4, "platform": 11}.get(team, 0)

agent = FunctionAgent(
    tools=[docs_tool, open_tickets],
    llm=Settings.llm,
    system_prompt="Use the tools. Cite the document name in each answer.",
)

async def main():
    result = await agent.run("How many platform tickets are open, and how do we rotate keys?")
    print(result)

asyncio.run(main())

Kwa kazi ya wakala, chagua modeli zenye tool calling. Tengeneza wasifu wa mnyororo wa akiba wenye modeli hizi pekee, kisha utumie auto:<profile> kama jina la modeli.

Modeli za bure zenye tool calling na muktadha wa 128K+
ModeliBure kwenyeMuktadhaVikomo vya bure (ofa bora)ZanaUoni
GLM 5.3UnoRouter1M1 maombi/dakika
1 req/min per model
NdiyoHapana
GLM 5.3 FlashNVIDIA NIM, UnoRouter1M40 maombi/dakika
40 RPM
NdiyoNdiyo
Gemini 3.8 FlashGoogle AI Studio1MRecurring provider quotaNdiyoNdiyo
Qwen3.8 Flash NextAMD Radeon Cloud262KRecurring daily account allowanceNdiyoHapana
Qwen3.8 FlashElectronHub1M$0.25/week shared recurring creditNdiyoNdiyo
Gemini 3.7 FlashGoogle AI Studio1M10 maombi/dakika
20 maombi/siku
250,000 tokeni/dakika
~3M
NdiyoNdiyo
GPT-5.6 LunaElectronHub1M$0.25/week shared recurring creditNdiyoNdiyo
Dots3 Note PreviewKilo Gateway, OpenRouter512K20 maombi/dakika
50 maombi/siku
50 req/day account-wide (1,000 after a $10 purchase)
NdiyoNdiyo
DeepSeek V4 FlashAMD Radeon Cloud, BazaarLink, ElectronHub1M10 maombi/dakika
50 maombi/siku
Shared rate limit
NdiyoHapana
Qwen3.8 27BElectronHub, Groq, OVH AI Endpoints262K30 maombi/dakika
250 maombi/siku
70,000 tokeni/dakika
~6M
NdiyoNdiyo

Inaonyesha 10 kati ya 109 modeli za bure zinazolingana. Tazama modeli zote.

Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.

Weka uorodheshaji ndani ya vikomo vya bure

Uorodheshaji hutuma maombi mengi ya embedding kwa muda mfupi. Tabaka za bure za embedding zina vikomo vya chini. Kwa mfano, baadhi huruhusu maombi 10 kwa dakika na nyingine huhesabu miito kwa mwezi. Panga uorodheshaji wa kwanza kabla ya kuanza.

  • Tumia vipande vikubwa. Vipande vichache humaanisha vekta chache na maombi machache. Ukubwa wa kipande karibu tokeni 1,024 ni mwanzo mzuri.
  • Tumia makundi. Weka embed_batch_size kuwa 16 au zaidi. Ombi moja kisha hubeba vipande vingi.
  • Hifadhi faharasa. Hifadhi faharasa baada ya kuendesha mara ya kwanza. Usiweke embeddings za faili zile zile tena.
  • Weka majaribio upya yawashwe. OpenAILikeEmbedding hujaribu tena hadi mara 10 kwa chaguo-msingi. Weka thamani hii kwa kazi ndefu za uorodheshaji.
  • Orodhesha kwa sehemu. Kwa mafaili elfu kadhaa, orodhesha folda moja kwa wakati.

Maswali ni nafuu kuliko uorodheshaji. Kila swali hutumia ombi moja la embedding na ombi moja la gumzo. Ombi la gumzo lenye vipande vinne vya tokeni 1,024 hutuma takriban tokeni 5,000. Tabaka za bure zenye vikomo vya chini vya tokeni kwa dakika zinaweza kukataa hili. FreeLLMAPI kisha hujaribu mtoa huduma anayefuata.

Utatuzi wa matatizo

TatizoSababuSuluhisho
Unknown model 'auto'Ulitumia darasa la OpenAITumia OpenAILike
Maombi yanaenda kwenye /v1/completionsis_chat_model haijawekwaWeka is_chat_model=True
Wakala haitumii zanais_function_calling_model imezimwa, au modeli haina tool callingIweke True kisha utumie wasifu wenye zana
Hitilafu 429 wakati wa uorodheshajiFamilia ya embedding imefikia kikomo kwa watoa huduma woteTumia makundi makubwa zaidi, subiri Retry-After kisha uendeshe tena
Majibu mabaya baada ya kubadilisha modeli ya embeddingVekta kutoka familia mbili kwenye faharasa mojaFuta faharasa iliyohifadhiwa kisha ujenge tena
Hitilafu ya urefu wa muktadhacontext_window ni kubwa kuliko modeli iliyojibuWeka thamani ndogo zaidi au tumia wasifu wa muktadha mrefu

Ili kuona mtoa huduma aliyejibu ombi, tumia dashibodi ya FreeLLMAPI. Ukurasa wa Analytics unaonyesha maombi, muda wa kujibu na hitilafu kwa kila mtoa huduma na modeli. Kila jibu pia lina kichwa cha X-Routed-Via. Ikiwa kipanga njia kilibadilisha mtoa huduma, kichwa cha X-Fallback-Trail kinataja kila jaribio lililoshindwa na sababu yake.

Tumia kila moja ya hizi kwa ufunguo mmoja. FreeLLMAPI ni kipanga njia huria, unachoendesha mwenyewe. Kinaweka tabaka hizi zote za bure nyuma ya kituo kimoja cha OpenAI kinachooana, na kinahamia kwenye nyingine pale kikomo kinapofikiwa. Vinjari katalogi au anza sasa.

Maswali yanayoulizwa mara kwa mara

Je, ninaweza kutumia LlamaIndex na LLM ya bure?

Ndiyo. Tumia OpenAILike na api_base iliyowekwa kuwa URL inayoendana na OpenAI. FreeLLMAPI hutoa URL moja kwa tabaka nyingi za bure za watoa huduma. Kila tabaka lina vikomo vyake.

Kuna tofauti gani kati ya OpenAI na OpenAILike katika LlamaIndex?

Darasa la OpenAI huangalia majina ya modeli dhidi ya modeli za OpenAI. OpenAILike hukubali jina lolote la modeli. Wewe mwenyewe unaweka dirisha la muktadha na bendera za gumzo na function-calling.

Modeli ipi ya bure ya embedding ni bora kwa LlamaIndex?

Chagua familia yenye maombi ya bure ya kutosha kwa uorodheshaji wako kisha uitumie kwa faharasa nzima. Familia ya bge-m3 ni ya bure kwa watoa huduma kadhaa, kwa hiyo failover inaweza kuhama kati yao.

Je, FunctionAgent inafanya kazi na modeli za bure?

Ndiyo, ikiwa modeli inaunga mkono tool calling na umeweka is_function_calling_model=True. Tumia wasifu wa mnyororo wa akiba wenye modeli za tool calling pekee.

Ninawezaje kuepuka vikomo vya kasi ninapojenga faharasa ya LlamaIndex?

Tumia vipande vikubwa, weka embed_batch_size kuwa 16 au zaidi, hifadhi faharasa na orodhesha folda kubwa kwa sehemu.

Miongozo inayohusiana

Vyanzo

Miongozo yote · Tazama modeli zote za bure