LLM ya bure kwa LlamaIndex
LlamaIndex ina madarasa mawili ya API zinazoendana na OpenAI: OpenAILike kwa gumzo na OpenAILikeEmbedding kwa embeddings. Mwongozo huu unaelekeza yote mawili kwa FreeLLMAPI, ambayo inakusanya tabaka 45 za bure za watoa huduma nyuma ya ufunguo mmoja. Utajenga faharasa ya retrieval-augmented generation (RAG), kuiuliza na kuongeza wakala. Msimbo unashughulikia vikomo vya kasi.
Msimbo, majina ya modeli na lebo za katalogi ya moja kwa moja zinaonyeshwa kwa Kiingereza.
Sakinisha pakiti
Sakinisha FreeLLMAPI kisha ongeza funguo za bure kutoka kwa watoa huduma wawili au zaidi kwenye ukurasa wa Keys wa dashibodi. Nakili ufunguo wa umoja. Kisha sakinisha pakiti za LlamaIndex.
Sakinisha
curl -fsSL https://freellmapi.co/install.sh | bash
pip install -U llama-index-core \
llama-index-llms-openai-like \
llama-index-embeddings-openai-like
export FREELLMAPI_BASE_URL=http://localhost:3001/v1
export FREELLMAPI_API_KEY=<unified-key>Usitumie darasa la kawaida la OpenAI kutoka llama-index-llms-openai na majina ya modeli ambayo OpenAI haiyajui. Darasa hilo huangalia majina ya modeli na ukubwa wa muktadha dhidi ya orodha ya modeli za OpenAI. OpenAILike haifanyi hivyo, kwa hiyo unaweka thamani hizi mwenyewe.
Sanidi LLM na modeli ya embedding
settings.py
import os
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
BASE = os.environ["FREELLMAPI_BASE_URL"]
KEY = os.environ["FREELLMAPI_API_KEY"]
Settings.llm = OpenAILike(
model="auto",
api_base=BASE,
api_key=KEY,
is_chat_model=True, # use /v1/chat/completions
is_function_calling_model=True, # needed for FunctionAgent
context_window=32000, # the smallest window in your chain
max_retries=5,
timeout=90,
)
Settings.embed_model = OpenAILikeEmbedding(
model_name="bge-m3", # one family, 1024 dimensions
api_base=BASE,
api_key=KEY,
embed_batch_size=16,
max_retries=10,
)- is_chat_model=True: bila hii, LlamaIndex hutuma maombi kwenye endpoint ya zamani ya completions.
- context_window: LlamaIndex hutumia thamani hii kuamua vipande vingapi vilivyorejeshwa vinatoshea kwenye prompt. Kwa
auto, kipanga njia kinaweza kuchagua modeli tofauti. Tumia dirisha dogo zaidi la muktadha kwenye mnyororo wako. - is_function_calling_model=True: iweke tu wakati mnyororo wako una modeli zenye tool calling.
- embed_batch_size: kila kundi ni ombi moja. Makundi makubwa hutumia maombi machache kwa dakika.
Prompt ya RAG hubeba vipande kadhaa vilivyorejeshwa, kwa hiyo modeli ya gumzo inahitaji nafasi kwa ajili yake. Tengeneza wasifu wa mnyororo wa akiba wenye modeli za muktadha mrefu kwa RAG. Kisha weka model="auto:<profile>" na upandishe context_window hadi dirisha dogo zaidi kwenye wasifu huo. Modeli hizi za bure zina madirisha makubwa zaidi ya muktadha leo:
| Modeli | Bure kwenye | Muktadha | Vikomo vya bure (ofa bora) | Zana | Uoni |
|---|---|---|---|---|---|
| OpenAI: GPT-6 Luna | ElectronHub | 1.1M | $0.25/week shared free credits | Hapana | Hapana |
| Gemini 3.8 Flash | APINex, Google AI Studio | 1M | 5 maombi/dakika 10M weighted tokens/day shared across APINex free routes; 3x token multiplier; 5 requests/minute/account | Ndiyo | Ndiyo |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 maombi/dakika 50 maombi/siku Shared rate limit | Ndiyo | Hapana |
| DeepSeek V4 Flash 0731 | ElectronHub, Router9 | 1M | 10,000 credits/month (~$1), shared; account-anniversary reset | Hapana | Hapana |
| Gemini 3.6 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| GLM-5.2 | ModelScope, Router9 | 1M | 100 maombi/siku Daily login: 200 credits + 50 for linked Alibaba Cloud; 24h expiry; shared; noncommercial experience | Hapana | Hapana |
| Gemini 3.5 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
Inaonyesha 8 kati ya 115 modeli za bure zinazolingana. Tazama modeli zote.
Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.
Ili kuweka mtumiaji mmoja kwenye modeli moja, tuma kichwa cha X-Session-Id. OpenAILike inakubali default_headers, kwa mfano default_headers={"X-Session-Id": "user-42"}. FreeLLMAPI huweka kikao kwenye modeli ile ile kwa dakika 30. Hii hutoa majibu ya mtindo mmoja katika gumzo moja.
Jenga na uulize faharasa ya RAG
Hati hii husoma faili kutoka kwenye folda, hutengeneza embeddings, huhifadhi faharasa na kujibu swali. Iendeshe mara moja kujenga faharasa. Mara zinazofuata, hupakia faharasa iliyohifadhiwa na haiweki embeddings za faili tena.
rag.py
import os
import settings # noqa: F401 (sets Settings.llm and Settings.embed_model)
from llama_index.core import (
SimpleDirectoryReader,
StorageContext,
VectorStoreIndex,
load_index_from_storage,
)
PERSIST = "./storage"
def load_index() -> VectorStoreIndex:
if os.path.isdir(PERSIST):
return load_index_from_storage(StorageContext.from_defaults(persist_dir=PERSIST))
docs = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(docs, show_progress=True)
index.storage_context.persist(persist_dir=PERSIST)
return index
if __name__ == "__main__":
query_engine = load_index().as_query_engine(similarity_top_k=4, streaming=True)
response = query_engine.query("How do I rotate the API keys?")
response.print_response_stream()
for node in response.source_nodes:
print("\nSource:", node.metadata.get("file_name"), round(node.score or 0, 3))./storage kisha ujenge faharasa upya.Modeli hizi za embedding ni za bure katika katalogi ya leo. Safu ya dimensions inaonyesha ukubwa wa vekta wa kila familia.
| Modeli | Mtoa huduma | Vipimo | Ingizo la juu zaidi | Kikomo cha bure |
|---|---|---|---|---|
Gemini Embeddinggemini-embedding-001 | Google AI Studio | 3072 | 2048 | 100 rpm · 1K req/day |
Nemotron Embed VL 1Bnvidia/llama-nemotron-embed-vl-1b-v2 | NVIDIA NIM | 2048 | 8192 | ~40 rpm |
Nemotron 3 Embed 1B (OR free)nvidia/nemotron-3-embed-1b:free | OpenRouter | 2048 | 32768 | 20 rpm · shared free daily cap |
SEA-LION E5 Embedding 600Maisingapore/SEA-LION-E5-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 300Maisingapore/SEA-LION-ModernBERT-Embedding-300M | SEA-LION | 1024 | 8192 | free · 10 rpm |
SEA-LION ModernBERT Embedding 600Maisingapore/SEA-LION-ModernBERT-Embedding-600M | SEA-LION | 1024 | 8192 | free · 10 rpm |
BGE-M3@cf/baai/bge-m3 | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
EmbeddingGemma 300M@cf/google/embeddinggemma-300m | Cloudflare Workers AI | 768 | 2048 | 10K neurons/day (shared) |
Qwen3 Embedding 0.6B@cf/qwen/qwen3-embedding-0.6b | Cloudflare Workers AI | 1024 | 8192 | 10K neurons/day (shared) |
Cohere Embed v4embed-v4.0 | Cohere | 1536 | 128000 | 1K calls/mo (shared w/ chat) |
Gemini Embedding 2gemini-embedding-2 | Google AI Studio | 3072 | 8192 | free-tier rate limited |
Nemotron 3 Embed 1Bnvidia/nemotron-3-embed-1b | NVIDIA NIM | 2048 | 4096 | ~40 rpm |
PLaMo Embedding 1B (CF)@cf/pfnet/plamo-embedding-1b | Cloudflare Workers AI | 2048 | 4096 | 10K neurons/day (shared) |
BGE Small EN v1.5 (CF)@cf/baai/bge-small-en-v1.5 | Cloudflare Workers AI | 384 | 512 | 10K neurons/day (shared) |
BGE Base EN v1.5 (CF)@cf/baai/bge-base-en-v1.5 | Cloudflare Workers AI | 768 | 512 | 10K neurons/day (shared) |
BGE Large EN v1.5 (CF)@cf/baai/bge-large-en-v1.5 | Cloudflare Workers AI | 1024 | 512 | 10K neurons/day (shared) |
Mistral Embedmistral-embed | Mistral | 1024 | 8192 | free mode · monthly cap (shared) |
Codestral Embedcodestral-embed | Mistral | 1536 | 8192 | free mode · monthly cap (shared) |
Llama Text Embed v2 (Pinecone)llama-text-embed-v2 | Pinecone | 1024 | 2048 | 5M tokens/mo per model |
Multilingual E5 Large (Pinecone)multilingual-e5-large | Pinecone | 1024 | 507 | 5M tokens/mo per model |
Nemotron 3 Embed 1B (Speka)nvidia/nemotron-3-embed-1b | Speka | 2048 | 8192 | $1/month shared across Speka models |
OpenAI: Text Embedding 3 Small (ElectronHub)text-embedding-3-small | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
OpenAI: Text Embedding 3 Large (ElectronHub)text-embedding-3-large | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Nemotron 3 Embed 8B (Inferbase)nvidia-nemotron-3-embed-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 0.6B (Inferbase)qwen3-embedding-0-6b | Inferbase | 1024 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 4B (Inferbase)qwen3-embedding-4b | Inferbase | 2560 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
Qwen 3 Embedding 8B (Inferbase)qwen3-embedding-8b | Inferbase | 4096 | 8192 | 5,000 requests/month; 100 requests and $0.50 usage/day shared |
LFM2.5 Embedding 350M (LLMTR)liquid/lfm-2.5-embedding-350m-free | LLMTR | 1024 | 512 | Ongoing zero-price route; provider terms apply |
BGE-M3 (SEA-LION)BAAI/bge-m3 | SEA-LION | 1024 | 8192 | free · 10 rpm |
Cohere Embed English v3embed-english-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Qwen3 Embedding 8B (OVH)Qwen3-Embedding-8B | OVH AI Endpoints | 4096 | 32768 | free · anonymous 2 req/min per IP |
BGE-M3 (OVH)bge-m3 | OVH AI Endpoints | 1024 | 8192 | free · anonymous 2 req/min per IP |
BGE Multilingual Gemma2 (OVH)bge-multilingual-gemma2 | OVH AI Endpoints | 3584 | 8192 | free · anonymous 2 req/min per IP |
Mistral: Codestral Embed (ElectronHub)codestral-embed | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Mistral: Mistral Embed (ElectronHub)mistral-embed | ElectronHub | 1024 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 2 (ElectronHub)gemini-embedding-2 | ElectronHub | 3072 | 8192 | $0.25/week shared free credits |
Google: Gemini Embedding 001 (ElectronHub)gemini-embedding-001 | ElectronHub | 3072 | 2048 | $0.25/week shared free credits |
OpenAI: Text Embedding Ada 002 (ElectronHub)text-embedding-ada-002 | ElectronHub | 1536 | 8192 | $0.25/week shared free credits |
Cohere Embed English Light v3embed-english-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual v3embed-multilingual-v3.0 | Cohere | 1024 | 512 | 1K calls/mo (shared w/ chat) |
Cohere Embed Multilingual Light v3embed-multilingual-light-v3.0 | Cohere | 384 | 512 | 1K calls/mo (shared w/ chat) |
Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.
Ongeza wakala kwa FunctionAgent
FunctionAgent hutumia tool calling. Unaweza kumpa injini ya maswali kama zana na kuongeza kazi zako za Python.
agent.py
import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool
from llama_index.core import Settings
from rag import load_index
docs_tool = QueryEngineTool.from_defaults(
query_engine=load_index().as_query_engine(similarity_top_k=4),
name="project_docs",
description="Answers questions about the project documentation.",
)
def open_tickets(team: str) -> int:
"""Return the number of open tickets for a team."""
return {"billing": 4, "platform": 11}.get(team, 0)
agent = FunctionAgent(
tools=[docs_tool, open_tickets],
llm=Settings.llm,
system_prompt="Use the tools. Cite the document name in each answer.",
)
async def main():
result = await agent.run("How many platform tickets are open, and how do we rotate keys?")
print(result)
asyncio.run(main())Kwa kazi ya wakala, chagua modeli zenye tool calling. Tengeneza wasifu wa mnyororo wa akiba wenye modeli hizi pekee, kisha utumie auto:<profile> kama jina la modeli.
| Modeli | Bure kwenye | Muktadha | Vikomo vya bure (ofa bora) | Zana | Uoni |
|---|---|---|---|---|---|
| GLM 5.3 | UnoRouter | 1M | 1 maombi/dakika 1 req/min per model | Ndiyo | Hapana |
| GLM 5.3 Flash | NVIDIA NIM, UnoRouter | 1M | 40 maombi/dakika 40 RPM | Ndiyo | Ndiyo |
| Gemini 3.8 Flash | Google AI Studio | 1M | Recurring provider quota | Ndiyo | Ndiyo |
| Qwen3.8 Flash Next | AMD Radeon Cloud | 262K | Recurring daily account allowance | Ndiyo | Hapana |
| Qwen3.8 Flash | ElectronHub | 1M | $0.25/week shared recurring credit | Ndiyo | Ndiyo |
| Gemini 3.7 Flash | Google AI Studio | 1M | 10 maombi/dakika 20 maombi/siku 250,000 tokeni/dakika ~3M | Ndiyo | Ndiyo |
| GPT-5.6 Luna | ElectronHub | 1M | $0.25/week shared recurring credit | Ndiyo | Ndiyo |
| Dots3 Note Preview | Kilo Gateway, OpenRouter | 512K | 20 maombi/dakika 50 maombi/siku 50 req/day account-wide (1,000 after a $10 purchase) | Ndiyo | Ndiyo |
| DeepSeek V4 Flash | AMD Radeon Cloud, BazaarLink, ElectronHub | 1M | 10 maombi/dakika 50 maombi/siku Shared rate limit | Ndiyo | Hapana |
| Qwen3.8 27B | ElectronHub, Groq, OVH AI Endpoints | 262K | 30 maombi/dakika 250 maombi/siku 70,000 tokeni/dakika ~6M | Ndiyo | Ndiyo |
Inaonyesha 10 kati ya 109 modeli za bure zinazolingana. Tazama modeli zote.
Data ya moja kwa moja kutoka katalogi ya FreeLLMAPI (toleo 2026.11.19, imekaguliwa 2026-11-19). Vikomo vya bure hubadilika mara kwa mara. Katalogi hukaguliwa kila siku.
Weka uorodheshaji ndani ya vikomo vya bure
Uorodheshaji hutuma maombi mengi ya embedding kwa muda mfupi. Tabaka za bure za embedding zina vikomo vya chini. Kwa mfano, baadhi huruhusu maombi 10 kwa dakika na nyingine huhesabu miito kwa mwezi. Panga uorodheshaji wa kwanza kabla ya kuanza.
- Tumia vipande vikubwa. Vipande vichache humaanisha vekta chache na maombi machache. Ukubwa wa kipande karibu tokeni 1,024 ni mwanzo mzuri.
- Tumia makundi. Weka
embed_batch_sizekuwa 16 au zaidi. Ombi moja kisha hubeba vipande vingi. - Hifadhi faharasa. Hifadhi faharasa baada ya kuendesha mara ya kwanza. Usiweke embeddings za faili zile zile tena.
- Weka majaribio upya yawashwe.
OpenAILikeEmbeddinghujaribu tena hadi mara 10 kwa chaguo-msingi. Weka thamani hii kwa kazi ndefu za uorodheshaji. - Orodhesha kwa sehemu. Kwa mafaili elfu kadhaa, orodhesha folda moja kwa wakati.
Maswali ni nafuu kuliko uorodheshaji. Kila swali hutumia ombi moja la embedding na ombi moja la gumzo. Ombi la gumzo lenye vipande vinne vya tokeni 1,024 hutuma takriban tokeni 5,000. Tabaka za bure zenye vikomo vya chini vya tokeni kwa dakika zinaweza kukataa hili. FreeLLMAPI kisha hujaribu mtoa huduma anayefuata.
Utatuzi wa matatizo
| Tatizo | Sababu | Suluhisho |
|---|---|---|
Unknown model 'auto' | Ulitumia darasa la OpenAI | Tumia OpenAILike |
Maombi yanaenda kwenye /v1/completions | is_chat_model haijawekwa | Weka is_chat_model=True |
| Wakala haitumii zana | is_function_calling_model imezimwa, au modeli haina tool calling | Iweke True kisha utumie wasifu wenye zana |
| Hitilafu 429 wakati wa uorodheshaji | Familia ya embedding imefikia kikomo kwa watoa huduma wote | Tumia makundi makubwa zaidi, subiri Retry-After kisha uendeshe tena |
| Majibu mabaya baada ya kubadilisha modeli ya embedding | Vekta kutoka familia mbili kwenye faharasa moja | Futa faharasa iliyohifadhiwa kisha ujenge tena |
| Hitilafu ya urefu wa muktadha | context_window ni kubwa kuliko modeli iliyojibu | Weka thamani ndogo zaidi au tumia wasifu wa muktadha mrefu |
Ili kuona mtoa huduma aliyejibu ombi, tumia dashibodi ya FreeLLMAPI. Ukurasa wa Analytics unaonyesha maombi, muda wa kujibu na hitilafu kwa kila mtoa huduma na modeli. Kila jibu pia lina kichwa cha X-Routed-Via. Ikiwa kipanga njia kilibadilisha mtoa huduma, kichwa cha X-Fallback-Trail kinataja kila jaribio lililoshindwa na sababu yake.
Maswali yanayoulizwa mara kwa mara
Je, ninaweza kutumia LlamaIndex na LLM ya bure?
Ndiyo. Tumia OpenAILike na api_base iliyowekwa kuwa URL inayoendana na OpenAI. FreeLLMAPI hutoa URL moja kwa tabaka nyingi za bure za watoa huduma. Kila tabaka lina vikomo vyake.
Kuna tofauti gani kati ya OpenAI na OpenAILike katika LlamaIndex?
Darasa la OpenAI huangalia majina ya modeli dhidi ya modeli za OpenAI. OpenAILike hukubali jina lolote la modeli. Wewe mwenyewe unaweka dirisha la muktadha na bendera za gumzo na function-calling.
Modeli ipi ya bure ya embedding ni bora kwa LlamaIndex?
Chagua familia yenye maombi ya bure ya kutosha kwa uorodheshaji wako kisha uitumie kwa faharasa nzima. Familia ya bge-m3 ni ya bure kwa watoa huduma kadhaa, kwa hiyo failover inaweza kuhama kati yao.
Je, FunctionAgent inafanya kazi na modeli za bure?
Ndiyo, ikiwa modeli inaunga mkono tool calling na umeweka is_function_calling_model=True. Tumia wasifu wa mnyororo wa akiba wenye modeli za tool calling pekee.
Ninawezaje kuepuka vikomo vya kasi ninapojenga faharasa ya LlamaIndex?
Tumia vipande vikubwa, weka embed_batch_size kuwa 16 au zaidi, hifadhi faharasa na orodhesha folda kubwa kwa sehemu.
Miongozo inayohusiana
- API ya LLM bure kwa RAG
- API ya bure ya embeddings
- API ya LLM ya bure kwa LangChain
- LLM API ya bure kwa Python
- API ya LLM bure yenye muktadha wa tokeni milioni 1
Vyanzo
- LlamaIndex: llama-index-llms-openai-like README (imekaguliwa 20 November 2026)
- LlamaIndex: llama-index-embeddings-openai-like source (imekaguliwa 20 November 2026)
- LlamaIndex: FunctionAgent (agent workflow) source (imekaguliwa 20 November 2026)
- FreeLLMAPI docs: REST API (embeddings families, routing strategies) (imekaguliwa 20 November 2026)