Practical overview
When it fits
Use Rag Cache when an application repeatedly asks an LLM identical or similar questions and you can operate the service alongside Redis and Qdrant.
Who it is for
- Developers integrating OpenAI or Anthropic query handling into an application.
- Teams looking to reuse responses to repeated or semantically similar LLM queries.
- Operators who want to inspect cache-hit and request metrics, including Prometheus-format output.
Documented uses
- Return a previously cached response for an identical query.
- Find and reuse a response for a semantically similar query.
- Forward a query to an LLM provider when neither cache path matches.
- Inspect application and cache metrics through HTTP endpoints.
Requirements & limits
The README describes a self-hosted setup requiring Python 3.11 or later, Docker and Docker Compose, and an OpenAI API key.
- The cache checks exact Redis matches before semantic Qdrant matches, then calls an LLM on a miss.
- The documented provider integrations are OpenAI and Anthropic; another provider is an Experimental adaptation.
- The README reports example latencies of 1.11 ms for an exact hit and 457.89 ms for a semantic hit; these are project-reported figures rather than guaranteed results.
- The service exposes query, health, metrics, Prometheus metrics, and interactive API documentation endpoints.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up the local caching service
Install Rag Cache, configure its OpenAI key, start Redis and Qdrant, and launch the application.
Before you start
- Python 3.11 or later
- Docker and Docker Compose
- An OpenAI API key
Steps
Create and activate a Python 3.11 virtual environment, then install dependencies from requirements.txt.
Copy .env.example to .env and add the OPENAI_API_KEY value.
Start the Redis and Qdrant services with Docker Compose.
Launch the FastAPI application with the documented Uvicorn command.
Expected outcome
The README describes the application running locally and provides a /health endpoint for checking its reported status.
Guide limitations
- This recipe covers local setup and launch; it does not establish compatibility with other environments.
- The documented prerequisites specify an OpenAI key; this setup recipe does not cover configuring Anthropic.
Send a query through the cache endpoint
Submit a query with caching enabled to the running service and inspect the returned cache information.
Before you start
- A running Rag Cache service
- An HTTP client such as curl
Steps
POST a JSON body containing a query and "use_cache": true to /api/v1/query.
Read the response's cache_info fields to see whether the request was an exact hit, semantic hit, or miss.
For a semantic-match example, submit a paraphrase of a previously cached query and inspect the reported similarity score.
Expected outcome
The endpoint returns the response and cache metadata; the documented examples distinguish Redis exact hits from Qdrant semantic hits.
Guide limitations
- The response reuse depends on a prior matching cached query; a cache miss is sent to an LLM provider.
- The documented semantic example reports a similarity score, but does not establish that all paraphrases will qualify as matches.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 18 October 2026
- Original website title
- zakariaf/RAG-Cache · Official homepage title · retained in its source language