Skip to content
#

semantic-caching

Here are 71 public repositories matching this topic...

Enterprise AI infrastructure platform: distributed vLLM inference, agentic workflows, hybrid multimodal RAG, CNN vision, QoS-tiered FastAPI gateway, guardrails and Prometheus/Grafana observability, fully Dockerized.

  • Updated Oct 16, 2026
  • Python
CacheVerifier

Research behind CacheVerifier (cacheverifier.com): verifier gating for semantic cache hits vs. static and adaptive thresholds, on ~210k requests across three public benchmarks plus real Twitter support traffic (4 datasets). Fine-tuning vs. off-the-shelf, label noise, drift, conformal risk control under shift, online ACI from randomized audits.

  • Updated Nov 8, 2026
  • Python

An intelligent gateway for Claude APIs that dynamically routes requests to the most cost-efficient model, caches responses, and escalates based on confidence signals — reducing LLM spend without sacrificing quality.

  • Updated May 6, 2026
  • Python

A retrieval-augmented generation pipeline in Python with a rigorous offline evaluation harness. Chunks and embeds documents, retrieves by vector similarity, and generates grounded answers — with pluggable LLM providers (including a deterministic local fake for tests) and metrics for retrieval quality and answer faithfulness. No API key required.

  • Updated Jul 20, 2026
  • Python

Add this topic to your repo

To associate your repository with the semantic-caching topic, visit your repo's landing page and select "manage topics."

Learn more