The economics
RAG workloads spend your fleet's GPU time re-reading the same documents on every question. Engram Smart CAG onboards documents once and serves every answer from the saved memory. Same model, same answers, measured head-to-head; the difference is the waste removed from your GPU spend.
See the work you'd save ↓One deliberately hard benchmark, identical on both sides: expert-written questions about scientific papers, the right paper hidden among 1,000, graded strictly against expert answers. Same answers, a fraction of the work.
Fleet scale
A real concurrency ramp on one 4-GPU serving tier — same model, same retrieval on both sides. Resident memories skip the re-read, so throughput keeps climbing while RAG flattens under its own re-prefill.
Throughput per GPU · 24 concurrent
Time to first token · 24 concurrent
GPU cost / 1k queries · 24 concurrent
Measured on one 4-GPU serving tier (Qwen3-30B FP8, TP=4), server-timed time-to-first-token and latency, GPU priced at $4.60/hr, 1–24 concurrent queries. RAG runs in the multi-tenant regime — a churned prefix cache, the realistic fleet operating point — so it pays its re-prefill on every request while cartridge memories stay resident.
Payoff calculator
Everything traces to the measured benchmark: onboarding in one forward pass per document, storage after compression and auto-tiering, and GPU savings — the measured fleet-scale $/query gap vs same-model RAG — set by how much context your RAG re-reads per question and the model you serve.
Context re-read per question today
chunks + history + boilerplate your clients re-send every query
Model you serve
bigger models make every avoided re-read worth more
Fixed cost, scaling savings
Saved memories are a flat monthly line item — our 4× memory compression plus automatic cloud tiering keeps storage costs low and constant while the GPU savings grow with volume: every question asked is savings increase. The busier the library, the further the savings pull ahead of the flat storage line — the calculator above nets the two for your numbers.
How it stacks up
Same model, same grounding, same answers — the difference is doing the reading once instead of on every query.
| Engram Smart CAG | Standard RAG | |
|---|---|---|
| GPU work per query | Dozens of tokens, not thousands | Re-reads context every query |
| Latency (time-to-first-token) | Low & stable — 28 ms, ~90 ms even at 24 concurrent | Grows with context |
| Grounded & sourced | Yes — quality tie with RAG | Yes |
| Setup / onboarding | One forward pass per doc — no training | Index once |
| Fresh on updates | Re-onboard the doc once (seconds) | Instant re-index |
| Storage footprint | Large — the trade-off: 4× compressed and tiered, measured | Small index |
| Private on your cloud | Yes — your VPC, your GPUs | Depends |
