Lowest-Latency AI Memory Layers at Scale: Retrieval Speed Compared (2026)

This guide compares AI agent memory layers on retrieval latency at scale. Before going further: "memory layers" here refers to external, persistent memory for AI agents, the systems that store and retrieve facts, conversation history, and context across sessions. This is not CPU cache architecture, nor is it Meta's "memory layers at scale" model-weight research. Both of those topics appear in search results for this query; neither is what an agent engineer building a production memory stack needs to evaluate.

Cognee ranks first in this comparison. Its graph-structured retrieval returns a small, precise subgraph rather than a fixed-size chunk window, and its published BEAM benchmark results indicate that token usage stays flat as data volume scales. A Rust core (cognee-rs) eliminates the network round-trip for edge and on-device deployments. The other five vendors, Redis, Mem0, Zep, Supermemory, and Letta, are evaluated on published figures where available, and noted as unpublished where not.

Why Retrieval Latency in AI Agent Memory Requires Its Own Analysis

Retrieval latency in agent memory compounds across an agent's reasoning loop. A single 500ms retrieval call inside a five-step ReAct loop adds roughly 2.5 seconds of wait before a model generates its first token. Published research on ReadAgent and MemoryBank found that vector search accounted for 85% and 81% of recall latency respectively in those architectures, which means the memory layer, not the LLM, determined perceived response time.

At 10K stored memories, most vector-first systems retrieve quickly. The contest changes at 10M memories: index traversal time grows, graph hop counts multiply, and reranking passes add latency that scales with corpus size. A system that posts 150ms at 10K entries may behave very differently at production volume. That scaling behavior is what this guide prioritizes, using only published or self-reported figures, labeled as such.

What Drives Latency in AI Agent Memory Retrieval

Latency in AI agent memory retrieval depends on several factors along the critical path. The type of index and its traversal cost influence performance: flat (brute-force) vector search scales linearly with corpus size, while approximate nearest neighbor (ANN) indexes such as HNSW reduce complexity to roughly logarithmic scale. However, graph traversal latency depends on hop depth and neighborhood size, which can increase retrieval time independently of corpus size. Each graph hop requires reading from the graph store, and in Cognee's retrieval pipeline, additional iterations add validation, generation, and data fetch calls, causing latency and token cost to scale approximately linearly with hop depth. Reranking passes introduce per-candidate inference time, and some systems omit reranking to meet low latency targets, while others prioritize precision. Network round trips add overhead, especially for cloud-hosted memory APIs, where each retrieval call incurs transport latency. Finally, the timing of LLM extraction-whether at ingestion or retrieval-affects latency; systems like Zep and Cognee perform structured graph construction during ingestion, enabling faster retrieval as a graph read, whereas others extract facts at query time, shifting latency to retrieval.

What to Look for in an AI Agent Memory Layer for Low-Latency Retrieval

Evaluating retrieval speed requires careful separation of instrumentation data from headline figures. Vendor-reported numbers often reflect hardware, dataset sizes, and concurrency levels chosen by the vendor. It is important to distinguish search latency from total latency, as some figures include LLM generation time while others isolate retrieval. For example, Mem0's ECAI 2025 paper reports a p50 search latency of 0.148s separately from a total response latency of 0.708s. Percentile metrics and hardware details matter: p50 figures may hide tail latency, so p95 or p99 percentiles provide better insight. Hardware configurations set latency floors; Redis's 200ms vector-search figure at 1 billion vectors was measured under 50 concurrent queries, and single-query latency at smaller scale is lower. Measuring latency at the actual corpus size intended for deployment is essential, as Cognee's BEAM results show different latency at 100K versus 10M token contexts. Network overhead must be accounted for, since in-process or on-device retrieval (as with Redis working memory or cognee-rs) eliminates round-trip latency, whereas cloud API calls add it.

How to Measure Retrieval Latency Yourself

The most reliable latency measurement comes from your own query distribution. Instrument five timestamps per request: request start, retrieval start, retrieval end, model start, and first token. Running at least 200 queries yields a stable p95 latency estimate. Retries and errors should be logged separately, as early returns from failed requests do not represent latency improvements. Dataset size and hardware should be recorded alongside each run to maintain reproducibility as the corpus grows. Supermemory's latency budget documentation recommends recording retrieval and model timings separately and notes that latency budgets must account for interaction type, such as live voice turns, support chats, or background research tasks, rather than a single benchmark number.

What a Realistic Latency Budget for AI Agent Memory Retrieval Looks Like

Latency budgets vary by interaction type. Conversational turn-by-turn agents require retrieval to complete within approximately 300ms to avoid perceptible delay before LLM generation. Voice agents have tighter budgets due to additional stages like end-of-turn detection and speech synthesis. Background research or batch processing can tolerate longer latencies, prioritizing accuracy over speed. Published MemoryArena research found that agents with external memory incur the highest end-to-end latency, with long-context agents being fastest, indicating that latency budgets must account for structural overhead of retrieval beyond index lookup time.

How Agent Builders Use Low-Latency Memory Layers

Retrieval latency constrains several architectural patterns. High-frequency per-turn retrieval demands completion within the agent loop's time budget; Cognee's auto-routing mechanism selects retrieval strategies based on query type, avoiding overhead from running all strategies on every call. Multi-agent shared memory introduces retrieval contention; Redis's pub/sub and in-memory data structures optimize for sub-millisecond working-memory reads across concurrent readers. Long-lived codebase agents benefit from Cognee's graph traversal, which enumerates every caller of a function across a codebase in milliseconds, a pattern that pure vector similarity search handles poorly. Edge and on-device agents use cognee-rs, the Rust port of Cognee's memory pipeline, which runs ingestion, graph construction, embedding, and retrieval without a server-side stack. Published benchmarks show Rust search returning in 0.23s on small document sets and 0.27s on larger corpora, compared with 2.66s and 4.03s for the Python SDK on the same workloads (self-reported). Temporal multi-hop queries benefit from Zep's Graphiti architecture, which pre-builds the temporal knowledge graph at ingestion, enabling P95 retrieval under 200ms without LLM calls.

Competitor Comparison: AI Agent Memory Layers for Retrieval Latency

The table below lists each vendor's published retrieval latency figure alongside the percentile, dataset size, hardware, and data source where available. Self-reported figures are labeled. Where a vendor has not published a retrieval latency figure, this is stated explicitly rather than estimated.

Note on comparability: These figures are not directly comparable. They use different datasets, hardware, percentiles, and definitions of "retrieval" (some include LLM calls; some exclude them). Redis's sub-millisecond figure covers key-value working memory, not vector search over long-term memories. Cognee's edge figures reflect on-device Rust execution, not cloud API calls. Treat all vendor-reported numbers as directional, not as head-to-head race results on shared infrastructure.

Cognee's position in this comparison rests not solely on raw millisecond counts but on the architecture that produces predictable latency at scale: graph-structured retrieval returns a bounded subgraph, and published BEAM benchmarks indicate token usage stays flat as data grows from 100K to 10M tokens, a property that flat vector stores do not share, as their retrieval cost scales with corpus size.

Best AI Agent Memory Layers for Low-Latency Retrieval in 2026

1. Cognee

Cognee is an open-source AI memory platform that unifies relational, vector, and graph storage into a single engine. Its core pipeline, remember, recall, improve, forget, runs on a single Postgres instance (self-hosted) or on Cognee Cloud, and a Rust SDK (cognee-rs) extends the full pipeline to edge and on-device environments without a server-side stack.

The architectural property that separates Cognee from flat vector stores is how retrieval scales. Rather than scanning a growing embedding space and returning fixed-size chunks, Cognee retrieves a small, precise subgraph anchored at the entities most relevant to the query. Published BEAM benchmark results report 0.79 at BEAM 100K and 0.67 at BEAM 10M (self-reported), with token usage stated to stay flat as data volume grows, a key advantage when evaluating latency at scale, since flat token growth means retrieval cost does not compound with corpus size.

Key Features:

• Rust SDK (cognee-rs): The full memory pipeline, ingestion, graph construction, embedding, and retrieval, runs on-device. Published benchmarks show Rust search at 0.23s on a small corpus and 0.27s on a larger one, compared with 2.66s and 4.03s for the Python SDK under the same conditions (self-reported). On a Samsung Galaxy S24 Ultra, retrieval stayed in the low-single-second range after memory construction.
• Flat token scaling: Per published guidance, token usage stays constant as the stored corpus grows, because retrieval returns a subgraph rather than a proportional slice of the embedding space.
• Auto-routing retrieval: Cognee's retrieval mechanism selects the appropriate strategy based on query type, vector seeds, graph traversal, hybrid completion, without requiring the caller to specify a mode.

Retrieval-Specific Offerings:

• Graph-aware retrieval for multi-hop questions: Internal benchmarking using HotPotQA multi-hop questions showed graph traversal with chain-of-thought reasoning substantially outperforming base RAG on multi-hop retrieval (per Cognee published content).
• Hybrid retrieval combining vector similarity and graph traversal: The default search pipeline applies both in sequence. Latency and token cost on deep graph traversal scale roughly linearly with max_iter depth.
• On-device retrieval via cognee-rs targeting sub-100ms recall on constrained hardware (per edge documentation; described as experimental).
• BEAM accuracy: 0.79 at 100K context and 0.67 at 10M context (self-reported), with the best previously reported results at 0.73 and 0.64.

Pricing:

$1.00 per 1M tokens processed, plus $5 per additional workspace. The free workspace includes 10M tokens, unlimited users, and unlimited API calls. Self-hosting via the open-source topoteretes/cognee repository carries no platform fee.

Pros:

• Token usage stays flat at scale (self-reported), which constrains retrieval cost growth as corpus size increases
• Rust SDK eliminates network round-trip latency for edge and on-device deployments
• Single Postgres instance for self-hosting removes graph database operational overhead
• Open-source (Apache 2.0-compatible) with managed cloud and edge options in one SDK
• BEAM 10M score of 0.67 is state-of-the-art among published results (self-reported)
• Auto-routing removes the need to manually configure retrieval strategy per query type

Cons:

• Deep graph traversal (high max_iter) adds latency linearly, hybrid retrieval should not be optimized away in pursuit of lower numbers
• Published latency figures are self-reported and measured on specific hardware; independent cross-vendor benchmarks on shared infrastructure do not yet exist
• cognee-rs is described as experimental in current documentation

Cognee's core architectural claim is that latency at 10M memories does not have to be worse than latency at 10K, because graph-structured retrieval returns a bounded result set regardless of corpus size. That property, combined with the Rust core for deployments where round-trip elimination is critical, is what earns Cognee the top ranking in a latency-at-scale evaluation rather than a point-in-time speed test.

2. Redis

Redis Agent Memory Server is a dual-tier architecture: working memory uses native in-memory data structures (hashes, sorted sets) for sub-millisecond reads; long-term memory uses Redis vector search (RediSearch VSS with HNSW, FLAT, and SVS-VAMANA indexing) for semantic retrieval across sessions. In a vendor-published benchmark with one billion vectors, Redis reported 90% precision at approximately 200ms median latency under 50 concurrent queries. Working memory reads complete in under 1ms in many workloads. Redis functions as a storage backend beneath frameworks like Mem0 and LangMem, not only as a standalone memory solution.

Key Features:

• Sub-millisecond working memory via in-memory data structures; no disk access on the hot path
• HNSW, FLAT, and SVS-VAMANA vector indexing via the Redis Query Engine
• Redis Iris packaging: Agent Memory, LangCache (semantic caching), and Redis Search on the same platform

Retrieval-Specific Offerings:

• Working memory: in-session context at sub-millisecond latency
• Long-term semantic retrieval: ~200ms median at 1B vectors, 50 concurrent queries (self-reported, vendor benchmark)
• LangCache: up to 15x faster responses on cache hits and up to 70% lower LLM costs in high-repetition workloads (self-reported)

Pricing:

Redis Cloud (managed) with free tier; Redis Stack (self-hosted). Agent Memory is currently in public preview on Redis Cloud. Exact managed pricing not listed on the public pricing page for Agent Memory specifically.

Pros:

• Sub-millisecond working memory is the fastest in-session retrieval of any vendor reviewed
• 20+ years of production infrastructure; battle-tested at global scale
• Composable as a backend beneath Mem0, LangMem, LangChain, and Kong AI Gateway
• One platform covers vector search, caching, and session state, no additional services to operate

Cons:

• Working memory sub-millisecond figures apply to key-value lookups, not semantic vector search over long-term memory
• Vector search latency at scale (200ms at 1B vectors) is self-reported under specific concurrency and hardware conditions not detailed in public documentation
• Agent Memory is in preview; production SLAs not published as of this writing
• Does not include graph-structured retrieval for multi-hop reasoning without additional integration

3. Mem0

Mem0 is an open-source memory layer that extracts atomic facts from conversation, stores them in a vector index, and retrieves them via ANN search. Its ECAI 2025 paper (arXiv:2504.19413) reported p50 search latency of 0.148s and p95 search latency of 0.200s on the LoCoMo dataset, the lowest published search latency figures among all systems evaluated in that paper, with Zep reported at p50 total of 1.292s and full-context baseline at p50 total of 9.870s. The 2026 token-efficient algorithm benchmarks report p50 latency of 0.88s-1.09s across LoCoMo, LongMemEval, and BEAM workloads with 6.7K-7.0K tokens per retrieval, against full-context baselines consuming 25,000+ tokens per query.

Key Features:

• Selective fact extraction: retrieves the most salient memories rather than fixed-size chunks, keeping token cost low
• ANN indexing for O(log N) retrieval over large memory stores
• Open benchmark harness: results record search query, retrieved memories, search latency, and ground-truth answer per run

Retrieval-Specific Offerings:

• p50 search latency: 0.148s on LoCoMo (arXiv:2504.19413); 0.88s-1.09s p50 on 2026 algorithm across four benchmarks
• Graph memory on Pro tier ($249/month): entity relationships and multi-hop queries
• Open-source evaluation tooling for reproducible latency measurement

Pricing:

Hobby (free): 10K memory adds, 1K retrievals/month. Starter: $19/month. Pro: $249/month (includes graph memory). Enterprise: custom.

Pros:

• Lowest published search latency in the ECAI 2025 LoCoMo comparison (0.148s p50)
• Strong benchmark coverage with open evaluation harness for reproducible testing
• ANN indexing keeps latency sub-linear as corpus grows
• Broad framework integration: Python, JavaScript, 21+ agent frameworks

Cons:

• 2026 algorithm benchmarks show p50 latency rising to 0.88s-1.09s range on production-representative workloads
• Graph memory, the feature that enables multi-hop retrieval, requires the $249/month Pro tier
• Published benchmark figures are self-reported; Zep published a rebuttal disputing Mem0's LoCoMo methodology, and both vendors have since published improved numbers on setups they chose
• Token usage at 6.7K-7.0K per retrieval grows with the complexity of what is retrieved, unlike Cognee's flat token claim

4. Zep

Zep builds a temporal knowledge graph (Graphiti) from chat messages, documents, and structured data. Graph construction happens at ingestion time, so retrieval is a graph read rather than real-time extraction. Published P95 retrieval latency is under 200ms without LLM calls, because the graph is pre-built. Zep's 2025 paper (arXiv:2501.13956) reported 94.8% on Deep Memory Retrieval and 71.2% on LongMemEval with GPT-4o. Mem0's paper reported Zep at p50 total latency of 1.292s on LoCoMo; Zep disputed that methodology and published a corrected score above Mem0's in its own evaluation setup.

Key Features:

• Temporal knowledge graph with bi-temporal bookkeeping: facts carry valid_at and invalid_at timestamps, so the graph does not serve superseded facts as current
• Graph pre-built at ingestion: retrieval is a read against a structured store, not real-time fact extraction
• Python, TypeScript, and Go SDKs; compatible with any agent framework

Retrieval-Specific Offerings:

• P95 retrieval under 200ms (without LLM calls, self-reported)
• LongMemEval: 71.2% with GPT-4o; DMR: 94.8%
• Graphiti open-source under Apache 2.0 for self-hosted graph deployments

Pricing:

Free: 10,000 credits/month. Flex: $125/month (50,000 credits). Flex Plus: $375/month (200,000 credits). Credits are consumed by Episode ingestion size; retrieval, storage, and users are unmetered.

Pros:

• Pre-built graph means retrieval does not pay extraction cost at query time
• Temporal fact tracking handles contradictions and fact supersession natively
• P95 under 200ms for graph retrieval without LLM calls (self-reported)
• Strong temporal reasoning on LongMemEval relative to flat vector approaches

Cons:

• Latency figure excludes LLM calls; full response latency will be higher depending on LLM
• Dataset size and hardware not specified for the P95 figure
• Benchmark dispute with Mem0 on LoCoMo is unresolved; vendor-reported scores depend on configuration
• Flex plan at $125/month is the highest entry-tier price of the vendors reviewed

5. Supermemory

Supermemory is a memory API that packages fact extraction, contradiction resolution, automatic forgetting, and retrieval behind a single endpoint. Its custom vector-graph engine claims sub-300ms retrieval latency at p50, citing 10x faster than Zep and 25x faster than Mem0 on internal benchmarks, figures that are self-reported and not independently verified. Supermemory reports 85.4% on LongMemEval, 92.3% on single-session user retrieval, and a first-place ranking on LoCoMo and ConvoMem (self-reported). A published profile latency figure of approximately 50ms appears in documentation, but the same documentation notes that this is not a universal latency promise and that server-side and end-to-end timings differ.

Key Features:

• Custom vector-graph engine optimized for sub-300ms recall
• Automatic fact expiry: temporary facts expire after their relevant date; contradictions are resolved automatically
• Full five-layer context stack in one API: connectors, extractors, retrieval, memory graph, and user profiles

Retrieval-Specific Offerings:

• Sub-300ms p50 recall (self-reported); 10x faster than Zep and 25x faster than Mem0 on internal benchmarks (self-reported, not independently verified)
• ~50ms server-side profile retrieval (self-reported; end-to-end figure varies)
• 99.4% context reduction vs. full-context (self-reported, Recall@15 metric)

Pricing:

Free: $0 + $5 of monthly usage. Pro: $19/month (~$20 of usage). Max: $100/month (~$130 of usage). Scale: $399/month (~$600 of usage). Usage is metered separately: storage at $5-10 per million tokens; SuperRAG retrieval at $1-2 per million tokens; search and graph traversal at $5 per million queries.

Pros:

• Claims the fastest retrieval latency of the vendors reviewed (sub-300ms p50, self-reported)
• Built-in contradiction resolution and automatic fact expiry reduce memory staleness
• Broad connector set: Google Drive, Notion, Gmail, GitHub, Slack, S3, and web crawler
• Lowest entry paid tier ($19/month) among vendors with self-reported sub-300ms latency

Cons:

• Speed claims are self-reported; competitor latency figures in comparisons are described as simulated, not measured
• Cloud-only deployment on most plans; self-hosted option requires Scale or Enterprise tier
• Stacked usage metering (storage, retrieval, search, background operations) complicates cost prediction at scale
• No independent third-party latency benchmark published as of this writing

6. Letta

Letta (formerly MemGPT) is a full agent runtime, not a pluggable memory layer. Its OS-inspired three-tier architecture, core (in-context), recall (disk cache), and archival (cold storage), lets agents manage their own memory by calling tool functions against each tier. Retrieval quality depends on the agent's model making correct tool-call decisions; poorly structured memory entries or missed tool calls degrade retrieval without a fallback. Letta has not published standardized retrieval latency figures on LongMemEval or LoCoMo as of this writing. Its LoCoMo score of 74.0% (GPT-4o-mini, filesystem-based storage, no dedicated retrieval tooling) is the only publicly cited benchmark figure found in the sources reviewed.

Key Features:

• OS-inspired three-tier memory: agents read from core memory in-context, and call tools to query recall or archival tiers
• Self-editing memory: the agent decides what to preserve, update, or discard
• Agent Development Environment (ADE) for inspecting memory state and git-style history of memory edits

Retrieval-Specific Offerings:

• Agentic tool calls against memory tiers, the agent controls retrieval strategy
• Archival memory: searchable database the agent queries on demand
• Context-Bench V1 (open), V2 (private) for evaluating memory usage and generation behavior

Pricing:

Free (self-hosted, Apache 2.0, all features). Cloud Free: 3 stateful agents. Pro: $20/month (unlimited stateful agents, $20 monthly API credits). Max: $200/month. Enterprise: custom.

Pros:

• Self-hosted with full feature parity at zero platform cost
• Auditable memory: every edit is visible, and the agent's own memory behavior is the evaluation target
• $20/month Pro tier is the lowest managed-cloud entry price of the vendors reviewed with a full agent runtime
• Flexible: agents can adopt any retrieval logic via tool calls

Cons:

• No published retrieval latency figure; latency depends on model inference for tool-call decisions
• Every memory operation costs additional inference tokens, since the agent reasons about what to store and retrieve
• Retrieval quality is model-dependent; a weaker model misses or poorly structures memory entries
• Adopting Letta means adopting its agent runtime, switching later requires rebuilding the agent loop

Evaluation Rubric for AI Agent Memory Layers in Latency-Critical Workloads

The criteria below reflect what an engineering evaluation for production deployment should prioritize, weighted by how much each factor affects latency-sensitive agent performance.

Cognee's architecture addresses the two highest-weight criteria directly: its graph-structured retrieval returns a bounded subgraph (flat token scaling), and the Rust SDK eliminates the network round-trip. No independent cross-vendor benchmark on shared hardware exists as of this writing, so the table above should be treated as a framework for your own evaluation, not a scoring of the vendors listed here.

Why Cognee Leads Low-Latency AI Agent Memory Retrieval at Scale

The premise of this comparison is that latency at 10K memories and latency at 10M memories are different problems. Most vendor latency figures are measured at small scale, on hardware the vendor chooses, and labeled with percentiles that differ across vendors. The only property that constrains latency growth at scale is architecture, specifically, whether retrieval cost scales with corpus size or stays bounded.

Cognee's published BEAM results demonstrate accuracy that holds from 100K to 10M token contexts (0.79 and 0.67 respectively, self-reported), with the stated property that token usage stays flat as data grows. Combined with a Rust core that eliminates the network round-trip for edge deployments, and an auto-routing retrieval mechanism that avoids running all strategies on every query, Cognee is the vendor in this set whose architecture is explicitly designed for the latency-at-scale problem rather than for low latency at small scale.

Redis wins on raw working-memory speed (sub-millisecond) but does not offer graph-structured multi-hop retrieval natively. Mem0 publishes the most rigorous latency data of the vendors reviewed, but its 2026 benchmark figures show latency rising to the 0.88-1.09s range on production workloads, and token usage grows with retrieval complexity. Zep's pre-built graph is a sound architectural decision for temporal queries, but its published latency excludes LLM calls and does not specify hardware or dataset size. Supermemory's speed claims are self-reported against simulated competitor figures. Letta has no published latency figure at all.

For production memory workloads where retrieval latency must stay predictable across corpus sizes, Cognee is the architecture to evaluate first.

FAQs About AI Agent Memory Layers for Retrieval Latency

Which agent memory layer is fastest at retrieval?

The answer depends on what "retrieval" measures. Redis is fastest for in-session working memory, with sub-millisecond key-value reads in many workloads. For semantic memory retrieval over long-term stored facts, Mem0 published the lowest search-phase p50 (0.148s) in a 2025 academic paper on the LoCoMo dataset. Supermemory claims sub-300ms p50 at scale, but those figures are self-reported. Cognee's Rust SDK returns results in 0.23-0.27s on document-scale corpora in on-device benchmarks (self-reported). No independent cross-vendor benchmark on shared hardware has been published as of October 2026.

What is a realistic latency budget for AI agent memory retrieval?

A realistic budget varies by interaction type. Conversational turn-by-turn agents generally require retrieval to complete in under 300ms before LLM generation starts. Voice agents need tighter budgets because speech synthesis adds additional stages. Background or batch agents can tolerate multi-second retrieval if accuracy is the priority. What Supermemory's latency budget documentation correctly notes is that the budget must be set per interaction type, not derived from a single vendor benchmark number. Instrument your own retrieval path, record request start, retrieval start, retrieval end, model start, and first token, before committing to a vendor's published figure.

What causes latency to increase as agent memory scales?

At small corpus sizes, most vector indexes return results quickly because the search space is small. As corpus size grows, flat (brute-force) search scales linearly; ANN indexes like HNSW reduce that to roughly logarithmic scale but introduce approximation error. Graph traversal adds hop-by-hop read costs that multiply with depth. Reranking adds per-candidate inference time. Cognee's architecture addresses this by returning a small, precise subgraph anchored at the most relevant entities, so the retrieval result size stays bounded regardless of how large the stored graph becomes, a property its BEAM benchmarks are designed to demonstrate.

How does Cognee's Rust SDK affect retrieval latency?

The cognee-rs Rust SDK ports the full Cognee memory pipeline to edge and on-device environments. Per published benchmarks (self-reported, cognee.ai/memory-on-device), Rust search returned in 0.23s on a small document set and 0.27s on a larger corpus, compared with 2.66s and 4.03s for the Python SDK on the same workloads. On a Samsung Galaxy S24 Ultra, retrieval after memory construction stayed in the low-single-second range. The latency advantage comes from two sources: elimination of the cloud round-trip, and native execution without a Python runtime. cognee-rs is currently described as experimental in Cognee's documentation.

Does graph-based retrieval always add latency compared to vector search?

Not necessarily, and the comparison depends on what the retrieval must return. A pure ANN vector search is fast at low depth but returns approximate semantic neighbors, not entity relationships or causal connections. Graph traversal adds hop-by-hop read cost, but Zep's architecture shows that pre-building the graph at ingestion time can keep P95 retrieval under 200ms without LLM calls. Cognee's hybrid approach applies vector seeds first, then graph traversal within the resulting neighborhood, combining speed from the initial ANN pass with precision from graph structure. The latency cost of graph traversal scales with max_iter depth, not directly with corpus size, which is architecturally different from flat vector search.

How should I evaluate agent memory latency for my own application?

Start by instrumenting five timestamps: request start, retrieval start, retrieval end, model start, and first token output. Run at least 200 queries drawn from your actual query distribution, not from benchmark datasets, to get a stable p95. Log dataset size and hardware for each run. Separate ingestion latency from retrieval latency; they have different causes and different fixes. Test at your target corpus size, not at a small prototype scale. Cognee's evaluation framework in the cognee repository and Mem0's open memory-benchmarks suite both provide starting infrastructure for this kind of instrumented evaluation.