Search that stays in your environment.A detailed benchmark.
We built this benchmark to see how Enclave performs with real data, real storage, and the same constraints our customers care about. This report shows what we tested, the numbers we got, what we fixed along the way, and one claim the results didn’t support.
Five tests. Public data,
synthetic scale, and real S3.
| Test | Data | Scale | What it measured |
|---|---|---|---|
| Scale curve | Synthetic 768-d vectors | 100K → 1M chunks | Latency, bytes-per-query, memory as corpus grows |
| Real-data quality | FiQA-2018 (BEIR) | 57,638 chunks | Recall@20, NDCG, vs. an in-memory baseline |
| Permission-aware retrieval | FiQA + SciFact + NFCorpus | 4 configurations | Recall under heavy permission restriction |
| Real-S3 latency | FiQA-2018 on AWS S3 | 57,638 chunks | End-to-end latency on actual object storage |
| Edge-cache validation | FiQA-2018 on AWS S3 | 57,638 chunks | Cache impact on warm and cold latency |
Methodology · held constant
HNSW search throughout. Embeddings generated with a fully local model (nomic-embed-text-v2-moe via Ollama), no external API anywhere in the pipeline, so the test uses no external model API. Public datasets from the BEIR benchmark suite. The harness lives in our repository under benchmarks/, and every number here is generated directly from its JSON output. Test runs from a developer laptop unless stated otherwise; cross-region (laptop → us-east-1) where storage is real S3.
The numbers that
came out of the tests.
Storage I/O
~13 KB / query
13,089 bytes at 100K chunks and 13,609 bytes at 1M chunks. Ten times more data increased storage I/O by only 1.04×.
Edge cache
52,835 ms → 0.96 ms
Warm p50 dropped after adding the missing cache layer. Bytes read per query remained roughly 13 KB.
Warm latency
p99 1.87 ms
Warm p50 / p90 / p99 measured 0.96 / 1.30 / 1.87 ms. Cold p99 was 26.7 s because the test fetched an uncached 64 MB partition cross-region from a laptop.
Retrieval quality
0.497 Recall@20
The in-memory baseline measured 0.518. Enclave measured NDCG@10 of 0.330 and 59 ms hybrid latency versus 169 ms for the baseline in this test.
Claim retired
No measurable recall advantage from during-search permission filtering.
Across four controlled configurations, the recall difference was +0.001 or smaller. The permission filtering remains correct, but the benchmark did not support the recall-improvement claim, so we retired it.