Most search benchmarks report accuracy. This one asks a different question for every answer: can a third party prove what happened? Which context the engine returned, that what was excluded was provably excluded, and that nobody altered the record afterwards.
Each layer is independently installable from PyPI and independently verifiable. None of them requires trusting the others — that is the point of a stack built around proofs rather than claims.
| Package | PyPI | The promise | How this benchmark holds it to it |
|---|---|---|---|
winnex-madhava | 1.9.22 | The search engine. Prunes with a Cauchy–Schwarz bound; every excluded document carries a certifiable upper bound. | Recall measured against the engine's own brute-force scan; search_exact checked against brute force in 120 queries; coverage reported per query, not just as a median. |
winnex-ai-normalize | 1.4.5 | The quality gate. Validates the corpus, routes a configuration, and can refuse to proceed. | The gate is fed NaN, inf, wrong dimension and degenerate recall — and has to block each. The fail_on_recall knob is tested for being real rather than decorative. |
winnex-nano | 0.1.47 | Native inference. A forward pass that is numerically the same computation as transformers, in a C++ engine. |
Logits compared against transformers on the same prompts; tokenizer output compared token-by-token against the model's BPE. |
winnex-tracer | 1.1.2 | The proof layer. Signs an Ed25519 commitment per query and chains it in a WORM log. | Every signed field tampered with individually; the chain exercised under a long-lived writer, under 4 concurrent processes, and with a tampered history. |
winnex-tracer-med | — | Clinical triage end to end over a real corpus, using the pieces above together. | Full pipeline over 250 records: bound soundness, signed commitment per query, WORM receipt per query, third-party reconstruction by audit_id. |
20,000 MedMCQA explanations, 300 queries, k = 3, encoder all-MiniLM-L6-v2. Recall is measured against the engine's own exact scan, which is first proven correct — otherwise the reference itself would be wrong and every number above it meaningless.
| Configuration | Recall@3 | Subject@3 | Anchor@3 | Seal coverage | Scope |
|---|---|---|---|---|---|
default (s1=64, s2=128) | 0.956 | 66% | 49.1% | 0.00% | pool_only |
pca_corpus | 0.986 | 66% | 48.7% | 77.22% | pool_only |
| full rank (s1 = s2 = d) | 1.000 | 66% | 48.7% | 99.98% | pool_only |
audit_exhaustive | 1.000 | 66% | 48.7% | 0.00% | exact_global |
Two different paths to the same answer. full rank and audit_exhaustive both reach recall 1.000, and their seal coverage is 99.98% versus 0.00%. That is not a contradiction — it is the core distinction of the certificate.
audit_exhaustive makes the candidate pool cover the whole corpus, so the top-k is exact by construction: nothing needs to be excluded because nothing is pruned. It pays O(N·d) per query. full rank reaches the same recall by pruning, excluding 99.98% of documents with proof. It pays for the proof instead.
High coverage is not high recall. Reading a seal requires both fields, because they say which of the two paths was taken.
search_exact agreed with a brute-force scan in 120 of 120 queries. This check matters because every recall figure above is measured against that scan — if it diverged from brute force, the reference would be wrong.
The subject metric saturates here, and the report says so. Topic@3 measured 80% against a measured chance of 72.7% — a margin of 7 points. At this corpus size a randomly drawn document hits the right topic four times in five, so a high topic score is not strong evidence of good retrieval. The metric that carries signal is anchor@3: 49% against 8.2% chance, about 6× the baseline. Where a metric saturates, this benchmark reports that it saturates instead of quoting the flattering number.
The claim is narrow and precise, and it is worth stating exactly what it covers. A bound violation would mean a document was pruned that could have been in the true top-k. Zero violations means that did not happen — given the vector space, the encoder and the corpus.
0 violations across every configuration and every query. The Cauchy–Schwarz bound never excluded a document that belonged in the answer.
At full rank, 99.98% of the corpus is excluded with a certificate per document — not filtered, not approximated.
12 of 12 signed-field tamperings were detected: exclusion count, bound violations, threshold, audit_id, query fingerprint, timestamp, indices, sample upper bound, and a foreign public key.
_latency_ms is added after signing and must remain VALID. If it invalidated the record, the seal would be useless for its purpose.
Intact across a long-lived writer beside ephemeral ones (6 blocks), and under 4 concurrent processes × 5 appends (20 blocks), with zero violations.
A tampered history is detected. A correction that made the chain accept altered history would be worse than the defect it fixed.
The certificate is not free. Measured over 30 repetitions: the signed commitment plus Ed25519 signature plus WORM receipt costs 2.36× the search itself, and the signed record is 2,190 bytes.
Anyone comparing this to a plain vector index should compare against the search alone, not against the full pipeline. The overhead is the product — it is what makes the answer auditable — but it is a cost, and it is declared.
winnex-nano reproduces the transformers forward pass: cosine similarity of the logits is ≥ 0.99988 on every prompt tested, and the top-3 next tokens are identical. That is what licenses treating native inference as the same computation in a different engine. It is a statement about this checkpoint, not about every architecture.
A benchmark that lists only what it proves is advertising. These are the limits, and they are the same ones printed in the report the notebook generates.
tracer-med is exercised as a mechanism demonstration, not for medical content.total_provably_excluded says the bound discarded many documents — it does not say the top-k is correct. They are different measurements and are reported separately.tracer-med pseudonymises identifiers, which is necessary and not sufficient for compliance.transformers for a 4B model; it is not proof across the architecture family.Every number on this page comes from a JSON file written by the run. Nothing was typed in by hand: the report is generated from the results, and the notebook embeds both the benchmark and the report generator, so what you read is what ran.
| What | Where |
|---|---|
| The full stack benchmark — packages, contracts, certificate, parity | stack-winnex — auditable inference |
| Knowledge effect — 160k corpus, 4 arms with placebo, McNemar | conhecimento-tracer-med v2 |
| Five real datasets — GloVe, ProtBERT, SBERT, FastText, Word2Vec | five datasets |
| Against FAISS — NDCG comparison | NDCG vs FAISS |
| Scale — BigANN-100M | BigANN-100M |
| The per-document certificate | certificate |
pip install winnex-madhava winnex-ai-normalize winnex-tracer winnex-nano |