Built by the Winnex technical team, winnex-xfactor is the operator that projects text — coming from the winnex-nano SpectralTokenizer — onto the real manifold of a pretrained model's embeddings. Computed structurally from embed_tokens, training-free, deterministic, never by running the model.
winnex-xfactor (v1.0.2) is a piece of the Winnex stack — same company, same BSL 1.1 license, same deterministic-no-black-box design principle. It is the operator that projects spectral text onto the principal subspace of a pretrained model's token embeddings.
The mathematics is classical and correct:
E (a sample of embed_tokens).C = ĒᵀĒ / V.r that captures a fraction τ of the variance.X = U[:, :r] and the orthogonal projector P = XXᵀ.Mathematical properties the notebook verifies:
• P² = P (idempotence) — maximum difference 0.000000
• P(Pv) = Pv (projection stability)
• Implicit symmetry
This is exactly the orthogonal projection onto the principal subspace of the model's embeddings. The contribution is the fast, deterministic implementation and its integration with the rest of the stack (spectral → expand → project).
The public notebook winnex-xfactor-benchmark-manifold uses real Qwen embeddings — a serious, honest evaluation.
It loads real embeddings via winnex-nano.Safetensors + sample_embeddings (without materializing the full tensor — avoids OOM), samples 2000 rows of transformer.wte.weight (D=2048), and subsamples columns to D_sub = 32/64/128.
| D_sub | Rank (τ=0.95) | Variance captured | Time |
|---|---|---|---|
| 32 | 31 | 0.9736 | 0.001 s |
| 64 | 60 | 0.9535 | 0.006 s |
| 128 | 118 | 0.9525 | 0.042 s |
| τ | Rank | Variance |
|---|---|---|
| 0.90 | 56 | 0.907 |
| 0.95 | 60 | 0.954 |
| 0.99 | 65 | 0.986 |
Full expand_spectral + projection pipeline works — output norm ~0.981.
The takeaway: the C++ implementation with power iteration + deflation delivers what it promises in speed for moderate ranks. D=128 → 0.042 s, D=64 → 0.006 s — fast enough for practical dimensions on a common CPU.
What is solid, and the precise scope of what the operator does and does not do.
winnex-nano (spectral → model manifold).What the operator does is project a vector (expanded from the spectral representation) onto the principal subspace of the token embeddings. This is not the internal residual stream, not the intermediate layer activations, and not the semantic geometry the model actually uses to generate text. It is a linear approximation of the input-embedding manifold.
Power iteration + deflation is a classical numerical-linear-algebra technique (Hotelling deflation, etc.). The O(D²r) vs O(D³) gain is real when r ≪ D. At D=2048 and r ~100–200 the "~2000×" figure depends strongly on implementation and iteration count; at D=128 the notebook shows 0.042 s — perfectly acceptable, not miraculous.
A 2000-embedding sample is a reasonable approximation of a ~150k-token vocabulary, not an exact decomposition of the full manifold. The reported rank (nearly D_sub) indicates the column-subsampled matrix is nearly full-rank — expected under random dimension sampling. The notebook proves the operator works (mathematics + speed + projection); downstream quality (better generation, less hallucination, better retrieval) remains a stack hypothesis to be demonstrated with standard metrics.
The X-Factor closes the loop.
text → SpectralTokenizer (nano) → expand_spectral → X-Factor projector
→ vector on the model's manifold → (Madhava / StreamEngine)
It is coherent with the project philosophy — deterministic operators, training-free, mathematical proof, no black boxes. Alongside Madhava (search with bounds) and Nano (spectral tokenizer), it forms a native, auditable inference pipeline.
Verdict: winnex-xfactor is a small, well-implemented, mathematically correct library that computes the orthogonal projector onto the principal subspace of a model's token embeddings. The notebook confirms it works on real Qwen embeddings, that the projection is stable, and that power iteration + deflation is fast enough for practical dimensions.
Not a revolution in LLM representation. It is applied linear geometry, consistent with the rest of the Winnex stack. Its real utility depends on the rest of the pipeline (spectral + manifold + Madhava) delivering quality generation or retrieval — not yet demonstrated publicly with standard metrics (perplexity, MMLU, human eval).
As a piece of deterministic, training-free engineering it is solid. As "the operator that puts text into the model's latent space," the claim is read precisely: it projects onto the principal span of the input embeddings, not the deep internal geometry of the transformer.