BWSPI — Zero-Hash, 1-Instruction Data Index in Rust

The description states:

• BWSPI (Bit-Width Sparse Pointer Index), also called the Sarkar Bucket Array, is a zero-hash sparse indexing system for unsorted dynamic data streams.
• It replaces hash functions with a single CPU instruction (LZCNT) to route data to buckets based on bit-width.
• The architecture uses two tables:
  • Table 1: Data Store — an append-only Vec that never gets sorted or moved.
  • Table 2: Bit-Width Index — routes lookups to the right bucket using bit-width, then performs a small local scan.
• Within each bucket, a lazy LSB radix tree progressively splits when leaves exceed 64 entries, giving sub-linear lookup without hashing.

Inference This is a low-level systems tool designed for high-performance data indexing in environments where latency and memory usage are critical. It is not a commercial product but a technical experiment or proof-of-concept.

The author states:

• The goal was to find an alternative to hashing that avoids O(n) rehash spikes and reduces CPU instruction overhead.
• They position BWSPI as being “~x(mostly win but with few regression) faster than HashMap,” using fewer resources, and avoiding unpredictable latency.
• It is described as a system that routes data by its binary structure rather than computing hashes — a fundamentally different approach from existing hash-based systems.

Inference The positioning is rooted in performance optimization for unsorted data streams. The claim evolution shows a shift from an experimental idea to a working prototype, though it remains self-reported and lacks independent validation or real-world use cases.

Not evidenced.

Not evidenced.

The description states:

• Built in pure Rust with zero dependencies for the core library.
• Uses AI-assisted architecture and implementation (GPT 5.5 and Claude Opus).
• SIMD acceleration via hand-written AVX2 intrinsics.
• Custom benchmarking suite measuring median-of-7 warm runs, cache topology, and CPU utilization.
• Comparison against std::HashMap, FxHashMap, AHashMap, SwissTable, and linear scan.

Inference The author has a strong systems programming background and is leveraging modern tools (AI, SIMD) to build a performance-focused tool. However, the delivery signals are limited to internal validation and lack external testing or adoption.

Not evidenced.

The description states:

• It is positioned as an alternative to HashMap/HashSet.
• It competes with systems like SwissTable, FxHashMap, AHashMap, and linear scan.
• The author notes that it is not better at everything but is fundamentally different from hashing-based approaches.

Inference It exists in the space of high-performance data structures for systems programming. However, no competitive analysis or market positioning beyond self-description is provided.

The description states:

• The uniform-width worst case (e.g., all 32-bit values) leads to one bucket with all elements — a fundamental trade-off.
• The architecture vs implementation gap was significant, and AI was used to bridge it.
• The design changed drastically multiple times.

1. What specific use cases are you targeting with this index?
2. Have you validated performance in real-world environments or under high-load conditions?
3. How does BWSPI compare to existing hash-based systems in terms of memory overhead and cache behavior?
4. Are there any known edge cases or failure modes that have not been addressed?
5. What is the timeline for a production-ready version, and what are the next steps in development?
6. Have you considered integration with other data structures or frameworks (e.g., databases, streaming systems)?