What Samsung code reveals about inference cost of weight compression below one bit

The prevailing belief among language-model engineers has been that the one-bit edge represents the minimum that cannot be crossed without the mathematical logic of the weights collapsing and processing accuracy being lost. However, the source code released by Samsung Labs through the Little Bit project repository pushes model architectures into a unfamiliar computational region, achieving compression rates that fall between a full bit and only a tenth of a bit per weight, without imposing any change on the model’s original software architecture during inference.

The core idea documented by the researchers at the NIPS 2025 conference relies on decomposing every dense weight matrix into low-rank latent factors by factorising individual values, then converting those factors into a strict binary format, while restoring signal capacity through lightweight numeric scales learned during training. When we examined the updated architecture, we found that the second version accepted at the ICMl 2026 conference by researchers Ban Seok-Lee and Yong-Min Kim addresses geometric alignment drift during the preparation stage, where the authors added an internal latent rotation mechanism supported by shared iterative quantisation, to adjust and align the extracted latent factors with the super-binary cube before quantisation training begins.

The engineering gain is concentrated in the absence of any additional cost at runtime.The Little Bit Linear layer is activated only during preparation, and the data flow during inference remains identical to the usual layers. However, achieving this extreme compression does not occur as a quick post-training conversion; it requires a full quantisation-aware training that relies on a smooth-sign function and residual decomposition support, and it needs computation to be distributed via the DeepSpeed library in its Zero-3 mode across multiple GPUs. The experiments announced in the repository test a broad matrix of models, including the Llama, Phoenix 4, Qwen 2.5 families, QWQ models, Gima up to Qwen 3, with accuracy evaluated on perplexity tasks in WikiText-2 and direct inference benchmarks such as HellaSwag, ARC, and BoolQ.

The field viability for regional tech teams is governed by two contradictory limits.From an infrastructure perspective, reducing memory to roughly a tenth of a bit represents an economically significant shift for data-centers and companies in the Gulf, Egypt and the Levant, as it lowers bandwidth-memory requirements and enables hosting large models on cheaper local servers, thereby strengthening data-processing sovereignty without relying exclusively on external cloud. However, the first obstacle stems from the project's licence, which is a Creative Commons Non-Commercial 4.0 licence, preventing financial institutions and service providers from using this code in profit-driven or commercial applications without direct licences. The second obstacle lies in the cost of engineering efficiency; the system does not provide one-click ready quantisation, but instead requires local teams to possess advanced expertise in managing distributed training and aligning mathematical weights.

This experiment shows that the memory bottleneck in AI servers can be computationally broken below the bit level, but the savings bill does not disappear from the budget; it shifts from inference-hardware cost to training compute and compliance with licence constraints.