This paper introduces NF-CoT, a framework that utilizes Normalizing Flows (NF) to model intermediate reasoning steps as compact continuous states within Large Language Models (LLMs). This approach bypasses the limitations of discrete Chain-of-Thought (CoT) by allowing intermediate reasoning.
Large language models currently use explicit Chain-of-Thought (CoT) to improve reasoning by verbalizing intermediate steps. However, CoT forces computation through a discrete, serial token stream, which is inefficient. We propose Latent Reasoning with Normalizing Flows (NF-CoT) to offer a higher-bandwidth alternative. NF-CoT instantiates a TARFlow-style normalizing flow inside the LLM backbone to model continuous thoughts distilled from explicit CoT. Continuous-thought positions are generated by an NF head, while standard text positions are generated by the LM head, all operating within the same causal stream.
This design yields several advantages: it provides exact likelihoods for latent thoughts, enables probabilistic left-to-right decoding compatible with KV-cache, and supports direct policy-gradient optimization in the latent reasoning space. Experiments on code-generation benchmarks show that NF-CoT improves pass rates over explicit-CoT and prior latent-reasoning baselines while substantially reducing the total intermediate-reasoning cost.