Hanson.HSChang

Attention and optimal-control heatmaps emphasize similar time indices in a binary prediction task.

Research
Paper

From Optimal Control to Transformer-like Inference

May 15, 2026

How filtering-control duality produces transformer-like layers, and what a simple prediction experiment reveals about attention and memory.

A state space model maps an input sequence to its outputs through a lower triangular semiseparable matrix, with low-rank blocks highlighted in blue.

Research
Review

Transformers and State Space Models: A Connection in Sequential Data Modeling

February 28, 2026

A dual perspective on how decoder-only Transformers and State Space Models can be viewed as two sides of the same coin in modeling sequential data, with an implication for efficient algorithm design.

Belief Net architecture and pseudocode show observations passing through emission, correction, transition, and estimation steps to update beliefs and predict the next observation.

Research
Paper

Belief Net: Bridging Probabilistic Modeling and Deep Learning for Sequential Data

February 21, 2026

A HMM-based learning framework that understand sequential data while ensuring full interpretability and fast convergence.

A token embedding is split into pairs of dimensions, with each pair rotated by a position-dependent angle at a different frequency.

Research
Review

Positional Encoding in Transformer Language Models

February 13, 2026

An exploration of the role of positional encodings in transformer language models, including their construction and impact on model performance.