Hanson.HSChang

A state space model maps an input sequence to its outputs through a lower triangular semiseparable matrix, with low-rank blocks highlighted in blue.

Research
Review

Transformers and State Space Models: A Connection in Sequential Data Modeling

February 28, 2026

A dual perspective on how decoder-only Transformers and State Space Models can be viewed as two sides of the same coin in modeling sequential data, with an implication for efficient algorithm design.

Positional Encoding in Transformer Language Models

February 13, 2026

An exploration of the role of positional encodings in transformer language models, including their construction and impact on model performance.

How regularization affects the critical points in linear neural networks

November 08, 2018

The existence and optimality properties of the critical points of the linear neural networks with mean-squared loss function in the face of regularization.