Mathematical Foundations of AI

Overview

Modern AI systems work remarkably well, yet much of why they work from algorithmic and scientific perspective remains poorly understood. This theme builds mathematical foundations for learning systems, treating optimization landscapes, training dynamics, and generalization as objects we can analyze rather than merely observe.

Our results span several fronts. From optimization side, we have studied how training can bypass stationary points, and also dabbled on fractional-order gradient methods. On deep learning model structure, we develop principled structural pruning — using bifurcation dynamics and projective geometry to decide what edges a network can lose without giving up its functional output. From the representation learning viewpoint, we analyzed feature learning as a form of covariance learning and study the emergent linear separability of features in a network’s last layer. The throughline is turning empirical phenomena into theory that predicts behavior of deep learning models, guides better model design, and deeper understanding of why modern AI works.

Core Questions

• What governs optimization dynamics, and when can training escape or bypass bad critical points?
• How much of a network is truly necessary, and how do we prune with provable structure rather than heuristics?
• How do useful features and clean geometric structure emerge in representations during training?

Representative Work

• Bypassing Stationary Points in Training Deep Learning Models — IEEE TNNLS, 2024
• Curse of Smoothness in Functional Neural Networks — IEEE Signal Processing Letters, 2025
• Catalyst: Structured Pruning with Robust Bifurcation Dynamics — ICML 2025 HiLD Workshop
• Feature Learning as a Virtual Covariance Learning — NeurIPS 2025 OPT Workshop
• Emergent Linear Separability of Unseen Data Points in High-dimensional Last-Layer Feature Space — ICML 2025 HiLD Workshop

See all work on the Publications page.

Related

• Logs tagged Math4AI
• Events and reading groups on learning theory

People

• Donghun Lee — Principal Investigator
• Taehun Cha — feature learning, last-layer geometry
• Jaeheun Jung — structured pruning, training dynamics
• Bosung Jung — optimization, unlearning

See People for the full lab.