Anton Sugolov


about

Currently a research engineer at Extropic working on probabilistic ML and numerical methods for Markov chain simulation.
Previously, I was a Math Master's student at University of Toronto, supervised by Vardan Papyan and Adrian Nachman.
I'm broadly interested in representation learning and mathematical methods for understanding empirical deep learning. I like to think about all things applied math, probability, and machine learning.


projects

Transformer Block Coupling
Transformer Block Coupling and its Correlation with Generalization in LLMs
Murdock Aubry*, Haoming Meng*, Anton Sugolov*, Vardan Papyan
ICLR 2025

We empirically find that the Jacobians of transformer blocks in pre-trained LLMs have highly similar singular vectors.

jepax
jepax: A JAX library for JEPA research
Owen Lockwood*, Anton Sugolov*

A JAX/Equinox implementation of Joint-Embedding Predictive Architecture (JEPA) models and related self-supervised learning methods.

notes