Weiyang Liu
University of Cambridge
Max Planck Institute for Intelligent Systems
I am currently conducting research at Cambridge and MPI Tübingen with Adrian Weller and Bernhard Schölkopf. As a member of the advising team at MeshCapade, I also work closely with Michael J. Black. Previously, I spent wonderful years at Georgia Tech. I have also spent time at Google Brain, Nvidia Research, and MERL.
I work on principled modeling of inductive bias in machine learning. My research seeks to understand how inductive bias determines generalization, and to develop "light-yet-sweet" generalizable models: (i) light: conceptually simple in methodology and easy to implement in practice, (ii) sweet: having clear intuitions and non-trivial theoretical guarantees.
Over the years, I always find myself fascinated by geometric invariance, symmetry, structures (graph, causality) and how they can benefit generalization as a guiding principle. Recently, I become very passionate about large language models (general intelligence) and generative modeling of the physicual world (physical intelligence). More specifically, I try to understand how LLMs perform reasoning and how to improve it in various scenarios.
I always believe in two principles in my research: (i) insight must precede application, and (ii) everything should be made as simple as possible, but not simpler. I try to follow certain research values.
I take great pleasure to (co-)mentor a few talented and highly motivated students. Mentoring and working with junior students is truely a privilege, and I constantly learn from and get inspired by them. I am fortunate to work with (time-wise order):
- Yamei Chen (2024 - now)
- M.S. student at Technical University of Munich
- Zeju Qiu (2024 - now)
- Ph.D. student at MPI for Intelligent Systems
- Tim Z. Xiao (2024 - now)
- Ph.D. student at University of Tübingen
Alumni list (nothing is more rewarding than seeing my mentees succeed)
Can Large Language Models Understand Symbolic Graphics Programs?
Zeju Qiu*, Weiyang Liu*, Haiwen Feng*, Zhen Liu**, Tim Z. Xiao**, Katherine M. Collins**, Joshua B. Tenenbaum, Adrian Weller, Michael J. Black, Bernhard Schölkopf
Preprint 2024
Verbalized Machine Learning: Revisiting Machine Learning with Language Models
Tim Z. Xiao, Robert Bamler, Bernhard Schölkopf, Weiyang Liu
Preprint 2024
Show selected / all by date / all by topic: Value-based Inductive Bias, Data-centric Learning, Generative Modeling
Verbalized Machine Learning: Revisiting Machine Learning with Language Models
Tim Z. Xiao, Robert Bamler, Bernhard Schölkopf, Weiyang Liu
Preprint 2024
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
Weiyang Liu*, Zeju Qiu*, Yao Feng**, Yuliang Xiu**, Yuxuan Xue**, Longhui Yu**, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, Yandong Wen, Michael J. Black, Adrian Weller, Bernhard Schölkopf
ICLR 2024
arXiv | code | project | huggingface | openreview | talk | slides | bib
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
Longhui Yu, Weisen Jiang, Han Shi, J. Yu, Z. Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, Weiyang Liu*
ICLR 2024 Spotlight
arXiv | code | project | huggingface | openreview | bib
Controlling Text-to-Image Diffusion by Orthogonal Finetuning
Zeju Qiu*, Weiyang Liu*, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, Bernhard Schölkopf
NeurIPS 2023
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap
Weiyang Liu*, Longhui Yu*, Adrian Weller, Bernhard Schölkopf
ICLR 2023
MeshDiffusion: Score-based Generative 3D Mesh Modeling
Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, Weiyang Liu*
ICLR 2023 Notable Top-25%
Structural Causal 3D Reconstruction
Weiyang Liu*, Zhen Liu*, Liam Paull, Adrian Weller, Bernhard Schölkopf
ECCV 2022
SphereFace Revived: Unifying Hyperspherical Face Recognition
Weiyang Liu*, Yandong Wen*, Bhiksha Raj, Rita Singh, Adrian Weller
TPAMI 2022
arXiv | code | opensphere | IEEE Xplore | bib
SphereFace2: Binary Classification is All You Need for Deep Face Recognition
Yandong Wen*, Weiyang Liu*, Adrian Weller, Bhiksha Raj, Rita Singh
ICLR 2022 Spotlight
Iterative Teaching by Label Synthesis
Weiyang Liu*, Zhen Liu*, Hanchen Wang*, Liam Paull, Bernhard Schölkopf, Adrian Weller
NeurIPS 2021 Spotlight
Learning with Hyperspherical Uniformity
Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf, Adrian Weller
AISTATS 2021
Orthogonal Over-Parameterized Training
Weiyang Liu*, Rongmei Lin*, Zhen Liu, James Rehg, Liam Paull, Li Xiong, Le Song, Adrian Weller
CVPR 2021 Oral
Learning towards Minimum Hyperspherical Energy
Weiyang Liu*, Rongmei Lin*, Zhen Liu*, Lixin Liu*, Zhiding Yu, Bo Dai, Le Song
NeurIPS 2018
Deep Hyperspherical Learning
Weiyang Liu, Yan-Ming Zhang, Xingguo Li, Zhiding Yu, Bo Dai, Tuo Zhao, Le Song
NIPS 2017 Spotlight
Iterative Machine Teaching
Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Yu Chen, Linda Smith, James Rehg, Le Song
ICML 2017
SphereFace: Deep Hypersphere Embedding for Face Recognition
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, Le Song
CVPR 2017 Paper Digest Most Influential Paper