Amartya Sanyal

I am an Assistant Professor in Machine Learning in the Department of Computer Science in the University of Copenhagen and an Adjunct Assistant Professor at the Department of Computer Science, IIT Kanpur. I am also affiliated with the Department of Mathematics at University of Copenhagen.

Email / GitHub / Google Scholar / CV

Prior to this, I was a postdoctoral fellow at the Empirical Inference group in Max Planck Institute for Intelligent Systems, Tubingen where I worked closely with Prof. Bernhard Schölkopf . Before that, I was a postdoctoral fellow at ETH Zurich AI Center where I worked closely with Prof. Fanny Yang . I completed my DPhil (PhD) at the Department of Computer Science, University of Oxford, funded by the Turing Doctoral Studentship. I was also a member of the Torr Vision Group. My DPhil advisors were Varun Kanade and Philip H.S. Torr. I am also a member of the ELLIS Society.

Prior to that, I completed my undergraduate (B.Tech in Computer Science) at the Indian Institute of Technology, Kanpur. On various occassions, I have spent time at Facebook AI Research (FAIR), Twitter Cortex , Laboratory for Computational and Statistical Learning, Montreal Institute of Learning Algorithms , and Amazon ML.

Openings with me

If you are interested in a PhD or Postdoc position with me, see here for more details

If you are a student at UCPH and interested in doing a undergraduate or masters project with me, see here for more details

Research Interests

I am interested in understanding both theoretical and empirical aspects of Trustworthy Machine Learning including privacy, robustness, fairness. In particular, I am focusing on the impacts that inadequate data and computation may have on the trustworthiness of ML algorithms, especially under adversarial settings, and how one can solve these issues with reasonable approximations and relaxations including semi-supervised and self-supervised learning.

Current Students

I am very lucky to be currently working with the following students.

Carolin Heinzler

Prof. Amir Yehudayoff

Johanna Düngler

Prof. Rasmus Pagh

Prof. Christoph H. Lampert

Prof. Bernhard Schölkopf

Prof. Fanny Yang

Upcoming/Recent Talks

Feb 19	ELLIS talk: Privacy with Correlated data @ Institute of Science and Technology, Austria (ISTA)
Feb 25	Tutorial - Building trustworthy ML: The role of label quality and availability @ Association for Advancement of Artificial Intelligence (AAAI)
Feb 28	Lower Bounds for Differentially Private Online Learning @ Columbia University
Mar 11	Keynote: Machine Unlearning: Promises, Failures, and New Directions @ Huawei Future Device Technology Summit - Aurora Summit

Recent News

Jan, 2025	I received the Villum Young Invesigator Grant and will be working on topics related to Privacy, unlearning, and online learning. I will be hiring motivated PhD students and Postdocs. Email me if you think you would be good match.
Jan, 2025	Three papers on differentially private alignment of LLMs, Machine unlearning, and training data attribution were accepted at ICLR 2025 and one paper on OOD robustness and label noise at AISTATS 2025 .
Oct, 2024	Alex Tifrea , and I are giving a tutorial on Building trustworthy ML: The role of label quality and availability” at AAAI, 2025.
Oct, 2024	Our paper titled Corrective Machine Unlearning is accepted at TMLR .
Sep, 2024	We will present two papers titled Robust Mixture Learning when Outliers Overwhelm Small Groups and TWhat Makes and Breaks Safety Fine-tuning? A Mechanistic Study in NeurIPS 2024.
Sep, 2024	I am serving as the General Chair of 3rd IEEE Conference on Secure and Trustworthy Machine Learning.
Sep, 2024	We are organising the ENCORE Workshop on Defining Holistic Private Data Science for Practice in UCSD along with Clement Canonne, Rachel Cummings, and Bailey Kacsmar.
Jul, 2024	We will present two papers titled Provable Privacy with Non-Private Pre-Processing and The Role of Learning Algorithms in Collective Action in ICML 2024.
May, 2024	Our paper titled On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective is accepted to COLT 2024. See you in Edmonton!

Publications

*AISTATS, 2025*	Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisation Amartya Sanyal , Yaxi Hu, Yaodong Yu, Yian Ma Yixin Wang ,Bernhard Schölkopf International Conference on Artificial Intelligence and Statistics, 2025 Arxiv /
*ICLR, 2025*	PSA: Differentially Private Steering for Large Language Model Alignment Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal International Conference on Learning Representations, 2025 Arxiv / Proceedings / code /
*ICLR, 2025*	Provable unlearning in topic modeling and downstream tasks Stanley Wei, Sadhika Malladi, Sanjeev Arora, Amartya Sanyal International Conference on Learning Representations, 2025 Arxiv / Proceedings /
*ICLR, 2025*	Protecting against simultaneous data poisoning attacks Neel Alex, Shoaib Ahmed Siddiqui, Amartya Sanyal, David Krueger International Conference on Learning Representations, 2025 Arxiv / Proceedings /
*TMLR, 2024*	Corrective Machine Unlearning Shashwat Goel, Ameya Prabhu, Philip Torr, Ponnurangam Kumaraguru, Amartya Sanyal Transactions on Machine Learning Research, 2024 Arxiv / Proceedings / code /
*NeurIPS, 2024*	Robust Mixture Learning when Outliers Overwhelm Small Groups Daniil Dmitriev, Rares-Darius Buhai, Stefan Tiegel, Alexander Wolters, Gleb Novikov, Amartya Sanyal , David Steurer, Fanny Yang Advances in Neural Information Processing Systems, 2024 Arxiv /
*NeurIPS, 2024*	What Makes and Breaks Safety Fine-tuning? A Mechanistic Study Samyak Jain, Ekdeep Singh Lubana, Kemal Oksuz, Tom Joy, Philip H.S. Torr, Amartya Sanyal , Puneet K. Dokania Advances in Neural Information Processing Systems, 2024 Arxiv /
*TPDP, COLT, 2024*	On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective Daniil Dmitriev, Kristóf Szabó, Amartya Sanyal Conference on Learning Theory, 2024 Arxiv / Proceedings /
*ICML, 2024*	The Role of Learning Algorithms in Collective Action Omri Ben Dov* , Jake Fakes* , Samira Samadi, Amartya Sanyal International Conference on Machine Learning, 2024 Arxiv / Proceedings /
*TPDP, ICML, 2024*	Provable Privacy with Non-Private Pre-Processing Yaxi Hu, Amartya Sanyal , Bernhard Schölkopf International Conference on Machine Learning, 2024 Arxiv / Proceedings /
*TPDP, AISTATS, 2024*	Certified private data release for sparse Lipschitz functions Konstantin Donhauser, Johan Lokna, Amartya Sanyal , March Boedihardjo, Robert Hönig, Fanny Yang, Artificial Intelligence and Statistics Theory and Practice of Differential Privacy , 2024 Arxiv / Proceedings /
NeurIPS, 2023 Spotlight Paper	Can semi-supervised learning use all the data effectively? A lower bound perspective Gizem Yüce* , Alexandru Țifrea , Amartya Sanyal , Fanny Yang Advances in Neural Information Processing Systems, 2023 Spotlight Paper Arxiv / Proceedings /
*TPDP, SaTML, 2023*	PILLAR: How to make semi-private learning more effective Francesco Pinto, Yaxi Hu, Fanny Yang, Amartya Sanyal IEEE Conference on Secure and Trustworthy Machine Learning Theory and Practice of Differential Privacy Workshop on Pitfalls of limited data and computation for Trustworthy ML , 2023 Arxiv / Proceedings / code / poster /
*TMLR, 2023*	Catastrophic overfitting can be induced with discriminative non-robust features Guillermo Ortiz-Jiménez, Pau de Jorge, Amartya Sanyal , Adel Bibi, Puneet Dokania, Pascal Frossard, Gregory Rogéz, Philip H.S. Torr TMLR , 2023 Arxiv / code /
*ICML, 2023*	Certifying Ensembles: A General Certification Theory with S-Lipschitzness Aleksandar Petrov, Francisco Eiras, Amartya Sanyal , Philip H.S. Torr, Adel Bibi International Conference on Machine Learning , 2023 Arxiv /
NeurIPS Workshop, 2023 Oral Paper	How robust accuracy suffers from certified training with convex relaxations Piersilvio De Bartolomeis, Jacob Clarysse, Fanny Yang, Amartya Sanyal Workshop on Understanding Deep Learning Through Empirical Falsification , 2023 Oral Paper Arxiv / Proceedings /
*Preprint, 2023*	Towards Adversarial Evaluations for Inexact Machine Unlearning Shashwat Goel, Ameya Prabhu, Amartya Sanyal , Ser-Nam Lim, Philip Torr, Ponnurangam Kumaraguru Arxiv, 2023 Arxiv /
*ICLR, 2023*	A law of adversarial risk, interpolation, and label noise Daniel Paleka, Amartya Sanyal International Conference on Learning Representations, 2023 Arxiv / Proceedings / poster /
*ICLR, 2023*	How robust is unsupervised representation learning to distribution shift? Yuge Shi, Imant Daunhawer, Julia E. Vogt, Philip H.S. Torr , Amartya Sanyal . International Conference on Learning Representations, 2023 Arxiv / Proceedings /
*NeurIPS, 2022*	Make Some Noise: Reliable and Efficient Single-Step Adversarial Training Pau De Jorge Aranda, Adel Bibi , Ricardo Volpi, Amartya Sanyal , Philip H.S. Torr, Grégory Rogez , Puneet Dokania Neural Information Processing Systems (NeurIPS) 2022., 2022 Arxiv / Proceedings /
UAI, 2022 Oral Paper	How unfair is private learning ? Amartya Sanyal , Yaxi Hu, Fanny Yang Conference on Uncertainty in Artificial Intelligence (UAI) , 2022 Oral Paper Arxiv / Proceedings / poster / slides /
*COLT, 2022*	Open Problem: Do you pay for Privacy in Online Learning ? Amartya Sanyal , Giorgia Ramponi Conference on Learning Theory (COLT) Open Problems, 2022 Arxiv / Proceedings / slides /
ICLR, 2021 Spotlight Paper	How benign is benign overfitting? Amartya Sanyal , Varun Kanade, Puneet Dokania, Philip H.S. Torr International Conference of Learning Representations , 2021 Spotlight Paper Arxiv / Proceedings / poster /
*ICLR, 2021*	Progressive Skeletonization: Trimming more fat from a network at initialization Pau De Jorge Aranda, Amartya Sanyal , Harkirat Behl , Philip H.S. Torr, Grégory Rogez , Puneet Dokania International Conference of Learning Representations , 2021 Arxiv / Proceedings / code /
*NeurIPS, 2020*	Calibrating Deep Neural Networks using Focal Loss Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal , Stuart Golodetz, Philip H.S. Torr, Puneet Dokania Advances in Neural Information Processing Systems (NeurIPS), 2020 Arxiv / Proceedings / code /
ICLR, 2020 Spotlight Paper	Stable Rank Normalization for Improved Generalization in Neural Networks and GANs Amartya Sanyal , Philip H.S. Torr, Puneet Dokania International Conference of Learning Representations , 2020 Spotlight Paper Arxiv / Proceedings / video /
*Preprint, 2019*	Robustness via Deep Low-Rank Representations Amartya Sanyal , Varun Kanade, Philip H.S. Torr, Puneet Dokania Preprint, 2019 Arxiv /
*ICML, 2018*	TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service Amartya Sanyal , Matt Kusner , Adrià Gascón Varun Kanade International Conference of Machine Learning (ICML), 2018 Arxiv / Proceedings / code /
*ICML Workshop, 2017*	Multiscale sequence modeling with a learned dictionary Bart van Merriënboer, Amartya Sanyal , Hugo Larochelle, Yoshua Bengio Machine Learning in Speech and Language Processing, 2017 Arxiv /

Design and source code from Jon Barron's website

Amartya Sanyal

Openings with me

Research Interests

Current Students

Upcoming/Recent Talks

Recent News

Jan, 2025

I received the Villum Young Invesigator Grant and will be working on topics related to Privacy, unlearning, and online learning. I will be hiring motivated PhD students and Postdocs. Email me if you think you would be good match.

Jan, 2025

Three papers on differentially private alignment of LLMs, Machine unlearning, and training data attribution were accepted at ICLR 2025 and one paper on OOD robustness and label noise at AISTATS 2025 .

Oct, 2024

Alex Tifrea , and I are giving a tutorial on Building trustworthy ML: The role of label quality and availability” at AAAI, 2025.

Oct, 2024

Our paper titled Corrective Machine Unlearning is accepted at TMLR .

Sep, 2024

We will present two papers titled Robust Mixture Learning when Outliers Overwhelm Small Groups and TWhat Makes and Breaks Safety Fine-tuning? A Mechanistic Study in NeurIPS 2024.

Sep, 2024

I am serving as the General Chair of 3rd IEEE Conference on Secure and Trustworthy Machine Learning.

Sep, 2024

We are organising the ENCORE Workshop on Defining Holistic Private Data Science for Practice in UCSD along with Clement Canonne, Rachel Cummings, and Bailey Kacsmar.

Jul, 2024

We will present two papers titled Provable Privacy with Non-Private Pre-Processing and The Role of Learning Algorithms in Collective Action in ICML 2024.

May, 2024

Our paper titled On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective is accepted to COLT 2024. See you in Edmonton!

Publications

Accuracy on the wrong line: On the pitfalls of noisy data for out-of-distribution generalisation

PSA: Differentially Private Steering for Large Language Model Alignment

Provable unlearning in topic modeling and downstream tasks

Protecting against simultaneous data poisoning attacks

Corrective Machine Unlearning

Robust Mixture Learning when Outliers Overwhelm Small Groups

What Makes and Breaks Safety Fine-tuning? A Mechanistic Study

On the Growth of Mistakes in Differentially Private Online Learning: A Lower Bound Perspective

The Role of Learning Algorithms in Collective Action

Provable Privacy with Non-Private Pre-Processing

Certified private data release for sparse Lipschitz functions

Can semi-supervised learning use all the data effectively? A lower bound perspective

PILLAR: How to make semi-private learning more effective

Catastrophic overfitting can be induced with discriminative non-robust features

Certifying Ensembles: A General Certification Theory with S-Lipschitzness

How robust accuracy suffers from certified training with convex relaxations

Towards Adversarial Evaluations for Inexact Machine Unlearning

A law of adversarial risk, interpolation, and label noise

How robust is unsupervised representation learning to distribution shift?

Make Some Noise: Reliable and Efficient Single-Step Adversarial Training

How unfair is private learning ?

Open Problem: Do you pay for Privacy in Online Learning ?

How benign is benign overfitting?

Progressive Skeletonization: Trimming more fat from a network at initialization

Calibrating Deep Neural Networks using Focal Loss

Stable Rank Normalization for Improved Generalization in Neural Networks and GANs

Robustness via Deep Low-Rank Representations

TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service

Multiscale sequence modeling with a learned dictionary