Research talk: Differentially private fine-tuning of large language models

Published on 27 Oct 2022, 15:40
We have come a long way in terms of protecting privacy when training ML models, particularly with large language models. We recently demonstrated that using differentially private stochastic gradient descent (DP-SGD) to fine-tune very large language models, such as GPT-3, is not only feasible but shows very promising results with respect to the privacy-utility tradeoff. In this talk, we highlight the challenges we have overcome over the past year and the opportunities our research enables for a range of product applications.


See related sessions in this track:

Learn more about the 2022 Microsoft Research Summit: