|
ZOOM LINK: https://uwmadison.zoom.us/j/96126547182
Abstract: By automating the error-prone math behind deep learning, systems such as TensorFlow and PyTorch have supercharged machine learning research, empowering hundreds of thousands of
practitioners to rapidly explore the design space of neural network architectures and training algorithms. In this talk, I will show how new programming language techniques, especially generalizations of automatic differentiation, make it possible to generalize
and extend such systems to support probabilistic models. Our automation is rigorously proven sound using new semantic techniques for reasoning compositionally about expressive probabilistic programs, and static types are employed to ensure important preconditions
for soundness, eliminating large classes of implementation bugs. Providing a further boost, our tools can help users correctly implement fast, low-variance, unbiased estimators of gradients and probability densities that are too expensive to compute exactly,
enabling orders-of-magnitude speedups in downstream optimization and inference algorithms.
To illustrate the value of these techniques, Iâll show how they have helped us experiment with new architectures that could address key challenges with todayâs dominant AI models. In particular, Iâll showcase systems weâve built for (1) auditable reasoning
and learning in relational domains, enabling the detection of thousands of errors across millions of Medicare records, and (2) probabilistic inference over large language models, enabling small open models to outperform GPT-4 on several constrained generation
benchmarks.
Bio: Alex Lew is a final-year PhD student at MITâs Probabilistic Computing Project, co-advised by Vikash Mansinghka and Josh Tenenbaum, and supported by an NSF Graduate Research Fellowship.
Before coming to MIT, he taught high-school computer science at Commonwealth School in Boston. And before that, he was a student at Yale, where he received a B.S. in computer science and math in 2015. (http://alexlew.net/)
|
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}