Pyro Now On Watson Machine Learning

Pyro, a universal probabilistic programming language (PPL) written in python and supported by PyTorch on the backend, is now available on IBM's Watson Machine Learning platform with PyTorch 0.4

Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models. It uses stochastic variational inference algorithms and probability distributions built on top of PyTorch, a GPU-accelerated deep learning framework. Pyro also uses Poutine, a library of composable building blocks for modifying the behavior of probabilistic programs. This means it can accommodate complex or model-specific algorithmic behavior.

The developers of Pyro say it unifies the best of modern deep learning and Bayesian modeling. It was designed to be as universal as possible, with the intention of being able to represent any computable probability distribution. Pyro also aims to be scalable, and to be minimal due to it being implemented with a small core of powerful, composable abstractions.

Pyro can be used to describe inference problems, roughly translated as:

“what must be true if this random choice had a certain observed value?”

Probabilistic programming systems provide universal inference algorithms that can perform inference with little intervention from the user. In Pyro, you describe the inference in a model via a second model called an inference model, or guide in Pyro. It's not possible to write the 'correct' guide, so you specify a family of guides with parameters, then optimize your model to move the guide to the desired outcome. This optimization is automated via automatic differentiation

Pyro builds on the PyTorch library, which includes automatic differentiation using very fast, GPU-accelerated tensor math. PyTorch constructs gradients dynamically, which enables Pyro programs to include stochastic control structure, that is, random choices in a Pyro program can control the presence of other random choices in the program. Stochastic control structure is crucial to make a PPL universal.

In Pyro, both the generative models and the inference guides can include deep neural networks as components. The developers say that the resulting deep probabilistic models have shown great promise in recent work, especially for unsupervised and semi-supervised machine learning problems.