TWiki> ANC Web>PIGS (10 Feb 2017, Main.sseth)EditAttach

Probabilistic Inference Group (PIGS)

The Probabilistic Inference Group (PIGS) is a paper discussion group with meetings held fortnightly. The group focuses on probabilistic and information theoretic approaches to machine learning problems. Meetings are generally held fortnightly on Mondays at 10:30am in room 2.33 of the Informatics Forum. Announcements are made through the PIGS mailing list.

Instructions for presenters:

1) Choose a mainstream ML paper (or two).
2) Provide paper(s) at least one week in advance of the meeting.
3) Lead a discussion of the paper(s) in the meeting.

Mainstream means a paper that does not depend heavily on domain specific background to be comprehensible. A possible test would be to consider if the techniques could fairly readily be transferred to another application area. Papers from conferences like NIPS, ICML, UAI, AISTATS, and journals like JMLR and ML papers from IEEE PAMI are likely to be in scope; but note that papers from other sources could well fit too.

Students should discuss their paper selections with their supervisor to make sure they are reasonable choices. It is acceptable to relate the selected papers to the presenter's research, but not at the expense of discussion of the selected paper.

If people want to make thematic groupings of readings it should be possible to arrange swaps in the rota in order to make this happen.

Upcoming discussions:

February 13: Elliot (confirmed) 4.31-4.33

Elliot will lead the discussion on "Adversarial Feature Learning" (arXiv), and perhaps also touch upon "Energy-based Generative Adversarial Networks" (arXiv).

February 27: Vaishak (confirmed) 2.33

Vaishak will give a brief overview of the following papers,

Probabilistic Inference in Hybrid Domains by Weighted Model Integration. Belle, V.; Passerini, A.; and Van den Broeck, G. In IJCAI, 2015. link

Hashing-based Approximate Probabilistic Inference in Hybrid Domains. Belle, V.; Van den Broeck, G.; and Passerini, A. In UAI, 2015. link

Component Caching in Hybrid Domains with Piecewise Polynomial Densities. Belle, V.; Van den Broeck, G.; and Passerini, A. In AAAI, 2016. link

March 6: Tim (confirmed) 4.31/4.33

March 13: Iain (confirmed) 5.42

March 27: Theo (confirmed) 2.33

April 10: Antonio 5.42

April 24: Harri (confirmed) 2.33

May 8: AISTATS - 4.31-4.33

May 22: James (confirmed) 2.33

June 5: Gavin 2.33

June 19: Charlie 2.33

July 3: Amos 2.33

July 17: Michael 5.42

July 31: Naomi 2.33

14th 5.42

28th 2.33

11th 5.42

25th 2.33

9th 5.42

23rd 2.33

6th 2.33

28th 2.33

4th 2.33

18th 2.33

Past discussions:

January 30: Matt (confirmed) 5.42

Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space.
Anh Nguyen, Jason Yosinski, Yoshua Bengio, Alexey Dosovitskiy, Jeff Clune. arXivwebsite

January 16: Chris (confirmed) (5.42)

The session will be about Generative Adversial Networks (GANs) and Noise Contrastive Estimation (NCE).

We will read

On distinguishability criteria for estimating generative models
Ian J. Goodfellow


Statistical Inference of Intractable Generative Models via Classification
Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski, and Jukka Corander

December 19: NIPS 2016 review

George: Learning to Learn by Gradient Descent by Gradient Descent,

Sohan: Bayesian optimization for automated model selection, Malkomes et al., PDF

Chris: Improving Variational Autoenciders with Inverse Autoregressive Flow, Kingma et al, PDF

Michael: Bayesian Optimization with Robust Bayesian Neural Networks, Springenberg et al, pdf

Gavin: A Probabilistic Framework for Deep Learning, Patel et al,

Naomi: Residual networks behave like ensembles of relatively shallow networks, Veit et al,

Matt: Measuring the reliability of MCMC inference with bidirectional Monte Carlo. Grosse, Anche and Roy PDF

November 21: George (confirmed)

"Rnyi Divergence Variational Inference" by Li and Turner (pdf)

November 7: Sohan (confirmed)

"Variational Autoencoder for Deep Learning of Images, Labels and Captions" by Pu et al. (pdf)

“Attribute2Image: Conditional Image Generation from Visual Attributes” by Yan et al. (pdf)

August 8: Jaroslav

Presentation of his KDD 2016 paper on: A Subsequence Interleaving Model for Sequential Pattern Mining


Recent sequential pattern mining methods have used the minimum description length (MDL) principle to define an encoding scheme which describes an algorithm for mining the most compressing patterns in a database. We present a novel subsequence interleaving model based on a probabilistic model of the sequence database, which allows us to search for the most compressing set of patterns without designing a specific encoding scheme. Our proposed algorithm is able to efficiently mine the most relevant sequential patterns and rank them using an associated measure of interestingness. The efficient inference in our model is a direct result of our use of a structural expectation-maximization framework, in which the expectation-step takes the form of a submodular optimization problem subject to a coverage constraint. We show on both synthetic and real world datasets that our model mines a set of sequential patterns with low spuriousness and redundancy, high interpretability and usefulness in real-world applications. Furthermore, we demonstrate that the quality of the patterns from our approach is comparable to, if not better than, existing state of the art sequential pattern mining algorithms.

July 11: ICML 2016 review

Matt: Slice Sampling on Hamiltonian Trajectories, Benjamin Bloem-Reddy and John P. Cunningham

Harri: Associative Long Short-Term Memory | Ivo Danihelka et al.

Pol: Autoencoding beyond pixels using a learned similarity metric Larsen et. al

Gavin: Noisy Activation Functions Caglar Gulcehre et al

June 20: Lukasz

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models

S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, Geoffrey E. Hinton

Learning to decompose for object detection and instance segmentation

Eunbyung Park, Alexander C. Berg

June 6: Charlie

Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation Jonathan Thompson, Arjun Jain, Yann LeCun and Christoph Bregler

Conditional Random Fields as Recurrent Neural Networks Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip H. S. Torr

May 23: Weng-Keen

Presentation of his ICML 2016 paper on: Efficient Multi-Instance Learning for Activity Recognition from Time Series Data Using an Auto-Regressive Hidden Markov Model


Activity recognition from sensor data has spurred a great deal of interest due to its impact on health care. Prior work on activity recognition from multivariate time series data has mainly applied supervised learning techniques which require a high degree of annotation effort to produce training data with the start and end times of each activity. In order to reduce the annotation effort, we present a weakly supervised approach based on multi-instance learning. We introduce a generative graphical model for multi-instance learning on time series data based on an auto-regressive hidden Markov model. Our approach models both the structure within an instance as well as the structure between instances in a bag. Our model has a number of advantages, including the ability to produce both bag and instance-level predictions as well as an efficient exact inference algorithm based on dynamic programming.

April 25: Amos

Knowledge Matters: Importance of Prior Information for Optimization C. Gulcehre and Yoshua Bengio

April 11: Chris

A survey of techniques for incremental learning of HMM parameters Wael Khreich Eric Granger, Ali Miri, Robert Sabourin

March 14: Theo


Feb 29: George

Automatic Variational Inference in Stan Alp Kucukelbir, Rajesh Ranganath, Andrew Gelman and David Blei

Feb 15: Matt

A note on the evaluation of generative models Lucas Theis, Aƒ¤ron van den Oord and Matthias Bethge

Feb 1: Sohan

Robust Spectral Inference for Joint Stochastic Matrix Factorization Moontae Lee, David Bindel and David Mimno

Jan 18: NIPS 2015 review

George: Bayesian Dark Knowledge Anoop Korattikara, Vivek Rathod, Kevin Murphy, Max Welling

Mingjun: Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference Edward Meeds, Max Welling

Harri: Semi-supervised learning with ladder networks Antti Rasmus, Harri Valpola et al

Theo: Generative Image Modeling Using Spatial LSTMs Lucas Theis, Matthias Bethge

Chris: Unsupervisd Learning by Program Synthesis Ellis, Solar-Lezama, Tenenbaum

Krzysztof: : Training Very Deep Networks Rupesh Kumar Srivastava, Klaus Greff, Jurgen Schmidhuber

Oct 26: Harri

Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Emily Denton, Soumith Chintala, Arthur Szlam, Rob Fergus

Oct 12: Gavin

Variational Dropout and the Local Reparameterization Trick Diederik P. Kingma, Tim Salimans, Max Welling

Meetings in 2015

Meetings in 2014

Meetings in 2013

Meetings in 2012

Meetings in 2011

Meetings in 2010

Meetings in 2009

Meetings in 2008

Meetings in 2007

Earlier meetings (2002-2006) on old website

Topic attachments
I Attachment Action Size Date Who Comment
zipzip manage 4178.0 K 15 Jul 2008 - 12:53 Main.s0565918  
pdfpdf latent-models-covariance.pdf manage 276.1 K 20 Jul 2007 - 13:38 Main.s9810791 Latent models for cross-covariance (PIGS 24th July 2007)
Topic revision: r462 - 10 Feb 2017 - 18:34:37 - Main.sseth
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies