Probabilistic Inference Group (PIGS) - Archive

Meetings in 2008

Tue 25 November 2008 (Chris Williams)

The discovery of structural form. Kemp, C. and Tenenbaum, J. B. (2008). Proceedings of the National Academy of Sciences. 105(31), 10687-10692.

see Josh's website for supporting information and commentary!

Tue 11 November 2008 (Jakup Piatkowski)

• Behrens et al., 2007: Probabilistic diffusion tractography with multiple fibre orientations: What can we gain?

• Jbabdi et al., 2007: A Bayesian framework for global tractography

Tue 28 October 2008

Brendan J. Frey and Delbert Dueck, Science, Vol 315, No 5814, pp 972-976, February 2007: Clustering by passing messages between data points

See also accompanying "Perspective" article. Both papers are available here: http://www.psi.toronto.edu/pubs2/publications.php

Tue 21 October 2008 (Nicolas Heess)

• R. Rao, Neural Computation 16:1-38 (2004) Bayesian Computation in Reucrrent Neural Circuits

• JM Beck, A Pouget, Neural Computation 19:1344-1361 (2007) Exact Inferences in a Neural Implementation of a Hidden Markov Model

• Omer Bobrowski , Ron Meir, Shy Shoham, Yonina Eldar (NIPS 2007) A neural network implementing optimal state estimation based on dynamic spike train decoding

Tue 23 September 2008 (Nicolas Heess)

• "Ideal Parent" Structure Learning for Continuous Variable Bayesian Networks (Gal Elidan, Iftach Nachman, Nir Friedman; JMLR 8(Aug):1799--1833, 2007)

Tue 09 September 2008 (Edwin Bonilla)

• Multi task Reinforcement Learning: A Hierarchical Bayesian Approach. (Aaron Wilson, Alan Fern, Soumya Ray, and Prasad Tadepalli. ICML-07)

• Model-based Bayesian Reinforcement Learning in Partially Observable Domains (Pascal Poupart and Nikos Vlassis. AI-Math 2008.) The following paper may be required as a background: An Analytic Solution to Discrete Bayesian Reinforcement Learning (Pascal Poupart, Nikos Vlassis, Jesse Hoey and Kevin Regan, ICML-06), which was presented at PIGS some time ago.

Tue 26 August 2008 (Kian Ming Chai)

http://www.jmlr.org/papers/volume8/pillai07a/pillai07a.pdf

Characterizing the Function Space for Bayesian Kernel Models

Natesh S. Pillai, Qiang Wu, Feng Liang, Sayan Mukherjee, Robert L. Wolpert; JMLR 8(Aug):1769--1797, 2007.

Kernel methods have been very popular in the machine learning literature in the last ten years, mainly in the context of Tikhonov regularization algorithms. In this paper we study a coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure. Priors on the random signed measures correspond to prior distributions on the functions mapped by the integral operator. We study several classes of signed measures and their image mapped by the integral operator. In particular, we identify a general class of measures whose image is dense in the reproducing kernel Hilbert space (RKHS) induced by the kernel. A consequence of this result is a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions. We discuss the construction of priors on spaces of signed measures using Gaussian and Lévy processes, with the Dirichlet processes being a special case the latter. Computational issues involved with sampling from the posterior distribution are outlined for a univariate regression and a high dimensional classification problem.

Tue 15 July 2008 (Amos Storkey)

UAI review (http://uai2008.cs.helsinki.fi/programme.shtml):

Tal El-Hay, Nir Friedman, Raz Kupferman: Gibbs Sampling in Factorized Continuous-Time Markov Processes

Gustavo Lacerda, Peter Spirtes, Joseph Ramsey, Patrik Hoyer: Discovering Cyclic Causal Models by Independent Components Analysis

Chong Wang, David Blei, David Heckerman Continuous Time Dynamic Topic Models

Tue 15 July 2008 (Chris Williams)

Download .zip file of all papers from http://icml2008.cs.helsinki.fi/papers/final-pdfs.zip, or papers below 2-up from https://wiki.inf.ed.ac.uk/pub/ANC/PIGS/icml-2up.zip

Paper #502: Data Spectroscopy: Learning Mixture Models using Eigenspaces of Convolution Operators. Tao Shi, Mikhail Belkin, and Bin Yu.

Paper #573: On the Quantitative Analysis of Deep Belief Networks. Ruslan Salakhutdinov and Iain Murray.

Paper #638: Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. Tijmen Tieleman.

Paper #413: Modeling Interleaved Hidden Processes. Niels Landwehr.

Paper #266: SVM Optimization: Inverse Dependence on Training Set Size. Shai Shalev-Shwartz and Nathan Srebro.

Paper #588: An Asymptotic Analysis of Generative, Discriminative, and Pseudolikelihood Estimators. Percy Liang and Michael Jordan.

Paper #520: Multi-Task Learning for HIV Therapy Screening. Steffen Bickel, Jasmina Bogojeska, Thomas Lengauer, and Tobias Scheffe

Paper #476: Improved Nystrom Low-Rank Approximation and Error Analysis. Kai Zhang, Ivor Tsang, and James Kwok.

Paper #241: Gaussian Process Product Models for Nonparametric Nonstationarity. Ryan Adams and Oliver Stegle.

Paper #419: Memory Bounded Inference in Topic Models. Ryan Gomes, Max Welling, and Pietro Perona.

Paper #182: Inverting the Viterbi Algorithm: an Abstract Framework for Structure Design. Michael Schnall-Levin, Leonid Chindelevitch, and Bonnie Berger

Tue 1 July 2008 (Lawrence Murray)

Fearnhead, P., Papaspiliopoulos, O. & Roberts, G.O. (2008) Particle filters for partially observed diffusions

For the temporally challenged, the following communication provides a very brief sketch of the essential points:

Fearnhead, P., Papaspiliopoulos, O. & Roberts, G.O. (2006) Particle filtering for diffusions avoiding time-discretisations IEEE Nonlinear Statistical Signal Processing Workshop, 141-143.

The following video and slides of Omiros Papaspiliopoulos's talk at a recent Newton Institute workshop may also be useful in understanding the material:

http://www.newton.ac.uk/webseminars/pg+ws/2008/sch/schw05/0620/papaspiliopoulos/

Tue 17 June 2008 (Edwin Bonilla)

• Katherine Heller, Sinead Williamson and Zoubin Ghahramani: Statistical Models for Partial Membership. To appear in ICML'08.
• Raquel Urtasun, David Fleet, Andreas Geiger, Jovan Popovic, Trevor Darrell and Neil Lawrence: Topologically-Constrained Latent Variable Models. To appear in ICML'08.

Tue 11 May 2008 (Nicolas Heess)

Unsupervised Grammar Learning

• Stolcke & Omohundro, 1994, ICGI: Inducing Probabilistic Grammars by Bayesian Model Merging
• S.F. Chen, 1995: Bayesian Grammar Induction for Language Modeling
• C. de Marcken, 1996, ACL: Linguistic Structure as Composition and Perturbation
• A. Clark, 2001: Unsupervised Induction of Stochastic Context-Free Grammars using Distributional Clustering
• Klein & Manning, 2001, NIPS: Natural Language Grammar Induction using a Constituent-Context Model

Tue 29 Apr 2008 (Kian Ming Chai)

J. N. Corcoran and R. L. Tweedie (2002) Perfect sampling from independent Metropolis-Hastings chains http://dx.doi.org/10.1016/S0378-3758(01)00243-9

Tue 15 Apr 2008 (Amos Storkey)

Neal, R.M. Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better. http://www.cs.toronto.edu/~radford/ftp/asymvar.pdf

Tue 1 Apr 2008 (Chris Williams)

Fukumizu, K., A. Gretton, X. Sun and B. Schölkopf: Kernel Measures of Conditional Dependence. Proceedings of the 20th Neural Information Processing Systems Conference (NIPS 2007), 1-13, MIT Press, Cambridge, Mass., USA (in press) (01 2008)

http://www.ism.ac.jp/~fukumizu/papers/fukumizu_etal_nips2007_extended.pdf

Gretton, A., K. Fukumizu, C. H. Teo, L. Song, B. Schölkopf and A. J. Smola: A Kernel Statistical Test of Independence. Proceedings of the 20th Annual Conference on Neural Information Processing Systems (NIPS 2007), 1-8, MIT Press, Cambridge, Mass., USA (in press) (01 2008)

http://www.kyb.mpg.de/publications/attachments/NIPS2007-Gretton_[0].pdf

Tue 11 Mar 2008 (Lawrence Murray)

• Holmes, M.P.; Gray, A. G. & Isbell, C.L. Jr. Ultrafast Monte Carlo for Statistical Summations. NIPS 2007.
• Gray, A. G. & Moore, A. W. Rapid Evaluation of Multiple Density Models. AI Stats 2003.

Tue 26 Feb 2008 (Matthias Seeger)

Expectation Propagation -- Experimental Design for the Sparse Linear Model

Expectation propagation (EP) is a novel variational method for approximate Bayesian inference, which has given promising results in terms of computational efficiency and accuracy in several machine learning applications. It can readily be applied to inference in linear models with non-Gaussian priors, generalised linear models, or nonparametric Gaussian process models, among others, yet has not been used in Statistics so far to our knowledge. I will give an introduction to this framework. I will then show how to address sequential experimental design for a linear model with non-Gaussian sparsity priors, giving some results in two different machine learning applications. These results indicate that experimental design for these models may have significantly different properties than for linear-Gaussian models, where Bayesian inference is analytically tractable and experimental design seems best understood. In fact, in the applications we considered, the quality of sequentially optimised designs improved more significantly over random designs, if non-Gaussian priors were employed. A satisfactory explanation for this beneficial interplay would be of high importance, yet has not been given, to our knowledge. I will show recent results in the area of measuring images linearly, which shed interesting light on the very active area of compressed sensing.

Tue 12 Feb 2008 (Edwin Bonilla)

• Ruslan Salakhutdinov, Andriy Mnih. Probabilistic Matrix Factorization.

• Shipeng Yu, Balaji Krishnapuram, Romer Rosales, Harald Steck, R. Bharat Rao. Bayesian Co-Training.

Tue 29 Jan 2008 (NIPS review 2)

Please select a paper on which to give a brief (5 min) overview and list it here.

• (LM) S. Siddiqi, B. Boots, and G. Gordon. A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
• (KMC) M. M. Mahmud, Sylvian Ray. Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations
• (AD) Max Welling, Ian Porteous, Evgeniy Bart Infinite State Bayes-Nets for Structured Domains
• (EVBP) Nicolas Chapados, Yoshua Bengio Augmented Functional Time Series Representation and Forecasting with Gaussian Processes

Tue 15 Jan 2008 (NIPS review 1)

Please select a paper and list it here.

• (LM) Cédric Archambeau, Manfred Opper, Yuan Shen, Dan Cornford, John Shawe-Taylor. Variational Inference for Diffusion Processes
• (KMC) Ricardo Silva, Wei Chu, Zoubin Ghahramani. Hidden Common Cause Relations in Relational Learning
• (CKIW) Sparse deep belief net model for visual area V2, Honglak Lee, Ekanadham Chaitanya, and Andrew Y. Ng. Paper has been circulated by email.
• (EB) David Wingate and Satinder Singh Exponential Family Predictive Representations of State
• (NH) Percy Liang, Dan Klein, Michael Jordan Agreement-Based Learning
• (AJS) Marc'Aurelio Ranzato, Y-Lan Boureau, Yann LeCun Sparse Feature Learning for Deep Belief Networks
• (AD) Peter Hoff Modeling homophily and stochastic equivalence in symmetric relational data

All papers as one PDF here (excluding CKIW's, see email).