Probabilistic Inference Group (PIGS) - Archive

Meetings in 2007

Tue 13 Nov 2007 (Kian Ming)

* Time-Varying Topic Models using Dependent Dirichlet Processes by Nathan Srebro, Sam Roweis http://people.csail.mit.edu/nati/Publications/UTML-TR-2005-003.pdf

* Order-Based Dependent Dirichlet Processes by Griffin, J.E; Steel, M.F.J http://www.ingentaconnect.com/content/asa/jasa/2006/00000101/00000473/art00020

Tue 16 October 2007 (Nicolas Heess)

David A. Ross & Richard S. Zemel. Learning Parts-Based Representations of Data. JMLR 7:2369-2397, 2006. http://www.jmlr.org/papers/volume7/ross06a/ross06a.pdf.

Tue 2 October 2007 (Amos)

* I'll present the paper

Bayesian Policy Learning with Trans-Dimensional MCMC by Matthew Hoffman, Arnaud Doucet, Nando de Freitas, Ajay Jasra

http://www.cs.ubc.ca/~nando/papers/pomdp1.pdf

which will be presented at the up and coming NIPS.

Tue 18 September 2007 (Chris Williams)

* Particle filters for mixture models with an unknown number of components. Paul Fearnhead, Statistics and Computing 14 11-21 (2004) Access via http://carlin.lib.ed.ac.uk:2109/content/8l7lxnpcgnbv/?p=a8a8e5f30a9f40d09853ef35b26d3918&pi=14

* A lecture on compressive sensing. Richard Baraniuk, IEEE Signal Proc Magazine July 2007 http://www.dsp.ece.rice.edu/cs/compressiveSensing-IEEE-SPMag-LectureNotes-15web.pdf

Tue 4 September 2007 (UAI 2007 session)

Each person to select a paper from UAI2007 on which to give a brief overview. Please post paper and URL below.

AS: Nonparametric Bayes Pachinko Allocation http://www.cs.umass.edu/~mccallum/papers/npbpam-uai2007s.pdf

LM: Large-Flip Importance Sampling, Firas Hamze and Nando de Freitas. http://www.cs.ubc.ca/~fhamze/MyPapers/UAI2007Final.pdf

[ckiw] Survey Propagation Revisited. Lukas Kroc, Ashish Sabharwal, Bart Selman. http://www.cs.cornell.edu/~sabhar/publications/surveyPropUAI07.pdf

[EB]: Learning Probabilistic Relational Dynamics for Multiple Tasks. Ashwin Deshpande, Brian Milch, Luke Zettlemoyer and Leslie Kaelbling. pdf

Tue 21 August 2007 (Lawrence Murray)

The topic will be parameter estimation within particle filters, using the following paper as a basis:

Doucet, A. & Tadic, V. B. Parameter estimation in general state-space models using particle methods. Annals of the Institute of Statistical Mathematics, 2003, 55, 409-422. http://carlin.lib.ed.ac.uk:2109/content/3122v536m6877217/?p=947bcb1cc4f84099adc72cd2a1b49646&pi=12

If you need a brief introduction to or recap of particle filters, I'd recommend Section 4 of the following:

Isard, M. & Blake, A. Condensation -- Conditional Density Propagation for Visual Tracking. International Journal of Computer Vision, 1998, 29, 5-28. http://www.robots.ox.ac.uk/~ab/abstracts/ijcv98.html

It's not thorough, but it is accessible and succinctly gives the most basic ideas. If you're really pushed for time, just watch the pretty animation at http://www.robots.ox.ac.uk/~misard/images/anim.mpg!

Tue 7 August 2007 (Mikio Braun)

Effective Dimensionality of Infinite-dimensional Feature Spaces: Understanding the Kernel Matrix by Convergence of Spectral Properties

It is well known that certain kernel functions give rise to infinite-dimensional feature spaces. Using accurate approximation bounds for eigenvalues of the kernel matrix and spectral projections with eigenvectors, we show that the relevant information about a learning problem is contained in a low-dimensional subspace. This insight leads to an important addition to the usual explanation of kernel methods which focusses on capacity control only to the effect that capacity control works because a suitable kernel constructs an embedding which makes economic use of feature space dimensions. Some implications for Gaussian processes are also discussed.

Tue 24 Jul 2007 (Kian Ming Adam)

Latent models for cross-covariance http://dx.doi.org/10.1016/j.jmva.2004.11.009

Tue 26 June 2007 ICML 2007 session 2

JQ: Most Likely Heteroscedastic Gaussian Process Regression. Kersting et al. pdf

SB: The Hierarchical Gaussian Process Latent Variable Model. Neil D. Lawrence and Andrew J. Moore. pdf

KMC: Uncovering Shared Structures in Multiclass Classification. Yonatan Amit, Michael Fink, Nathan Srebro and Shimon Ullman pdf

EB: Two-view Feature Generation Model for Semi-supervised Learning. Rie Ando and Tong Zhang. pdf

AD: Parameter Learning for Relational Bayesian Networks. Manfred Jaeger. pdf

AS: Three new Graphical Models for Statistical Language Modelling pdf
AS:An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation pdf

CW: Infinite Mixtures of Trees. Sergey Kirshner and Padhraic Smyth. pdf

FA: Conditional Random Fields for Multi-agent Reinforcement Learning. X. Zhang et al. pdf

SH: A Permutation-augmented Sampler for DP Mixture Models pdf

NH: Self-taught Learning: Transfer Learning from Unlabeled Data. Raina et al. pdf

Tue 12 June 2007 ICML 2007 session

An all in one copy of this is temporarily available from ~amos/public/all.pdf. Useful hint: to concatenate pdfs use texexec --pdfarrange --result all.pdf docdirectory/*.pdf

JQ: Modeling Changing Dependency Structure in Multivariate Time Series. Xiang Xuan and Kevin Murphy. pdf

SB: Multifactor Gaussian Process Models for Style-Content Separation. Jack M. Wang, David J. Fleet and Aaron Hertzmann. pdf

KMC: The Matrix Stick-Breaking Process for Flexible Multi-Task Learning. Ya Xue and David Dunson and Lawrence Carin pdf

EB: Bayesian Actor-Critic Algorithms. Mohammad Ghavamzadeh and Yaakov Engel. pdf

AD: Relational Clustering by Symmetric Convex Coding. Bo Long, Zhongfei Zhang, Xiaoyun Wu and Philip S. Yu. pdf

CW: Large-scale RLSC Learning Without Agony. Wenye Li, Kin-Hong Lee and Kwong-Sak Leung. pdf

FA: Asymptotic Bayesian Generalization Error When Training and Test Distributions Are Different. K. Yamazaki et al. pdf

SH: Restricted Boltzmann Machines for Collaborative Filtering pdf

MA: Unsupervised Prediction of Citation Influences

Tue 1 May 2007 (Chris Williams & Amos Storkey)

Deep belief nets.

Y. Bengio and Y. Le Cun Scaling Learning Algorithms towards AI. To appear in "Large-Scale Kernel Machines", L. Bottou, O. Chapelle, D. DeCoste, J. Weston (eds) MIT Press, 2007.
http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+lecun_chapter2007.pdf [esp sections 1-3]

Hinton, G. E. To recognize shapes, first learn to generate images. Technical Report UTML TR 2006-004.
http://www.cs.toronto.edu/~hinton/absps/montrealTR.pdf

Tue 17 Apr 2007 (Wolfgang Lehrach)

Laskey, K.B. & Myers J.W. Population Markov Chain Monte Carlo. Machine Learning 50:175-196, 2003. http://staff.science.uva.nl/~mtjspaan/emergentia/population_mcmc.pdf

Jasra, A., Stephens D.A. & Holmes, C.C. Population-based reversible jump Markov chain Monte Carlo. http://stats.ma.ic.ac.uk/das01/public_html/Papers/MULTMIXRJPOP4.PDF

Tue 3 Apr 2007 (Felix Agakov)

Causality

• NIPS 2006 workshop on causality and feature selection
  http://research.ihost.com/cws2006/
  Gregory F. Cooper, "An Introduction to Causal Modeling and Discovery Using Graphical Models".
  http://research.ihost.com/cws2006/Cooper%20NIPS%20workshop%20talk%20gfc%20revised.ppt
  X. Sun, D. Janzing, B. Schoelkopf, "Inferring causal directions by evaluating the complexity of conditional distributions".
  http://research.ihost.com/cws2006/janzingNIPS2006Final.pdf
  
  Sun et. al.'s talk is about using the Hilbert-Schmidt criterion for independence testing; Cooper's talk is an introduction.

• P. Spirtes, R. Scheines, C. Glymour, T. Richardson, and C. Meek. "Causal Inference", The SAGE Handbook of Quantitative Methodology for the Social Sciences, D. Kaplan, ed., SAGE Publications, Thousand Oaks, CA., 2004, pp. 447-477.
  http://www.phil.cmu.edu/projects/tetrad/download/Kaplan%20Chapter%2024.pdf
  
  A review paper which talks about constraint-based and Bayesian algorithms, latent variables, etc.

• Dash and Druzdzel, "Robust Independence Testing for Constraint-Based Learning of Causal Structure", UAI 2003.
  http://www.pitt.edu/~druzdzel/psfiles/uai03.pdf
  
  A relatively recent paper on improving CB algorithms, which uses other independence tests.

• Ali et. al., "Towards characterizing Markov equivalence Classes for Directed Acyclic Graphs with Latent Variables", UAI 2005.
  http://www.phil.cmu.edu/projects/tetrad/download/uai_2005_23mar2005.pdf
  
  Discusses effects of latent variables on the causal explanations of the observations.

Mon 19 Mar 2007 (David MacKay)

Nested sampling.

Tue 6 Mar 2007 (John Quinn)

A. Hyvärinen. Estimation of non-normalized statistical models using score matching. Journal of Machine Learning Research 6:695-709, 2005. http://www.cs.helsinki.fi/u/ahyvarin/papers/JMLR05.pdf

-- AthinaSpiliopoulou - 27 Jan 2009