We will discuss the following paper:

- Andrew Wilson, Zoubin Ghahramani (NIPS 2010): Copula Processes

We will discuss the following paper:

- Weinberger, K., Dasgupta, A., Langford, J., Smola, A. and Attenberg, J. (ICML 2009): Feature hashing for large scale multitask learning

Amos will give an overview of a number of papers around the subject of sequential Monte-Carlo, particle MCMC and inference and parameter estimation for continuous time systems.

This session will be more of a presentation than a paper reading and is intended to give a basic encapsulation of a number of papers, thus covering several topics in summary.

We will discuss the following papers:

- Sinead Williamson, Peter Orbanz, Zoubin Ghahramani: Dependent Indian Buffet Processes
- Yener Ulker, Bilge Günsel, Taylan Cemgil: Sequential Monte Carlo Samplers for Dirichlet Process Mixtures

We will discuss the following paper:

- D. A. Henderson, R. J. Boys, and D. J. Wilkinson, Biometrics 66 (1), 2009: Bayesian Calibration of a Stochastic Kinetic Computer Model Using Multiple Data Sources

We will discuss the following papers:

- Zeiler, M.D., Kirshnan, D., Taylor, G.W., and Fergus R., CVPR 2010: Deconvolutional Networks
- Eriksson, A., and Henge, A., CVPR 2010:Efficient Computation of Robust Low-Rank Matrix Approximations in the Presence of Missing Data using the L1 Norm

We will discuss the following papers:

- Faivishevsky, L. and Goldberger, J., ICML 2010: A Nonparametric Information Theoretic Clustering Algorithm
- Masaeli, M., Fung, G. and Dy, J., ICML 2010: From Transformation-Based Dimensionality Reduction to Feature Selection

We will discuss the following paper:

- Salakhutdinov, R. and Hinton, G. E., NIPS 2009: Replicated Softmax: an Undirected Topic Model

We will discuss the following paper:

- Le Roux, N. & Fitzgibbon, A. , ICML 2010: A fast natural Newton method

- NH Salakhutdinov: Learning deep Boltzmann machines using adaptive MCMC
- NH Tang & Eliasmith: Deep networks for robust visual recognition
- NH Long & Servedio: Restricted Boltzmann Machines are Hard to Approximately Evaluate or Simulate
- PO Chen & Welling: Dynamical Products of Experts for Modeling Financial Time Series
- CS Asuncion et al Particle Filtered MCMC-MLE...
- FD Saatci et al.: Gaussian Process Change Point Models
- CW Song et al: Hilbert Space Embeddings of Hidden Markov Models
- AS Blundell et al: Bayesian Rose Trees

- NH Vickrey et al.: Non-Local Contrastive Objective
- PO Martens: Deep learning via Hessian-free optimization
- CS Blei & Gerrish A Language-based Approach to Measuring Scholarly Impact
- DR Nair & Hinton: Rectified Linear Units Improve Restricted Boltzmann Machines
- AE Elidan: Inference-less Density Estimation using Copula Bayesian Networks

We will discuss the following paper

- Emily B. Fox, Erik B. Sudderth, Michael I. Jordan and Alan S. Willsky: An HDP-HMM for Systems with State Persistence

AISTATS 2010 feedback:

- Gutmann and Hyvarinen: Noise contrastive estimation: A new estimation principle for unnormalized statistical models
- Siddiqi, Boots, Gordon: Reduced rank Hidden Markov Models
- Martens and Sutskever: Parallelizable sampling of Markov Random Fields

Plus quick reviews for:

- Poczos, Kirshner, Szepesvari: REGO -- rank-based estimation of Renyi information using Euclidean Graph Optimization
- Jaakko Riihimäki, Aki Vehtari: Gaussian processes with monotonicity information
- G. Desjardins, A. Courville, Y. Bengio, P. Vincent, O. Delalleau: Parallel Tempering for Training of Restricted Boltzmann Machines
- Chatterjee and Russell: Why are DBNs sparse?
- Marlin, Swersky, Chen, deFreitas: Inductive Principles for Restricted Boltzmann Machine Learning

We will discuss the following paper

- A. Doucet, P. Del Moral and A. Jasra, J. Royal Statist. Soc. B, vol. 68, no. 3, pp. 411-436, 2006. Sequential Monte Carlo Samplers

We will discuss the following paper

- R. P. Adams, H. M. Wallach and Z. Ghahramani, AISTATS (2010): Learning the Structure of Deep Sparse Graphical Models.

Talk by Kathrine Heller. Please see title and abstract below.

Beyond basic clustering: Bayesian statistical methods for heterogeneous data

In this talk I will focus on using Bayesian methods to model data in situations where we would like to discover latent cluster structure, but where our data is too heterogeneous to be properly modeled using standard clustering algorithms, which assume that data belong exclusively to a single cluster.

In the first half of this talk, I'll discuss the IBP Compound Dirichlet (ICD) process, a nonparametric Bayesian method which is an alternative to the Hierarchical Dirichlet Process (HDP) for mixed-membership modeling. In this work, we address the HDP assumption that the probability of a mixture component (ie cluster) contributing to a data point is positively correlated with the amount that it contributes. In many settings, this is an undesirable prior assumption. For example, in topic modelling a topic (component) might be rare throughout the corpus but dominant within those documents (data points) where it occurs. The ICD integrates features from both the HDP and the Indian buffet process (IBP).We use an ICD mixture model, the focused topic model (FTM), to analyze text corpora, and demonstrate superior performance over the HDP-based topic model.

In the second half of this talk, I'll discuss Bayesian Sets, a Bayesian method for performing information retrieval, based on queries formed from examples. Here retrieved examples can be seen as belonging to the same cluster as the examples from the query set. This retrieval process performs "clustering on demand", where a potentially unbounded

number of alternative clusterings of the data exist (through different queries). We formulate this as a Bayesian inference problem and describe a very simple and efficient algorithm for solving it. We use this Bayesian information retrieval system to perform content-based image retrieval, as well as demonstrations on multiple other types of data.

We will discuss the following papers

- M. Ranzato, A. Krizhevsky and G. Hinton, AISTATS 2010: Factored 3-Way Restricted Boltzmann Machines For Modeling Natural Images.
- R. Salakhutdinov and H. Larochelle, AISTATS 2010: Efficient Learning of Deep Boltzmann Machines.

We will discuss the following paper

- A. Asuncion, Q. Liu, A. T. Ihler, and P. Smyth, AISTATS (2010): Learning with Blocks: Composite Likelihood and Contrastive Divergence

- Yee Whye Teh and Dilan Gorur, NIPS (2009): Indian Buffet Processes with Power-law Behavior
- Cosmin Bejan, Matthew Titsworth, Andrew Hickl and Sanda Harabagiu, NIPS (2009): Nonparametric Bayesian Models for Unsupervised Event Coreference Resolution

Title: New applications for some old variational tricks

Abstract: I'll discuss two applications of variational inference. The first regards so-called local variational methods that have been been popular in recent years in Bayesian logistic regression and Bayesian image analysis (MRI). I'll show how these local variational methods are related to weakened global variational methods. The second application is to novel solution methods for Markov Decision Processes, and related issues in reinforcement learning.

Note: This session will be held in IF.4.33

- PO: Zero-shot Learning with Semantic Output Codes, Palatucci et al
- JK: Variational Inference for the Nested Chinese Restaurant Process, Chong Wang, David Blei
- JK: Sharing Features among Dynamical Systems with Beta Processes, Emily Fox, Erik Sudderth, Michael Jordan, Alan Willsky
- ASP: Kernel Methods for Deep Learning, Youngmin Cho, Lawrence Saul
- FA: Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models, Jing Gao, Feng Liang, Wei Fan, Yizhou Sun, Jiawei Han
- AJS: Construction of Nonparametric Bayesian Models from Parametric Bayes Equations, Peter Orbanz

- DR: 3D Object Recognition with Deep Belief Nets, Vinod Nair, Geoffrey Hinton
- CW: The tree-dependent components of natural scenes are edge filters, Daniel Zoran, Yair Weiss
- CW: Learning in Markov Random Fields using Tempered Transitions , Ruslan Salakhutdinov
- FD: Time-Varying Dynamic Bayesian Networks, Le Song, Mladen Kolar, Eric Xing
- AE: Learning models of object structure, Joseph Schlecht, Kobus Barnard
- AE: Semi-Supervised Learning in Gigantic Image Collections , Rob Fergus, Yair Weiss, Antonio Torralba
- CS: Particle-based Variational Inference for Continuous Systems, Alexander Ihler, Andrew Frank, Padhraic Smyth
- CS: Posterior vs Parameter Sparsity in Latent Variable Models, Joao Graca, Kuzman Ganchev, Ben Taskar, Fernando Pereira
- AD: Spatial Normalized Gamma Processes, Vinayak Rao, Yee Whye Teh
- AD: Bayesian Nonparametric Models on Decomposable Graphs, Francois Caron, Arnaud Doucet

Edit | Attach | Print version | History: r1 | Backlinks | Raw View | Raw edit | More topic actions

Topic revision: r1 - 12 Jan 2011 - 13:59:50 - AthinaSpiliopoulou

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding TWiki? Send feedback

This Wiki uses Cookies

Ideas, requests, problems regarding TWiki? Send feedback

This Wiki uses Cookies