TWiki> ANC Web>PIGlets (01 Feb 2017, Main.s0805516)EditAttach


The purpose of the session is for PhD students interested in machine learning to get to grips with advanced material sitting in difficulty above the taught courses here and below the discussions of cutting-edge material in the usual PIGS meetings. Anyone else who's interested in joining us for the same is welcome, but the point is to get a solid handle on the techniques rather than debate their finer points. Meetings are organised by Gavin Gray and generally held every second Monday at 10:30am in room 2.33 in the Informatics Forum. Announcements are made through the PIGS mailing list.

Remit from 2016

To solve problems of persistence in teaching material and to encourage group discussion, we're going to try some changes to the format. Previously, one person lead the discussion with the expectation that everyone in the room had read the paper and wanted to discuss it. Unfortunately, this means that one person is stuck with a lot of work to learn the material in order to teach it to others. Also, just reading a paper has limited scope for discussion and is not necessarily the best way to learn about every topic. Alternatives to this format may be useful, such as a tutorial format with exercises released the week before, or a practical session introducing the problem that motivated the topic of interest. Volunteers in charge of a topic are free to choose whichever format they think is best, or even outsource leading the discussion (if they can).

Another problem we're hoping to solve is not having enough teaching material, because it's a lot of work to prepare it. Making the sessions more flexible, and urging people who have volunteered for a topic to prepare as much material as they can, whenever they are studying the topic, should let us schedule greedily based on what's available. For example, if someone presents on variational inference, preparing a talk around a fundamental paper, but at the same time comes across some good exercises, these exercises could be greedily scheduled at a later date for a tutorial session. We're hoping to aggregate this material in a Git Book. This has the added advantage of making the material persistent; future students in ANC will have access to an index of materials for advanced topics.

Proposed topics for 2017

Volunteers to own a given topic are next to each topic:

  • Conditional Random Fields
  • Autoencoders: Gavin Gray
  • Approximate Bayesian Computation: Harri Edwards
  • Dimensionality Reduction
  • Probabilistic Programming: Svetlin Penkov
  • Sparse Regression
  • Time-series Analysis: Alina Selega
  • Gaussian Processes
  • Learning Theory
  • Reinforcement Learning: Harri Edwards
  • Bayesian Neural Networks Part 2: George Papamakarios
  • Experiment Logging: Gavin Gray

Topics Covered in 2016

So far, as of 22nd of February 2016:

  • Bayesian Neural Networks: George Papamakarios
  • Advanced MCMC: HMC, SGLD etc.: Matt Graham
  • Variational Inference: Gavin Gray
  • Bayesian Optimisation: James Wilson

Proposed topics for 2014/2015

  • Graphical Models Review/ Factor graphs/ EP
  • Dimensionality reduction
  • Probabilistic programming
  • Variational inference
  • Sparse regression
  • Sampling methods
  • Time-series analysis
  • Gaussian processes

Future meetings

Previous meetings

Gaussian Processes

  • Iain (16 March 2015): Introduction to GPs [ slides] [ book]

Variational bayesian inference

  • Gavin (02 Feb 2015) : Variational Bayes EM (Mixture of Gaussians). Section 21.6 from Murphy or 10.2 from Bishop.
  • Agamemnon (19th Jan 2015) : Variational inference introduction (Section 21.1 - 21.4 from Murphy's textbook).
  • George (02 March 2015): Variational message passing

Graphical models review / Expectation propagation

Time Series / Sequential Data

Conditional Random Fields

Bayesian Non-parametrics

Variational Inference and EP

Large Data/Online and Streaming


Boltzmann machines, MRFs etc.

Theory of kernel methods

Dynamical Models

Conditional Random Fields

Model Combination



Topic Models

Discriminative and generative methods

Sparse ICA

  1. Ali (September 14th): Independent Components Analysis: Algorithms and Applications, Aapo Hyvärinen and Erkki Oja (2000)
  2. Konrad (September 28th): Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1?,, Olshausen BA, Field DJ (1997)
  3. Peter (October 12th): An Introduction to Compressive Sampling, Emmanuel J. Candes and Michael B. Wakin (2008)

Markov Chain Monte Carlo methods

Probabilistic inference using Markov chain Monte Carlo methods, Radford M. Neal (1993)

  1. Jakub (July 6th): Pages 1-59
  2. Ali (July 20th): Pages 59-87
  3. Ali (August 3rd): Pages 59-87
  4. Nicolas (August 17th): Pages 87-121

Slice Sampling , Radford M. Neal (2003)

  1. Iain (August 31st): Pages 710-720


Visit PIGlets 2009 for a list of meetings held in 2009.

Topic revision: r173 - 01 Feb 2017 - 11:38:36 - Main.s0805516
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies