Sergey Nikolenko

Books
Research papers
Talks and posters
Students
Popular science
Other stuff

Research
CS and crypto
Bioinformatics
Machine learning
Algebraic geometry
Algebra
Bayesian networks
Earth sciences

Teaching
2014
ML, KFU
Game Theory, HSE
Mech. Design, HSE
ML, CSClub Kazan
Game theory, HSE
Math. logic, AU
Machine learning, STC
Machine learning, AU
2013
Discrete math, HSE
Machine learning, STC
Math. logic, AU
Cryptography, AU
2012
Machine learning, STC
Math. logic, AU
Machine learning II, AU
Machine learning, AU
Machine learning, EMC
2011
Cryptography, AU
Math. logic, AU
Machine learning, AU
2010
Math. logic, AU
Machine learning, AU
Cryptography, AU
2009
Crypto in CS Club
Statistics
Machine learning, AU
Cryptography
2008
Speech recognition
MD for CS Club
ML for CS Club
Mechanism design
2007
Machine Learning
Probabilistic learning

External links
Google Scholar profile
DBLP profile
LiveJournal account
nikolenko (in Russian)

Teaching activities

Machine Learning at the Academic University

This is a one-semester course in machine learning presented in the spring of 2011 in the St. Petersburg Academic University. The course aims to provide a comprehensive review of machine learning (mainly from the Bayesian perspective) in a relatively short course.

The course itself (all slides and lecture notes are in Russian):

1. Introduction. History of AI. Probability theory basics. Bayes theorem and maximal a posteriori hypotheses. A review of useful distributions.: Slides (.pdf, 957kb)
2. Artificial Neural Networks. Perceptrons; learning a linear perceptron. Non-linear perceptrons, sigmoid functions, gradient descent. ANNs and the backpropagation algorithm. Modifications: momenta, regularizers.: Slides (.pdf, 728kb)
3. Bayesian classifiers. The classification problem. Optimal classifier, Gibbs classifier. Naive Bayes approach. Two naive models: multivariate and multinomial naive Bayes.: Slides (.pdf, 342kb)
4. Support vector machines. Separation with linear hyperplanes, margin maximization. The kernel trick and nonlinear SVMs. SVM modifications for outlier search.: Slides (.pdf, 837kb)
5. Clustering. Hierarchical clustering. The EM algorithm. Formal EM justification. EM for clustering. k-means algorithm.: Slides (.pdf, 598kb)
6. Hidden Markov models. The three problems. The Baum-Welch algorithm and its justification. Continuous observables, time spent at states.: Slides (.pdf, 312kb)
7. Priors. Conjugate priors. Conjugate priors for Bernoulli trials. Conjugate priors for the normal distribution: learning the mean for fixed variance, learning the variance for fixed mean.: Slides (.pdf, 464kb)
8. Conjugate priors for the normal distribution: learning the mean and variance simultaneously.: Slides (.pdf, 300kb)
9. Bayesian decoding. MAP codeword decoding problem. Linear codes and dynamic programming Bayesian decoding algorithm.: Slides (.pdf, 269kb)
10. Marginalization on the graph of functions and variables. Min-product and max-sum. The general message passing algorithm.: Slides (.pdf, 1393kb)
11. Markov random fields. Moralization and triangulation. Perfect elimination orderings. Join trees and junction trees. Inference in general BBNs (with undirected cycles).: Slides (.pdf, 1248kb)
12. Approximate Bayesian inference. Variational approximations. QMR-DT and its variational inference algorithm.: Slides (.pdf, 1287kb)
13. Boltzmann machines. Mean field theory. Approximate inference and learning in Boltzmann machines.: Slides (.pdf, 574kb)
14. Sigmoid belief networks. Approximate inference in sigmoid belief networks. The LDA model, LDA inference and learning.: Slides (.pdf, 478kb)
15. Hebbian learning, bidirectional associative memory, and Hopfield networks.: Slides (.pdf, 441kb)
16. Bayesian rating models. The Elo rating. Bradley-Terry models and minorization-maximization learning algorithms. The TrueSkill model and its modifications.: Slides (.pdf, 2503kb)

Selected references

David J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003.
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, Information Science and Statistics series, 2006.
A.L. Tulupyev, S.I. Nikolenko, A.V. Sirotkin. Bayesian Networks: A Probabilistic Logic Approach. St.-Petersburg, Nauka, 2006. (two first pages of the book: .pdf, 815kb, in Russian, ozon.ru)