Sergey Nikolenko

Books
Research papers
Talks and posters
Students
Popular science
Other stuff

Research
CS and crypto
Bioinformatics
Machine learning
Algebraic geometry
Algebra
Bayesian networks
Earth sciences

Teaching
2014
ML, KFU
Game Theory, HSE
Mech. Design, HSE
ML, CSClub Kazan
Game theory, HSE
Math. logic, AU
Machine learning, STC
Machine learning, AU
2013
Discrete math, HSE
Machine learning, STC
Math. logic, AU
Cryptography, AU
2012
Machine learning, STC
Math. logic, AU
Machine learning II, AU
Machine learning, AU
Machine learning, EMC
2011
Cryptography, AU
Math. logic, AU
Machine learning, AU
2010
Math. logic, AU
Machine learning, AU
Cryptography, AU
2009
Crypto in CS Club
Statistics
Machine learning, AU
Cryptography
2008
Speech recognition
MD for CS Club
ML for CS Club
Mechanism design
2007
Machine Learning
Probabilistic learning

External links
Google Scholar profile
DBLP profile
LiveJournal account
nikolenko (in Russian)

Teaching activities

Machine Learning at the Academic University

This is a year-long course in machine learning presented in 2012 at the St. Petersburg Academic University. The course aims to provide a comprehensive review of machine learning (mainly from the Bayesian perspective) in a relatively short course.

The course itself (all slides and lecture notes are in Russian):

1. Introduction. History of AI. Probability theory basics. Bayes theorem and maximal a posteriori hypotheses. Example: Laplace's rule.: Slides (.pdf, 545kb)
2. Least squares and nearest neighbors. Statistical decision theory. Linear regression. Linear regression from the Bayesian standpoint.: Slides (.pdf, 1003kb)
3. Curse of dimensionality. Example: polynomial approximation, overfitting. Regularization: ridge regression. Bias-variance-noise decomposition. How ridge regression follows from Gaussian priors.: Slides (.pdf, 1991kb)
4. Linear regression: various forms of regularization. Bayesian predictions in linear regression. Equivalent kernel. Bayesian model selection.: Slides (.pdf, 1303kb)
5. Classification. Least squares for classification. Fischer linear discriminant. Perceptron and proof of its convergence.: Slides (.pdf, 1359kb)
6. Linear discriminant analysis. Quadratic discriminant analysis. Naive Bayes. Multinomial and multivariate naive Bayes.: Slides (.pdf, 1404kb)
7. Logistic regression. Iterative reweighted least squares. Multiclass logistic regression. Probit regression. Laplace approximation. Bayesian information criterion. Bayesian logistic regression.: Slides (.pdf, 584kb)
8. Support vector machines. Linear separation and max-margin classifiers. Quadratic optimization. Kernel trick and radial basis functions.: Slides (.pdf, 963kb)
9. SVM variants: ν-SVM, one-class SVM, SVM for regression. Relevance vector machines: RVM for regression, RVM for classification.: Slides (.pdf, 969kb)
10. Clustering. Hierarchical clustering. Combinatorial methods, graph algorithms for clustering. The EM algorithm, its formal justification. EM for clustering.: Slides (.pdf, 744kb)
11. Hidden Markov models. The three problems. Dynamic programming: sum-product and max-sum. The Baum-Welch algorithm. Variations on the HMM theme.: Slides (.pdf, 557kb)
12. Model combination. Bayesian averaging. Bootstrapping and bagging. Boosting: AdaBoost. Weak learners: decision trees, learning decision trees. Exponential error minimization. RankBoost.: Slides (.pdf, 742kb)
13. Artificial neural networks. Two-layered networks, error functions. Backpropagation. Example: RankNet and LambdaRank.: Slides (.pdf, 836kb)

Selected references

Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, Information Science and Statistics series, 2006.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, 2009.
David J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003.