Sergey Nikolenko
Main pageBooks';
print '
Research papers';
print '
Talks and posters';
print '
Students';
print '
Popular science';
print '
Other stuff';
print '
Research';
print '
CS and crypto';
print '
Bioinformatics';
print '
Machine learning';
print '
Algebraic geometry';
print '
Algebra';
print '
Bayesian networks';
print '
Earth sciences';
print '
Teaching';
print '
2014';
print '
ML, KFU';
print '
Game Theory, HSE';
print '
Mech. Design, HSE';
print '
ML, CSClub Kazan';
print '
Game theory, HSE';
print '
Math. logic, AU';
print '
Machine learning, STC';
print '
Machine learning, AU';
print '
2013';
print '
Discrete math, HSE';
print '
Machine learning, STC';
print '
Math. logic, AU';
print '
Cryptography, AU';
print '
2012';
print '
Machine learning, STC';
print '
Math. logic, AU';
print '
Machine learning II, AU';
print '
Machine learning, AU';
print '
Machine learning, EMC';
print '
2011';
print '
Cryptography, AU';
print '
Math. logic, AU';
print '
Machine learning, AU';
print '
2010';
print '
Math. logic, AU';
print '
Machine learning, AU';
print '
Cryptography, AU';
print '
2009';
print '
Crypto in CS Club';
print '
Statistics';
print '
Machine learning, AU';
print '
Cryptography';
print '
2008';
print '
Speech recognition';
print '
MD for CS Club';
print '
ML for CS Club';
print '
Mechanism design';
print '
2007';
print '
Machine Learning';
print '
Probabilistic learning';
print '
External links';
print '
Google Scholar profile';
print '
DBLP profile';
print '
LiveJournal account
nikolenko (in Russian)
Teaching activities
Machine Learning for CS Club
This course unites «Machine Learning» and «Probabilistic learning» and aims to provide a comprehensive review of machine learning (mainly from the Bayesian perspective) in a relatively short course.
This course is presented in the «Computer Science Club» in the Steklov Mathematical Institute supported by Anton Likhodedov and run by Edward A. Hirsch and Alexander Kulikov.
The course itself (all slides and lecture notes are in Russian):
- 1. Introduction. History of AI. Probability theory basics. Bayes theorem and maximal a posteriori hypotheses. A review of useful distributions.
- Slides ()
- 2. Decision trees. ID3. Bayesian analysis of classification problems. Complexity measures based on decision trees and bounds on decision tree complexity.
- Slides (.pdf, 1056kb)
- 3. Artificial Neural Networks. Perceptrons; learning a linear perceptron. Non-linear perceptrons and gradient descent. ANNs and the backpropagation algorithm. Modifications: momenta, regularizers.
- Slides (.pdf, 999kb)
- 4. Sampling. The sampling problem, why it is hard. Importance sampling and rejection sampling. MCMC (Monte-Carlo Markov Chain) methods. The Metropolis-Hastings method, Markov chains, slice sampling. Hamiltonian mechanics and Hamiltonian MCMC.
- Slides (.pdf, 652kb)
- 5. Genetic algorithms. Crossover on binary strings and trees. Genetic programming. Baldwin effect. Ant Colony optimization. Simulated annealing.
- Slides (.pdf, 1384kb)
- 6. Bayesian classifiers: optimal, Gibbs, and naive. Concept learning: Find-S and Candidate Elimination algorithms.
- Slides (.pdf, 622kb)
- 7. Bayesian Belief Networks. Definitions, d-separability, decomposition. Propagation in polytree BBNs. Propagation in BBNs with undirected cycles: join and junction trees, triangulation, message passing algorithm.
- 8. Brute force marginalization. Conjugate priors. Some coding theory: error-correcting codes, codes and their trellises. Decoding as Bayesian inference. Sum-product and min-sum algorithms.
- Slides (.pdf, 947kb)
- 9. Marginalization on a graph. Sum-product and min-sum in the general case. Approximate marginalization: Laplace method. Marginalization with Kikuchi approximation methods.
- Slides (.pdf, 702kb)
- 10. Hidden Markov models. The model, three inference tasks. The Viterbi, sum-product, and Baum-Welch algorithms.
- Slides (.pdf, 720kb)
- 11. Associative memory. Hebbian learning. Bidirectional associative memory. Hopfield networks and a proof of their convergence.
- Slides (.pdf, 461kb)