Generative Models at HUST, 2023

This is a short course on modern generative models presented at the  Center of Mathematical Sciences of the  Huazhong University of Science and Technology in November 2023.

Lecture materials

1.
Attention mechanisms in deep learning. The Transformer architecture.
2.
BERT and GPT families. The variational autoencoder (VAE). Discrete latent spaces in VAE: VQ-VAE and VQ-GAN.
3.
Transformer + dVAE = DALL-E. Transformers for images: Visual BERT, ViT, Swin, Perceiver. Multimodal latent spaces: CLIP and BLIP. Our recent work: LAPCA for cross-lingual retrieval and question answering.
4.
Transformers for medical imaging: classification, segmentation, and image synthesis.
5.
Transformers for video processing and retrieval.
6.
Our recent work on math and Transformers: Sinkhorn transformations for postprocessing in video retrieval and topological data analysis for AI-generated text detection and model interpretation.