Lecture 6

Learned Representations

Unsupervised learning, autoencoders and VAEs, self-supervised pre-training and transfer learning — what happens when the model, not you, chooses the features.

Jules Schleinitz
90 min
Back to schedule
Slides (PDF) Recording (YouTube) Companion Tutorial



Gómez-Bombarelli et al. — “Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules” Paper
ACS Cent. Sci., 2018, 4, 268 · The chemical VAE: 250k ZINC molecules, a 196-D latent space, and the joint property head
Krenn et al. — “SELFIES: A 100% robust molecular string representation” Paper
Mach. Learn.: Sci. Technol., 2020, 1, 045024 · A grammar that cannot emit an invalid molecule
Skinnider — “Invalid SMILES are beneficial rather than detrimental to chemical language models” Paper
Nature Machine Intelligence, 2024, 6, 437 · Breaking SELFIES on purpose improves distribution match
Hu et al. — “Strategies for Pre-training Graph Neural Networks” Paper
ICLR, 2020 · Node-level plus graph-level pre-training (+7.2 ROC-AUC) — and explicit negative transfer
Ross et al. — “Large-scale chemical language representations capture molecular structure and properties” Paper
Nature Machine Intelligence, 2022, 4, 1256 · MoLFormer-XL: 1.1 B SMILES on 16 GPUs
Wang et al. — “MolCLR: Molecular Contrastive Learning of Representations via Graph Neural Networks” Paper
Nature Machine Intelligence, 2022, 4, 279 · Contrastive pre-training and its three graph augmentations
Schwaller et al. — “Mapping the space of chemical reactions using attention-based neural networks” Paper
Nature Machine Intelligence, 2021, 3, 144 · Self-supervised pre-training alone recovers 82% of 792 reaction classes
Han & Choi — “Transfer Learning from Simulation to Experimental Data: NMR Chemical Shift Predictions” Paper
J. Phys. Chem. Lett., 2021, 12, 3662 · The learning-curve crossover, measured
Deng et al. — “A systematic study of key elements underlying molecular property prediction” Paper
Nature Communications, 2023, 14, 6395 · 62,820 models; fixed descriptors win below ~10³–10⁴ labels
Praski, Adamczyk & Czech — “Benchmarking Pretrained Molecular Embedding Models” Paper
arXiv:2508.06199, 2025 · 25 models × 25 datasets: nearly all show no improvement over ECFP
van Tilborg, Alenicheva & Grisoni — “Exposing the Limitations of Molecular Machine Learning with Activity Cliffs” Paper
J. Chem. Inf. Model., 2022, 62, 5938 · Where every smooth representation breaks
Wattenberg, Viégas & Johnson — “How to Use t-SNE Effectively” Website
Distill, 2016 · Why gap widths, cluster sizes and clusters-in-noise will mislead you