Unsupervised learning, autoencoders and VAEs, self-supervised pre-training and transfer learning — what happens when the model, not you, chooses the features.
Labels are the bottleneck, not molecules. We can enumerate 1011 molecules and have measured 103. Unsupervised and self-supervised learning exist to spend the first to make up for the second.
Unsupervised methods find real structure — but a 2-D map is a picture of your fingerprint. Word2vec on 3.3 M abstracts recovers the periodic table; masked pre-training recovers 82% of 792 reaction classes with no labels. Equally, t-SNE produces convincing clusters from pure noise.
The VAE's decoder disappointed; its encoder changed the field. As generators, molecular VAEs lose to a 30-second graph genetic algorithm and, on some benchmarks, to random screening. The pretext-task-then-keep-the-encoder pattern is what survived.
Validity was the easy metric. JT-VAE and SELFIES reach 100% validity by construction, not by learning — and the model with perfect validity often matches real chemistry worst.
Pre-training is a prior. Worth ~3× at a hundred labels (NMR transfer: 0.43 vs 1.35 ppm), worth nothing by a thousand, and occasionally worth less than nothing — pre-training a GAT cost 6.5 points of ROC-AUC.
Always run ECFP + random forest, and always split by scaffold. In the largest independent replication (25 models × 25 datasets, 2025), almost nothing beat that baseline significantly.