A World Model of Protein Biology

ESMFold2 uses a looped transformer architecture rather than searching for evolutionarily related sequences (MSAs). It operates directly from ESMC's learned protein representations, capturing evolutionary information encoded during language model pretraining. Achieves 55% on antibody-antigen complexes and 9.4s per 1024-length protein.

Five clinically relevant targets: EGFR and PDGFRβ (receptor tyrosine kinases), PD-L1 and CTLA-4 (immune checkpoints), and CD45 (immune cell signaling regulator). An ESMFold2-designed scFv bound PD-L1 with 4.3 nM affinity.

ESM Atlas contains 6.8 billion sequences and 1.1 billion predicted structures, enabling the sequences and structures of proteins across all of life to be studied as a complete picture.

ESMC was trained on approximately 2.8 billion sequences drawn from across all of life. A scaling law links compute power used in training to how accurately representations capture biology. This powers linear returns with scale, leading to state-of-the-art protein representations.

Minibinders are compact, de novo protein scaffolds with no predetermined structure used for binder design. scFvs (single-chain variable fragment antibodies) are antibody-derived molecules using unstructured loops to bind targets. ESMFold2 can design both computationally with therapeutic-relevant affinities.

Sparse autoencoders decomposed ESMC's internal representations into more than 16,000 distinct features. The model independently recovered basic organizing principles of biology: amino acid chemistry, local structural interactions, abstract functional concepts, and evolutionary themes connecting all of life.

ESMFold2 uses a looped transformer where representations pass through the same parameters multiple times. Each pass refines the structural representation based on previous computations. This allows compute scaling at inference by running more loops without retraining.

AWS Bio Discovery, Benchling AI, Modal, Phylo, SandboxAQ, Tamarind Bio, and Tool Universe. ESMFold2, ESMC, and ESM Atlas are available at the Biohub Platform.

ESM enables computational protein binder design validated against five clinically relevant targets in oncology and immunology. When digital representations of biology become accurate enough, protein designs can be tested computationally before they reach the bench. Particularly promising for cancer and rare diseases where much of disease is individual.