The three-dimensional arrangement of atoms in a protein, determined by the amino acid sequence, which determines the protein's biological function.
Protein Structure
How does ESMFold2 differ from other structure prediction models?
has accuracy metric
Protein structure prediction models that predict 3D protein structures from amino acid sequences
Structure Prediction Model
ESMFold2 architecture that passes representations through the same parameters multiple times, optimized through this loop during training. Enables compute scaling at inference time by running more loops.
Looped Transformer Architecture
Evaluate Predicted Binders
Review computational predictions for binding affinity and selectivity. For scFvs, expect success rates around 21% with higher compute. Select top candidates for experimental validation.
Partner platform for ESM access
AWS Bio Discovery
The protein language model component of ESM, trained on approximately 2.8 billion sequences from across all of life, providing representations that capture fundamental principles of protein structure and function.
ESMC (ESM Cambrian)
ESM Protein Biology Glossary
What are minibinders and scFvs in protein design?
USD
Comprehensive maps of protein sequences and predicted structures across life
Sequence Atlas
What therapeutic targets has ESMFold2 been validated against?
Deep dive into the technical architecture of ESMC and ESMFold2 including transformer language models, looped transformer architecture, and scaling laws
Technical Details of ESM Architecture
ESMFold2 has been validated against five clinically relevant targets: EGFR and PDGFRβ (receptor tyrosine kinases in tumor growth), PD-L1 and CTLA-4 (immune checkpoints cancer cells exploit), and CD45 (immune cell signaling regulator). An ESMFold2-designed scFv bound PD-L1 with 4.3 nM affinity.
Protein Structure Prediction
State-of-the-art protein structure prediction model using looped transformer architecture that predicts protein structure and powers the design of new proteins
ESMFold2
55% antibody-antigen complexes; 71% protein-protein interactions (single sequence); 77% with alignment data
Not applicable (inference-time compute scaling via loop iterations)
Looped Transformer
Sparse autoencoders decomposed ESMC's internal representations into more than 16,000 distinct features. The model independently recovered basic organizing principles of biology: amino acid chemistry, local structural interactions, abstract functional concepts recurring across unrelated proteins, and evolutionary themes connecting all of life.
CRISPR-Cas Discovery
In SAE feature clusters derived from ESM Atlas, a cluster of RNA-guided DNA endonucleases - functional class including CRISPR-Cas systems - brings together eukaryotic Fanzor proteins and their evolutionary ancestor prokaryotic TnpB, despite high evolutionary divergence and low sequence similarity.
Cytotoxic T-Lymphocyte Antigen 4 - an immune checkpoint that cancer cells exploit to evade immune surveillance, one of five clinically relevant therapeutic targets validated with ESMFold2
CTLA-4
The two primary secondary structure arrangements that form when a protein backbone folds into its most stable conformations.
Alpha Helix and Beta Sheet
Iterate and Optimize
Use experimental feedback to refine design parameters. ESM enables rapid iteration - designs can be computationally validated before bench experiments, accelerating the design cycle for therapeutic protein development.
What is the scale of ESM Atlas?
Single-chain variable fragment antibodies - antibody-derived molecules using unstructured loops to bind targets. Success rates nearly doubled from 12% to 21% across five targets.
scFv (Single-chain Variable Fragment)
uses architecture
Programmed Death-Ligand 1 - an immune checkpoint that cancer cells exploit to evade immune surveillance, one of five clinically relevant therapeutic targets validated with ESMFold2. An ESMFold2-designed scFv bound PD-L1 with 4.3 nM affinity.
PD-L1
Partner providing TransformerEngine and cuEquivariance kernels for training ESMC and ESMFold2
NVIDIA
Personalized Medicine Potential
While disease follows common patterns, much of disease is individual. For cancer and rare diseases, potential is most immediate. ESM can design lab-validated protein binders for five clinically relevant targets in days.
Benchling AI
Validate in Laboratory
Test designed binders experimentally using cell-based assays to measure affinity and functional activity. Verify structure using cryo-electron microscopy if crystal structure confirmation is needed.
Cryo-EM Verification
Using cryo-electron microscopy, researchers verified structure of ESMFold2-designed minibinder bound to its target. High concordance between computational and observed models demonstrated 1.204 Å RMSD, exemplifying ESMFold2's design capabilities.
Feature Decomposition with SAEs
Sparse autoencoders decomposed ESMC's internal representations into more than 16,000 distinct features. The model, trained only to predict amino acid sequences with no prior biological knowledge, independently recovered many basic organizing principles of biology.
Epidermal Growth Factor Receptor - a receptor tyrosine kinase implicated in tumor growth, one of five clinically relevant therapeutic targets validated with ESMFold2
EGFR
Phylo
Primary secondary structure arrangement that forms when a protein backbone folds into stable conformations. Features representing alpha helices were discovered in ESMC's latent space.
Alpha Helix
What partner platforms provide access to ESM?
What did sparse autoencoders reveal about ESMC's latent space?
ESMC was trained on approximately 2.8 billion sequences drawn from across all of life. Researchers identified a scaling law linking compute power used in training to how accurately representations capture biology. In ESMC, this powers linear returns with scale, leading to state-of-the-art protein representations.
Single-chain variable fragment antibodies - antibody-derived molecules that use unstructured loops to bind their targets, particularly demanding examples of the binder design problem.
ESM Atlas contains 6.8 billion sequences and 1.1 billion predicted structures, enabling the sequences and structures of proteins across all of life to be studied as a complete picture.
Future of Biology
Future of biology will depend on systems combining scaled data, predictive models, and experimental feedback. This world model is an important step toward that future: an open foundation for understanding proteins, discovering new biology, and designing molecular tools that can help prevent, treat, or cure disease.
Future implications of ESM for medicine, drug discovery, and understanding biology
What This Makes Possible
Protein Sequence Atlas
A map of 6.8 billion sequences and 1.1 billion predicted structures, enabling the sequences and structures of proteins across all of life to be studied as a complete picture
ESM Atlas
6.8 billion sequences; 1.1 billion predicted structures
Compact, de novo protein scaffolds with no predetermined structure used for binder design. Success rates averaged 54% to 70% at higher compute across five targets.
Minibinder
Compact, de novo protein scaffolds with no predetermined structure used for designing protein binders computationally.
ESMFold2 uses a looped transformer where representations pass through the same parameters multiple times. The model learns to use each pass to refine its structural representation based on previous computations. This allows compute scaling at inference by running more loops without retraining, allocating more capacity to harder predictions.
All ESM models released under MIT license
MIT License
Modal
ESM enables computational protein binder design validated against five clinically relevant targets in oncology and immunology. When digital representations of biology become accurate enough, protein designs can be tested computationally before they reach the bench. This is particularly promising for cancer and rare diseases where much of disease is individual.
SandboxAQ
How was ESMC trained and what scaling laws were discovered?
Prokaryotic evolutionary ancestor of eukaryotic Fanzor proteins, appearing in the same SAE feature cluster in ESM Atlas despite high evolutionary divergence and low sequence similarity
TnpB
ESM is a world model of protein biology comprising three artifacts: ESMFold2 (structure prediction), ESM Atlas (map of 6.8B sequences/1.1B structures), and ESMC (protein language model trained on ~2.8B sequences). It learns from protein sequences produced by evolution to represent, map, predict, and design proteins.
Minibinders are compact, de novo protein scaffolds with no predetermined structure used for binder design. scFvs (single-chain variable fragment antibodies) are antibody-derived molecules that use unstructured loops to bind targets. ESMFold2 can design both formats computationally with therapeutic-relevant affinities.
Nucleophilic Elbow Example
Example: nucleophilic elbow, a catalytic motif that evolved independently many times, appearing in enzymes sharing no common ancestor and adopting entirely different folds. Features representing fundamental concepts in proteins combine to create complex functions.
Platelet-Derived Growth Factor Receptor beta - a receptor tyrosine kinase implicated in tumor growth, one of five clinically relevant therapeutic targets validated with ESMFold2
PDGFRβ
A suite of structure prediction benchmarks used to evaluate ESMFold2's state-of-the-art accuracy
Foldbench
What is ESM (Evolutionary Scale Models)?
binder design algorithm
has training scale
has validation method
What is the significance of ESM for medicine?
Five Validated Targets
Five targets central to oncology and immunology: EGFR and PDGFRβ (receptor tyrosine kinases implicated in tumor growth); PD-L1 and CTLA-4 (immune checkpoints that cancer cells exploit to evade immune surveillance); CD45 (regulator of immune cell signaling).
Chains of 20 chemical building blocks (amino acids) whose order determines how the chain folds into a three-dimensional structure, and folding determines function.
Protein Sequence
Feature Categories Discovered
Features discovered: specific amino acids; classes of amino acids (aromatics, small hydrophobics); context-dependent configurations; alpha helices and beta sheets; abstract concepts like cellular localization signals, post-translational modifications, local biochemical and electrostatic environments, functional classes independent of fold.
Primary secondary structure arrangement that forms when a protein backbone folds into stable conformations. Features representing beta sheets were discovered in ESMC's latent space.
Beta Sheet
A technique developed to identify interpretable structure in large language models by decomposing internal representations into distinct features.
Sparse Autoencoders (SAE)
How ESM Atlas reveals evolutionary relationships and functional connections across all of life
ESM Atlas: Map of Known and Unknown Biology
Select Target Molecule
Choose a clinically relevant molecular target such as EGFR, PDGFRβ, PD-L1, CTLA-4, or CD45. Define the binding site and desired affinity characteristics for therapeutic activity.
Results of ESMFold2 validation against five clinically relevant targets in oncology and immunology
Therapeutic Protein Design Results
ESM is available through partner platforms including AWS Bio Discovery, Benchling AI, Modal, Phylo, SandboxAQ, Tamarind Bio, and Tool Universe. ESMFold2, ESMC, and ESM Atlas are available at the Biohub Platform.
Tamarind Bio
Cryo-electron microscopy used to verify the structure of ESMFold2-designed minibinders bound to targets. High concordance between computational and observed models (1.204 Å RMSD) demonstrated.
Cryo-EM
How sparse autoencoders revealed the structure of concepts learned by ESMC without any prior biological knowledge
Organization of Concepts in ESMC's Latent Space
ESMFold2 uses a looped transformer architecture rather than searching for evolutionarily related sequences (MSAs). It operates directly from ESMC's learned protein representations, capturing evolutionary information encoded during language model pretraining. It achieves 55% on antibody-antigen complexes and can predict a 1024-length protein structure in 9.4 seconds.
Step-by-step guide to computationally designing protein binders using ESMFold2 for clinically relevant molecular targets
How to Design Protein Binders with ESMFold2
Fast protein structure prediction variant that predicts a 1024-length protein structure in 9.4 seconds while outperforming existing models on antibody-antigen folding
ESMFold2-Fast
9.4 seconds per 1024-length protein
Looped Transformer (Optimized)
Protein Language Modeling
State-of-the-art protein language model trained on approximately 2.8 billion sequences drawn from across all of life, providing a foundation for modeling sequence, structure, and function of proteins
~2.8 billion sequences
Transformer
State-of-the-art protein structure prediction model using looped transformer architecture that predicts protein structure from amino acid sequences without requiring multiple sequence alignments.
A catalytic motif that has evolved independently many times, appearing in enzymes that share no common ancestor and adopt entirely different folds. Represented as an abstract feature in ESMC's latent space.
Nucleophilic Elbow
How does ESMFold2 achieve compute-time scaling?
Choose Binder Format
Select between minibinders (compact de novo scaffolds with no predetermined structure) or scFvs (single-chain variable fragment antibodies) depending on therapeutic requirements and target characteristics.
Scientific Research Software
A world model of protein biology comprising ESMFold2, ESM Atlas, and ESMC for protein structure prediction, sequence mapping, and language modeling
ESM (Evolutionary Scale Models)
Cloud Platform
Define Design Constraints
Establish parameters for the binder design including required affinity (nanomolar potency), specificity requirements, and structural constraints for the target binding interface.
Tool Universe
Scaling Laws and Representation Quality
ESMC identified scaling law linking compute power used in training to degree representations accurately capture biology. Linear returns with scale lead to state-of-the-art protein representations.
Cluster of Differentiation 45 - a regulator of immune cell signaling, one of five clinically relevant therapeutic targets validated with ESMFold2
CD45
Eukaryotic RNA-guided DNA endonuclease proteins that cluster with prokaryotic TnpB in ESM Atlas SAE features, despite high evolutionary divergence and low sequence similarity
Fanzor
Naturally occurring gene-editing systems that bacteria evolved to defend themselves against viruses. Featured in ESM Atlas SAE feature clusters, bringing together Fanzor and TnpB proteins despite high evolutionary divergence.
CRISPR-Cas Systems
AI agent skill for generating knowledge graphs from web content
kg-generator skill
Transformer Language Model Evolution
ESM program developed the first transformer language model of protein sequences in 2019. Core scientific hypothesis: training a language model across sequences of all life causes it to internalize fundamental properties governing protein biology - rules underlying protein folding, interaction, and function.
Run ESMFold2 Design Algorithm
Use ESMFold2's design algorithm that searches through the joint model of sequence and structure for proteins predicted to bind the target. Allocate compute resources based on desired success rates (higher compute yields up to 70% minibinder success).
Scale and Coverage
ESM Atlas is a map of 6.8 billion sequences and 1.1 billion predicted structures. The atlas enables sequences and structures of proteins across all of life to be studied as a complete picture.
Computational Scale Advantages
Computing structures for 1.1 billion proteins in a week, or screening millions of candidates in hours, isn't just faster science. Ability to run experiments virtually at scale not possible in laboratory starts to unlock new possibilities in biology.
Evolutionary Scale Models - A world model of protein biology comprising ESMFold2, ESM Atlas, and ESMC that learns from protein sequences produced by evolution to represent, map, predict, and design proteins.
Biohub releases ESM (Evolutionary Scale Models) - a world model of protein biology comprising ESMFold2 (structure prediction), ESM Atlas (map of 6.8B sequences/1.1B structures), and ESMC (protein language model trained on ~2.8B sequences).
A World Model of Protein Biology
Design Success Rates
Averaged across targets, minibinder success rates rose from 54% to 70% at higher compute. For scFvs, success rates nearly doubled from 12% to 21%. ESMFold2-designed de novo scFv bound PD-L1 with measured affinity of 4.3 nM.
A map of 6.8 billion protein sequences and 1.1 billion predicted structures, enabling comprehensive study of proteins across all of life.
Frequently Asked Questions about ESM
ESMFold2 uses looped transformer where representations pass through same parameters multiple times, optimized through this loop during training. Each pass refines structural representation based on previous computation.
Technique developed to identify interpretable structure in large language models, used to decompose ESMC's internal representations into more than 16,000 distinct features
A scientific research organization developing world models of protein biology
Biohub
Base class for protein models developed at Biohub
Protein Model
Language models trained on protein sequences that learn evolutionary patterns and biological properties
Protein Language Model
Preprint describing full results of ESM research program, to be posted on bioRxiv
ESM Research Preprint