From Anfinsen to AlphaFold:
48 years of protein structure prediction

Despite recent progress existing methods fall far short of atomic accuracy, especially when no homologous structure is available.

The prediction of protein structures has had a long and varied development, which is extensively covered in a number of reviews.

Despite the long history of applying neural networks to structure prediction they have only recently come to improve structure prediction.

The BERT objective is trained jointly with the normal PDB structure loss on the same training examples and is not pre-trained, in contrast to recent independent work.

In the companion paper additional quantification of the reliability of pLDDT as a confidence measure is provided. d, Correlation between pTM and full chain TM-score.

The other substantial limitation that we have observed is that AlphaFold is much weaker for proteins that have few intra-chain or homotypic contacts compared to the number of heterotypic contacts (further details are provided in a companion paper).

This opens up the exciting possibility of predicting structures at the proteome-scale and beyond—in a companion paper we demonstrate the application of AlphaFold to the entire human proteome.

Despite the long history of applying neural networks to structure prediction they have only recently come to improve structure prediction.

A few recent studies have been developed to predict the 3D coordinates directly but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines.

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

AlphaFold has already demonstrated its utility to the experimental community, both for molecular replacement and for interpreting cryogenic electron microscopy maps.

AlphaFold has already demonstrated its utility to the experimental community, both for molecular replacement and for interpreting cryogenic electron microscopy maps.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

For MSA lookup at both training and prediction time, we used Uniref90 v.2020_01, BFD, Uniclust30 v.2018_08 and MGnify v.2018_12.

We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14) demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods.

The physical interaction programme heavily integrates our understanding of molecular driving forces into either thermodynamic or kinetic simulation of protein physics or statistical approximations thereof.

The AlphaFold architecture is able to train to high accuracy using only supervised learning on PDB data, but we are able to enhance accuracy (Fig. 4a) using an approach similar to noisy student self-distillation.

First, 2,423,213,294 protein sequences were collected from UniProt (Swiss-Prot&TrEMBL, 2017-11) a soil reference protein catalogue and the marine eukaryotic reference catalogue and clustered to 30% sequence identity, while enforcing a 90% alignment coverage of the shorter sequences using MMseqs2/Linclust.

For neural network construction, running and other analyses, we used TensorFlow Sonnet NumPy Python and Colab.

Per-residue Neff values were obtained by counting the number of non-gap residues in the MSA for this position and weighting the sequences using the Neff scheme with a threshold of 80% sequence identity measured on the region that is non-gap in either sequence.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

First, 2,423,213,294 protein sequences were collected from UniProt (Swiss-Prot&TrEMBL, 2017-11) a soil reference protein catalogue and the marine eukaryotic reference catalogue and clustered to 30% sequence identity, while enforcing a 90% alignment coverage of the shorter sequences using MMseqs2/Linclust.

Despite recent progress existing methods fall far short of atomic accuracy, especially when no homologous structure is available.

Despite recent progress existing methods fall far short of atomic accuracy, especially when no homologous structure is available.

The CASP assessment is carried out biennially using recently solved structures that have not been deposited in the PDB or publicly disclosed so that it is a blind test for the participating methods, and has long served as the gold-standard assessment for the accuracy of structure prediction.

In contrast to previous work this operation is applied within every block rather than once in the network, which enables the continuous communication from the evolving MSA representation to the pair representation.

In particular, we add an extra logit bias to axial attention to include the ‘missing edge’ of the triangle and we define a non-attention update operation ‘triangle multiplicative update’ that uses two edges to update the missing third edge (see Supplementary Methods 1.6.5 for details).

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

Additionally, we randomly mask out or mutate individual residues within the MSA and have a Bidirectional Encoder Representations from Transformers (BERT)-style objective to predict the masked elements of the MSA sequences.

Second, we use random masking on the input MSAs and require the network to reconstruct the masked regions from the output MSA representation using a BERT-like loss.

The prediction of protein structures has had a long and varied development, which is extensively covered in a number of reviews.

A few recent studies have been developed to predict the 3D coordinates directly but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines.

A few recent studies have been developed to predict the 3D coordinates directly but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines.

A few recent studies have been developed to predict the 3D coordinates directly but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines.

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

A related construction that uses classic geometric invariants to construct pairwise features in place of the learned 3D points has been applied to protein design.

For finding template structures at prediction time, we used a copy of the PDB downloaded on 14 May 2020, and the PDB70 clustering database downloaded on 13 May 2020.

For MSA search on BFD + Uniclust30, and template search against PDB70, we used HHBlits and HHSearch from hh-suite v.3.0-beta.3 (version 14/07/2017).

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

This bioinformatics approach has benefited greatly from the steady growth of experimental protein structures deposited in the Protein Data Bank (PDB) the explosion of genomic sequencing and the rapid development of deep learning techniques to interpret these correlations.

Our models were trained on a copy of the PDB downloaded on 28 August 2019.

First, 2,423,213,294 protein sequences were collected from UniProt (Swiss-Prot&TrEMBL, 2017-11) a soil reference protein catalogue and the marine eukaryotic reference catalogue and clustered to 30% sequence identity, while enforcing a 90% alignment coverage of the shorter sequences using MMseqs2/Linclust.

Second, we added 166,510,624 representative protein sequences from Metaclust NR (2017-05; discarding all sequences shorter than 150 residues) by aligning them against the cluster representatives using MMseqs2.

We show experimental structures from the PDB with accession numbers 6Y4F 6YJ1 6VR4 6SK0 6FES 6W6W 6T1Z and 7JTL.

Despite recent progress existing methods fall far short of atomic accuracy, especially when no homologous structure is available.

Despite the long history of applying neural networks to structure prediction they have only recently come to improve structure prediction.

In this procedure, we use a trained network to predict the structure of around 350,000 diverse sequences from Uniclust30 and make a new dataset of predicted structures filtered to a high-confidence subset.

Third, for each of the clusters, we computed an MSA using FAMSA and computed the HMMs following the Uniclust HH-suite database protocol.

For MSA lookup at both training and prediction time, we used Uniref90 v.2020_01, BFD, Uniclust30 v.2018_08 and MGnify v.2018_12.

For sequence distillation, we used Uniclust30 v.2018_08 to construct a distillation structure dataset.

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

Second, we added 166,510,624 representative protein sequences from Metaclust NR (2017-05; discarding all sequences shorter than 150 residues) by aligning them against the cluster representatives using MMseqs2.

For constrained relaxation of structures, we used OpenMM v.7.3.1 with the Amber99sb force field.

The iterative refinement using the whole network (which we term ‘recycling’ and is related to approaches in computer vision) contributes markedly to accuracy with minor extra training time (see Supplementary Methods 1.8 for details).

These approaches effectively leverage the rapid improvement in computer vision systems by treating the problem of protein structure prediction as converting an ‘image’ of evolutionary couplings to an ‘image’ of the protein distance matrix and then integrating the distance predictions into a heuristic system that produces the final 3D coordinate prediction.

Third, for each of the clusters, we computed an MSA using FAMSA and computed the HMMs following the Uniclust HH-suite database protocol.

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

For MSA lookup at both training and prediction time, we used Uniref90 v.2020_01, BFD, Uniclust30 v.2018_08 and MGnify v.2018_12.

For neural network construction, running and other analyses, we used TensorFlow Sonnet NumPy Python and Colab.

Through an enormous experimental effort the structures of around 100,000 unique proteins have been determined but this represents a small fraction of the billions of known protein sequences.

Empirically, this final relaxation does not improve the accuracy of the model as measured by the global distance test (GDT) or lDDT-Cα but does remove distracting stereochemical violations without the loss of accuracy.

The predicted structure is compared to the true structure from the PDB in terms of lDDT metric as this metric reports the domain accuracy without requiring a domain segmentation of chain structures.

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

These approaches effectively leverage the rapid improvement in computer vision systems by treating the problem of protein structure prediction as converting an ‘image’ of evolutionary couplings to an ‘image’ of the protein distance matrix and then integrating the distance predictions into a heuristic system that produces the final 3D coordinate prediction.

The prediction of protein structures has had a long and varied development, which is extensively covered in a number of reviews.

Inputs to the network are the primary sequence, sequences from evolutionarily related proteins in the form of a MSA created by standard tools including jackhmmer and HHBlits and 3D atom coordinates of a small number of homologous structures (templates) where available.

For MSA search on BFD + Uniclust30, and template search against PDB70, we used HHBlits and HHSearch from hh-suite v.3.0-beta.3 (version 14/07/2017).

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

These approaches effectively leverage the rapid improvement in computer vision systems by treating the problem of protein structure prediction as converting an ‘image’ of evolutionary couplings to an ‘image’ of the protein distance matrix and then integrating the distance predictions into a heuristic system that produces the final 3D coordinate prediction.

For MSA search on Uniref90 and clustered MGnify, we used jackhmmer from HMMER3.

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

The iterative refinement using the whole network (which we term ‘recycling’ and is related to approaches in computer vision) contributes markedly to accuracy with minor extra training time (see Supplementary Methods 1.8 for details).

Inputs to the network are the primary sequence, sequences from evolutionarily related proteins in the form of a MSA created by standard tools including jackhmmer and HHBlits and 3D atom coordinates of a small number of homologous structures (templates) where available.

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

These approaches effectively leverage the rapid improvement in computer vision systems by treating the problem of protein structure prediction as converting an ‘image’ of evolutionary couplings to an ‘image’ of the protein distance matrix and then integrating the distance predictions into a heuristic system that produces the final 3D coordinate prediction.

Predicting the three-dimensional structure that a protein will adopt based solely on its amino acid sequence—the structure prediction component of the ‘protein folding problem’ —has been an important open research problem for more than 50 years.

Exact enforcement of peptide bond geometry is only achieved in the post-prediction relaxation of the structure by gradient descent in the Amber force field.

For constrained relaxation of structures, we used OpenMM v.7.3.1 with the Amber99sb force field.

We also find that the global superposition metric template modelling score (TM-score) can be accurately estimated (Fig. 2d).

Domain accuracies in CASP are reported as GDT and the TM-score is used as a full chain global superposition metric.

Empirically, this final relaxation does not improve the accuracy of the model as measured by the global distance test (GDT) or lDDT-Cα but does remove distracting stereochemical violations without the loss of accuracy.

Domain accuracies in CASP are reported as GDT and the TM-score is used as a full chain global superposition metric.

The prediction of protein structures has had a long and varied development, which is extensively covered in a number of reviews.

Despite the long history of applying neural networks to structure prediction they have only recently come to improve structure prediction.

The CASP assessment is carried out biennially using recently solved structures that have not been deposited in the PDB or publicly disclosed so that it is a blind test for the participating methods, and has long served as the gold-standard assessment for the accuracy of structure prediction.

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

The evolutionary programme has provided an alternative in recent years, in which the constraints on protein structure are derived from bioinformatics analysis of the evolutionary history of proteins, homology to solved structures and pairwise evolutionary correlations.

The physical interaction programme heavily integrates our understanding of molecular driving forces into either thermodynamic or kinetic simulation of protein physics or statistical approximations thereof.

In parallel, the success of attention-based networks for language processing and, more recently, computer vision has inspired the exploration of attention-based methods for interpreting protein sequences.

For neural network construction, running and other analyses, we used TensorFlow Sonnet NumPy Python and Colab.

For neural network construction, running and other analyses, we used TensorFlow Sonnet NumPy Python and Colab.

For neural network construction, running and other analyses, we used TensorFlow Sonnet NumPy Python and Colab.

These timings are measured using our open-source code, and the open-source code is notably faster than the version we ran in CASP14 as we now use the XLA compiler.