(Not) getting misled by crystal structures part 6: low ligand occupancies

It’s been a while since our last “getting misled by crystal structures” post. That one described unrecognized conformational heterogeneity of ligands. A more basic issue is ligand occupancy. It’s normally assumed that every protein in the crystal lattice has a bound ligand. A new paper in Structure by Timothy Stachowski and Marcus Fischer at St. Jude Children’s Research Hospital reveals that this is not always the case.

 

The researchers selected roughly 10,000 protein-ligand structures from the Protein Data Bank (PDB) and did a simple re-refinement of the ligand occupancies and B-factors (measures of conformational heterogeneity and modeling errors). 10% of the structures already presumed ligand occupancies at or below 0.9, but re-refinement saw the fraction jump to 35%, more than three-fold higher. There were no overall differences between covalent and non-covalent ligands, but 37% of fragments (defined as having MW <300 Da) saw decreased occupancy in re-refinement compared to just 22% of larger ligands. A few structures even saw occupancy drop completely. The authors wrote that “manual reviewing these revealed that ligands were built into spurious electron-density.”

 

Crystallographers use several metrics to assess the quality of their structures. In addition to B-factors, unique to each residue or atom, real-space correlation coefficients (RSCC) and real-space R values are commonly used, and the researchers compared these parameters before and after re-refinement. In many cases the metrics improved with decreased occupancy, but not for all metrics, and not always meaningfully. This means that standard assessments do not always flag partially occupied ligands.

 

OK, so a ligand binds at only 80% occupancy rather than the 100% assumed: does this matter? The researchers describe three categories where the answer may be yes. In the first, correcting ligand occupancy reveals alternative conformations of protein side chains, which could be informative for understanding the mechanism of binding. In the second, correcting ligand occupancy can reveal water molecules that interact with the ligand and/or protein. As we noted more than a decade ago, water is an essential player in protein-ligand interactions, and a single water molecule can make the difference between a binder and a non-binder. Finally, correcting ligand occupancy can reveal alternative binding modes for the ligand, and even ligands binding to other sites.

 

Importantly, this analysis was done on individually refined structures in the PDB, and it seems likely that the issues would be even more severe for structures batch-refined in high-throughput crystallographic fragment screens. As we wrote last year, the community needs to figure out how to deal with the increasing number of these structures.

 

The fact that roughly a third of PDB structures have less than 100% ligand occupancy has implications for training AI models. It also has implications for individual targets. As the researchers note, “non-crystallographers who rely on the PDB often assume that deposition itself is an implicit stamp of approval. Structural biologists, however, know that this is not always the case.” Before using a structure, it would be wise to re-refine it yourself.

Hundreds of crystallographic ligands for FABP4 – many not as expected

The ten human fatty-acid binding proteins (FABPs) shuttle lipids around cells. As we noted several years ago, FABP4 and FABP5 are potential drug targets for diabetes and atherosclerosis, but selectivity over FABP3 is needed to avoid cardiotoxicity. Markus Rudolph and colleagues at Hoffmann-La Roche describe progress towards selective molecules in three consecutive open-access Acta. Cryst. D papers. Perhaps more importantly, they gift a massive high quality data set to the scientific community – along with some important caveats about data for protein-ligand structures.

 

The first paper focuses on purification and NMR characterization of FABP4. Recombinant FABPs are normally expressed in E. coli, and they always contain natural fatty acids that copurify with the protein. This can complicate ligand binding studies, since the endogenous fatty acids act as competitors. Indeed, the researchers highlight two structures in the protein data bank (PDB) whose supposed ligands are probably fatty acids.

 

To solve this problem, the researchers denature FABP4, separate the fatty acid, and then refold the protein. This truly apo form of the protein was studied by NMR, revealing that the protein becomes more rigid upon ligand-binding.

 

The second paper is of more general interest. It reports a set of 229 crystal structures of various FABPs, of which 216 have a bound ligand. Of these, 75 have associated IC₅₀ values for at least one FABP, and 50 compounds have IC₅₀ values reported for FABP3, FABP4, and FABP5. Importantly, the structures are solved to high resolution, with a median of 1.12 Å. Two crystal forms are particularly suitable for soaking, and compounds were typically soaked at 60 mM in 30% DMSO overnight.

 

All the crystal structures are deposited in the PDB, and all the binding data are provided in the supporting information. Given FABPs’ predilection for carboxylic acids, the ligands contain a variety of carboxylic acid mimetics. This wealth of high-quality data should be valuable for constructing machine-learning binding models, and the researchers conclude by calling “on other industrial organizations to also make their legacy data available such that prediction models with broader applicability may be developed more quickly.”

 

But it was the third paper that really caught my attention: the researchers summarized it as “what is written on the bottle is not what is in the crystal.” In fact, of the 216 ligands reported, a whopping 33 (15%) do not match the compound registered. These are grouped into several categories and described in detail.

 

Human error is the simplest to explain: the researchers show an example where a 1,2-benzoxazole was registered as a 1,3-benzoxazole. Because the molecules have the same molecular weight, mass spectrometry could not distinguish them. Similarly, the researchers find several cases where the wrong enantiomer or diastereomer was registered. In another case, a racemic mixture led to a single enantiomer bound to FABP4, with the protein acting as a “chiral sponge.”

 

Other cases are more unusual, and include ring closing, ring opening, acyl shifts, hydrolysis, and instances of ligand decomposition or incomplete reactions. The researchers note that small amounts of impurities could be particularly problematic at the high ligand concentrations used for soaking; they calculate that just 0.06% impurity would be equivalent to the total amount of FABP in a crystal. Some fragment screens are done at even higher concentrations, further increasing the risk of enriching impurities.

 

A 15% rate of unexpected ligands is comparable to the numbers we blogged about here, but those were commercial libraries, whereas this set is from Roche, which likely has better internal quality control. One factor that led to the recognition of the problem is the high resolution, where a single atom change could be readily seen. Another is the buried nature of the ligands; ligands bound on the surface of a protein may have more dynamically disordered bits, which would be difficult to distinguish from missing moieties caused by decomposition.

 

Indeed, the researchers examine two other proteins, PDE10 and ATX, for which they have also released ~200 ligand-bound structures but at lower average resolutions. There are some unexpected ligands for these proteins too, but many fewer than for the FABPs – or perhaps we just can’t observe some of them.

 

As we noted back in 2014, up to a quarter of ligand-containing crystal structures in the PDB may contain serious errors, and the researchers cite a study suggesting that 12% are “just bad.” These could have obvious negative consequences for training computational models, and the researchers call on the community to set standards to create a rigorously chosen training set. Perhaps this discussion could be held in parallel with the discussion on how to house fragment screening data, which we wrote about last month.

How to avoid metal artifacts

Back in 2017 we observed with characteristic subtlety that “heavy metals suck.” That post described a hit-finding campaign which foundered when the apparent activity of the fragments turned out to be due to contaminating zinc. A new paper in J. Med. Chem. by Thomas Gerstberger, Peter Ettmayer, and colleagues at Boehringer Ingelheim (BI) describes a similar story, along with suggestions of how to avoid being misled.

 

BI had a collaboration with FORMA Therapeutics that entailed screening roughly 1.7 million compounds against ten targets using biochemical and cell-based assays. The effort resulted in chemical probes against BCL6 and SOS1 and a clinical compound against the latter. Another target was the activated (GTP-loaded) form of KRAS^G12D. Of the 6917 hits from the primary AlphaScreen assay, 1535 gave dose-response curves and passed various counter screens. Of these, 87 representative compounds were tested in STD NMR and thermal shift assays. Only seven confirmed by STD NMR, but these did not confirm by SPR or crystallography.

 

In parallel, the researchers were successfully using FBLD to develop inhibitors of KRAS^G12D, which we wrote about here. Some of the fragment hits were structurally similar to those from the HTS screen, and further searching of the FORMA library led to fairly potent (high nanomolar or low micromolar) hits in the AlphaScreen assay. Two of these even yielded crystal structures, though despite their chemical similarity to one another they bound to the protein in completely different orientations.

 

Unfortunately, follow-up work “revealed erratic structure-activity relationships,” and upon resynthesis the compounds were much less active. At this point the researchers became suspicious, and analyses of the original samples showed they contained >20,000 ppm of palladium contamination. Furthermore, PdCl₂ itself turned out to be a low micromolar inhibitor in the assay.

 

Metals are frequently used as catalysts or reagents in organic synthesis and can be difficult to completely remove during purification. Worse, their presence is often not detectable using standard purity assessments such as HPLC and NMR. Particularly in the case of fragments, which are expected to have low affinities, a small amount of metal contaminant could give a reasonable-looking but misleading signal in an assay.

 

To avoid this problem in the future the researchers developed a Metal Ion Interference Set, or MIIS, consisting of a dozen different metal ions and other salts, all soluble in DMSO so as to be compatible with typical screens. The MIIS is now routinely screened before initiating HTS campaigns, and the results of 74 assays are summarized in the paper. Pd²⁺, Au³⁺, and Ag¹⁺ are particularly nasty, often giving IC₅₀ values < 1 µM, but every metal gave IC₅₀ values < 10 µM in at least two assays. Biochemical assays such as AlphaScreen or TR-FRET were more susceptible to artifacts, with 20.9% showing IC₅₀ < 10 µM, while biophysics assays such as mass spectrometry were better behaved, with only 2.3% showing IC₅₀ < 10 µM. Cellular assays were also surprisingly robust, with 6.3% showing IC₅₀ < 10 µM.

 

This is a nice paper showing that even a massive screen may produce no useful chemical matter. Soberingly, the fact that some of the fragments gave reasonable-looking crystal structures even though the functional activity came from metal contaminants is a salutary reminder that just because you have a crystal structure of a bound ligand doesn’t mean you have a viable starting point.

 

Forewarned is forearmed, and the MIIS appears to be a valuable tool for assessing assay sensitivity to metal ions, which are all too often lurking invisibly in compound samples.

What belongs in the Protein Data Bank?

The rise of high-throughput crystallography is among the most exciting recent developments for fragment finding. Historically deemed too slow for primary screening, crystallography was reserved for select hits from an assay cascade. Now crystallographic screens up-front sometimes yield hundreds of hits. Many have been deposited in the Protein Data Bank (PDB). In a recent (open access) Protein Sci. commentary, Mariusz Jaskolski (Mickiewicz University), Bernhard Rupp (Medical University Innsbruck), and collaborators in the US question this practice.

 

In particular, the researchers ask whether molecules processed using Pan-Dataset Density Analysis (PanDDA) belong in the PDB. The method, which we described here, is typically used when hundreds of compounds have been soaked into crystals of the same protein. Most molecules will not bind, and these empty structures can be averaged to provide a background map to better identify weakly-bound ligands that may have only partial occupancy.

 

The researchers seem suspicious of this technique, referring to “supposed ligands” that may “confuse most biomedical researchers” and “degrade the PDB integrity,” the effect of which “could be disastrous.” To support their argument, they provide two examples from the PDB where the atomic models diverge from the electron density calculated using conventional methods and one with wonky statistics.

 

To avoid “contamination of the PDB by suboptimal structures,” the researchers suggest depositing structures from large-scale crystallographic screens in a separate database. Alternatively, they suggest clearer annotation. (To be fair, all three of the examples cited are already prominently marked “PanDDA analysis group deposition.”)

 

Needless to say, this is controversial. In a bioRxiv preprint, Manfred Weiss (Helmholtz-Zentrum Berlin) and collaborators in the US, Germany, Sweden, and the Netherlands, some of whom co-developed PanDDA, take a different view.

 

The researchers agree that group depositions need to be marked clearly, but they argue that they squarely belong in the PDB rather than in a separate repository. Moreover, “commentaries that underestimate the knowledge of PDB users, that ignore the opportunities present in heterogenous crystallographic data, and that miss out on chances for education on structure quality do more harm than good.”

 

The three examples described by Jaskolski and colleagues are re-examined, and while it is true that two of them do show poor occupancy using conventional methods, the ligands are clearly visible when PanDDA is used. (In the third case, there was an error in the resolution cutoff during automated processing, but the data could be successfully reprocessed manually.)

 

PanDDA was developed specifically to identify small, low occupancy ligands, so the researchers argue that these entries “cannot and should not be treated in the same way” as other ligands. Banning them from the PDB would potentially impede future research.

 

Weiss and colleagues refer to the Structural Genomics campaign of the late 1990s and early 2000s to solve myriad structures of diverse proteins, most of which were not being otherwise studied. At the time some commentators derided this effort as “stamp collecting.” Yet the number and diversity of structures thus deposited into the PDB likely contributed to the success of automated protein folding algorithms such as AlphaFold2.

 

Similarly, including structures from PanDDA processing could lead to unforeseen advances. For example, Weiss and colleagues suggest we may be able to “extract all aspects of conformational as well as of compositional heterogeneity out of all these data sets.” A better understanding of the role of protein dynamics in ligand binding is likely to require thousands of similar datasets of the kind being uploaded.

 

Personally, I believe that scientists should be wary of all published information. As the old saying goes, trust, but verify. As evidenced by my five-part series “Getting misled by crystal structures,” even conventional structures in the PDB should not necessarily be taken at face value. With that precaution, I’ll hold with the conclusion of Weiss and colleagues: “As long as the data is there, let’s embrace it and make it available!”

Fragments vs DDAH – covalent and noncovalent

Using biochemical assays to find fragments sidesteps the need for expensive biophysical instruments, but is fraught with difficulties. In a recent paper in Bioorg. Med. Chem., Thomas Linsky and Walter Fast at the University of Texas, Austin, describe how they used functional screening to discover fragment inhibitors of dimethylarginine dimethylaminohydrolase (DDAH), an enzyme involved in nitric oxide production.

The researchers used a 4000-member fragment library from ChemBridge, which they screened against two different isoforms of DDAH at 0.1 mM in a biochemical screen. This resulted in 79 hits against the human enzyme and 44 hits against a bacterial (P. aeruginosa) DDAH, 101 in total. 66 of these were then repurchased along with 41 analogs, and tested in a completely different biochemical assay against human DDAH-1; 31 showed >20% inhibition at 0.4 mM, including only 22 of the original hits, suggesting that many of the initial hits were indeed false positives. Further studies showed that 5 of the 31 compounds interfered with this secondary assay (ie, they showed “inhibition” even in the absence of enzyme), suggesting that they were false positives in both the primary and secondary biochemical assays.

DDAH contains an active-site cysteine, and the researchers wanted to exclude molecules that might be generically reactive, so they incubated the remaining 26 compounds with the low-molecular weight thiol glutathione and then retested them; this eliminated another 21 compounds.

Finally, the remaining 5 compounds were examined by mass-spectrometry, and one of these turned out to be a compound other than what was listed on the bottle! This left just 4 legitimate fragment hits.

Two of these compounds were 4-halopyridines, which, although not generally reactive with thiols, could covalently modify the active site cysteine of DDAH (see here for more details). The other two compounds were reversible, competitive inhibitors of human DDAH-1. Although low affinity (Ki values of 0.8 and 1.7 mM), they had respectable ligand efficiencies (0.38 and 0.29 kcal/mol/atom, respectively).

Interestingly, when Linksy and Fast retrospectively analyzed where the four validated fragments came from, they found that only the two halopyridines were detected in the primary screen; the two reversible fragment hits had been purchased for the secondary round of screening as analogs of primary hits.

This is a richly detailed and well-executed example of fragment-based screening in academia. It demonstrates once again that high-concentration biochemical screens can be used to find fragments, but be prepared to wade through a lot of junk: only about 2% of the original hits proved to be legitimate (see here for similar results from Vernalis on a different target). It also illustrates the utility of exploring analogs of initial fragment hits; in this case, even though most of the primary hits didn’t hold up, they nonetheless led to new fragments. Of course, this does raise the question of how spurious primary hits can lead to genuine inhibitors – what do you think?