Catching virtual cheaters

As experienced practitioners of fragment-based lead discovery will know, the best way to avoid being misled by artifacts is to combine multiple methods. (Please vote on which methods you use if you haven’t already done so.) Normally this advice is for physical methods, but what’s true in real life also applies to virtual reality, as demonstrated in a recent J. Med. Chem. paper by Brian Shoichet and collaborators at University of California San Francisco, Schrödinger, and University of Michigan Ann Arbor.

 

The Shoichet group has been pushing the limits of computational screening using ever larger libraries. Five years ago they reported screens of more than 100 million molecules, and today multi-billion compound libraries are becoming routine. But as more compounds are screened, an unusual type of artifact is emerging: molecules that seem to “cheat” the scoring function and appear to be virtual winners but are completely inactive when actually tested. Although rare, as screens increase in size these artifacts can make up an increasingly large fraction of hits.

 

Reasoning that these types of artifacts may be peculiar to a given scoring function, the researchers decided to rescore the top hits using a different approach to see whether the cheaters could be caught. They started with a previous screen in which 1.71 billion molecules had been docked against the antibacterial target AmpC β-lactamase using DOCK3.8, and more than 1400 hits were synthesized and tested. These were rescreened using a different scoring approach called FACTS (fast analytical continuum treatment of solvation). Plotting the scores against each other revealed a bimodal distribution, with most of the true hits clustering together. Of the 268 molecules that lay outside of this cluster, 262 showed no activity against AmpC even at 200 µM.

 

Thus encouraged, the researchers turned to other studies in which between 32 and 537 compounds had been experimentally tested. The top 165,000 to 500,000 scoring hits were tested using FACTS, and 7-19% of the initial DOCK hits showed up as outliers and thus likely cheaters. For six of the targets, none of these outliers were strong hits. For each of the other three, a single potent ligand had been flagged as a potential cheater.

 

To evaluate whether this “cross-filtering” approach would work prospectively as well as retrospectively, the researchers focused on 128 very high scoring hits from their previous AmpC virtual screen that had not already been experimentally tested. These were categorized as outliers (possible cheaters) or not and then synthesized and tested. Of the 39 outliers, none were active at 200 µM. But of the other 89, more than half (51) showed inhibition at 200 µM, and 19 of these gave K_i values < 50 µM. As we noted back in 2009, AmpC is particularly susceptible to aggregation artifacts, so the researchers tested the ten most potent inhibitors and found that only one formed detectable aggregates.

 

In addition to FACTS, the researchers also used two other computational methods to look for cheaters: AB-FEP (absolute binding free energy perturbation) and GBMV (generalized Born using molecular volume), both of which are more computationally intensive than either FACTS or DOCK. Interestingly, GBMV performed worse than FACTS, finding at best only 24 cheaters but also falsely flagging 9 true binders. AB-FEP was better, finding 37 cheaters while not flagging any of the experimentally validated hits.

 

This is an important paper, particularly as virtual screens of multi-billion compound libraries become increasingly common. Indeed, the researchers note that “as our libraries grow toward trillions of molecules… there may be hundreds of thousands of cheating artifacts.”

 

And although the researchers acknowledge that their cross-filtering aproach has only been tested for DOCK, it seems likely to apply to other computational methods too. I look forward to seeing the results of these studies.