I grew up on a wonderful little island called Jamestown (although technically it is Conanicut island and the town is Jamestown). It was a great place to grow up, especially because in the summer we lived walking distance to the beach. One of the very cool things about the beach is that it has an awesome sand bar that pokes up at low tide. That was the most fun part of the beach to me. One of the things we did was stand on the sand bar and dig our feet into it. You scrunch your toes into the sand until you hit something hard. Then you excavate with your toes around it. If you got a foot or so down (this took some patience) and got lucky you would find a quahog. Thems is good eating. Many an hour was spent doing this and bringing home dinner. Sometimes, you found something hard and started more excavation around it...and WHAM! Not a quahog, but a razor clam instead! There goes your day as blood starts gushing out of your foot stuck in a foot of mud. You can come up with a different approach to find the clam, but you still get hurt by razor clams. Eventually, you give up digging for clams with your feet because you hit one too many razor clams and you get your clams from Zeek's Creek Bait Shop.
We here at Practical Fragments have a great job, we get to pontificate on fragment papers. As most people know, its Good Cop (Dan)/Bad Cop (Me) by and large. It works for us and the blog gets read by more than our mothers. But this is an opinion blog, and as everyone knows (G-rated version): Opinions are like belly buttons, everyone has one. We welcome contrary opinions, sometimes even try to provoke them. Dan and I do very little coordinating for this blog beyond the "I will have something for Monday". So, when we both find something that bothers us, well that's worth discussion. One topic in particular Dan and I have found is PAINS (Pan Assay INterference Compounds).
PAINS are gaining traction as things to avoid in screening collections; there's even a Facebook page. The literature is pretty clear as to what these are and why they are bad. In my eyes, I am fine just removing them all from my screening deck and being done with them. In fragment space, there is MORE than enough other compounds that I don't worry about missing whatever chemical space they live in. However, as I have often said, a fragment is like pornography, the viewer knows it when they see it. As you may know, I am not one for hard and fast rules. If you want to keep PAINS-like compounds in your collection, fine by me. BUT, you must be aware that they are PAINS. You must know that they must be kept to a higher standard of evidence, you must do more controls, etc. And of course, if you are making tools then it doesn't matter if it is a PAIN (Dan and I disagree here.)
So, Practical Fragments gadfly Pete Kenny has a post up at his (recently renamed blog) about PAINS. In it, he takes Practical Fragments to the woodshed over PAINS, even though his main point is about how we make decisions on data. He starts his commentary by pointing to this post and calling it a "vapid rant". As noted in the comments to my post Pete immediately took exception to it believing the burden of proof should be on the blogger to demonstrate the guilt of the compound(s) in question. He also cites this post as one that should be wary of calling something crap or pollution. He then goes in to the ontogeny of PAINS and raises some points:
- PAINS study is irreproducible because structures and targets are not revealed
- Only 6 HTS campaigns were analyzed when 40+ were available
- All screens used Alpha-Screen, so this may not be very "PAN"
He then goes deep into the actual structure of rhodanines and how some are good, or less bad. I think Pete has lost the forest for the trees, or shrubs. Its not that there are probably some rhodanines that are NOT bad actors; but we know many are, and these require a higher level of confirmation than other compound classes. Not every clam you dig for is going to slice your foot open, but when enough do, but after enough do, you change your approach.
I found this to be very informative and I learned a lot about clams (but nothing about drug discovery). Before you dig yourself in any deeper (or step on any more of those razor shap clams) you might want to take a real close look at structures C1-C5 in your 'PAINS Shaming, part deux post'...
ReplyDeleteSince I wrote more than a thousand words in the comments to the rhodanine shaming post I'll be brief here.
ReplyDeleteThe point of categorizations such as PAINS is to make people aware of potential pitfalls. Scientific resources are scarce, and when someone publishes a new probe for "Hot Target X" it is likely to be sold by suppliers and used by other researchers. If that probe has not been sufficiently validated, this can lead to erroneous conclusions and pollution of the scientific literature.
The real world rarely falls into neat categories, but certain moieties are prone to exhibit a variety of bad behaviors. In these cases the burden of proof should be on the authors to show that their molecule is acting legitimately. Pointing out cases where authors fail to meet high standards is more than ranting, it is serving science.
Even in cases where PAINS are subsequently recognized as such, the damage may be hard to undo. For example, SJ-172550 was reported as an inhibitor of an important protein-protein interaction, but its discoverers subsequently published an extensive follow-up study concluding that it should not be used as a probe. (This conclusion was recently confirmed here.) Despite these warnings, the compound is currently sold by more than a dozen vendors, none of whom mention the caveats. Thus, despite the best efforts of the top-tier scientists who actually discovered the molecule, it is potentially misleading investigators.
While it is beyond the scope of Practical Fragments to call out more than a few examples of sloppy science (please see HTSPains for that) doing so occasionally will hopefully be educational, as well as entertaining.
Hi Dan, I agree that “the burden of proof should be on the authors to show that their molecule is acting legitimately” when they present compound(s) as probes or leads. Where we differ is that I would argue that that burden remains whether or not the compounds show a high hit rate against a selected panel of six AlphaScreen assays. I agree that, “The real world rarely falls into neat categories, but certain moieties are prone to exhibit a variety of bad behaviors”. However link between moiety and bad behaviors needs to be clearly demonstrated. Put another way, you can’t categorize real world events as ‘correlated’ or ‘uncorrelated’ and a strong correlation cannot be taken as definitive proof of a causal relationship. The key question here is how confident are we that the behavior(s), in contact with a specific protein, of one or two compounds containing a substructure can be extrapolated to any compound that contains that substructure in contact with any protein. The concept of activity cliffs may be relevant here.
ReplyDeleteLet’s take a look at another PAIN. A few posts ago, you featured a nice NMR paper by Prof Krimm describing the use of catechols as tools for exploring binding modes. She did mention that catechols had been described as PAINS but she certainly didn’t need to because she was characterizing the interaction of the catechols directly by NMR. Put another way, if you’re characterizing binding directly by NMR the fact that some compounds that incorporate the catechol substructure show a high hit rate against a selected panel of six AlphaScreen assays is of vanishingly small relevance. There is also the question of whether any of the catechols that hit some of the six Alphascreens had any PAIN (e.g. rhodanine substructure) present in the remaining portions of their structures.
So please humor me and explain the problem with compounds C1 to C5 in the paper that Teddy blasted?
The burden of proof for any new putative ligand of a protein must be on the authors, be it PAIN or not.
ReplyDeleteTo publish without showing biochemical mechanism seems a huge oversight to me. Formal competition experiments would rule out the vast majority of PAINs. Any compound which appeared to be neither competitive or uncompetitive (that is to say not dependent of concentration of the displaced partner) should immediately be held to higher scrutiny.
This benefits everyone, PAINs are weeded out, and if we are lucky there may be a new allosteric site/mechanism of inhibition for future studies.
Anonymous, is it fact that "Formal competition experiments would rule out the vast majority of PAINs" or merely your opinion?
ReplyDeleteYes, it a fact. Pains are as a whole, non-competitive inhibitors. Their mechanism does not rely on the concentration of the displaced binding partner. It is incredibly rare that an small molecule is 100% non competitive as they generally favor one isoform of the protein.
ReplyDeleteNon-competition alone is not a reason to discard a ligand, but it certainly means the mechanism should be held to a higher degree of scrutiny.
To clarify; I meant one conformation of the protein, not isoform.
ReplyDeleteAnonymous, If you're going to assert that "Yes, it a fact. Pains are as a whole, non-competitive inhibitors" you're really going to have to back this up with some evidence. In the original PAINS article the offending chemotypes were identified on the basis of hitting multiple assays in a panel of six AlphaScreens and it would not be possible to tell whether the disruption of the protein-protein interactions was competitive or otherwise (not sure how these terms apply for PPIs). I have encountered problems with non-competitive inhibitors but I wouldn't have classified the relevant screening hits as PAINS in that situation.
ReplyDeletePete, I'm going to assume you aren't an ezymologist or pharmacologist here. If you are, I apologize in advance if this sound patronizing.
ReplyDeleteIf we take a simple inhibition system, where a small molecule (it could be a labeled peptide etc.) is displaced from a target as our model, we can look at how some common PAINs mechanisms will affect it in regards to competition between the displaced ligand and the 'HIT'. For all these assays we will assume the protein concentration is so low as not to affect the assay.
Aggregators will self assemble at a physiochemical concentration limit, much like the CMC for surfactants. The resultant nano-particles sequester the protein on to the surface, but are less likely to interact with the peptide. This CMC will not change in regards to the peptide concentration (or will not change with respect to the cheng prusoff equation) thus the PAINs appear noon competitive. As follow up one may vary the concentration of protein instead, if the IC50 changes here (and [I] is at leat 10*>[P]) then the first-order assumption is violated and one should have serious questions about either the compounds or the assay. For publications see work by Brian Shoichet's group. Of particualr interest is a paper where they profiled the SAR of series which switched from competitive to non-competitive aggregatior.
For redox compounds the mechanism of action is either directly reacting with the protein, or production of H2O2 via a reducing agent in the buffer. In both cases the mechanism will not appear competitive with the displaced ligand by standard kinetics. For the former, depending on the reversibility of the reaction the mechanism will appear time dependent, non-competitive with ligand and roughly competitive with reducing agent. For the latter the reaction will appear non-competitive with ligand and appear uncompetitive with reducing agent. In both cases the readout will be time dependent, which should be obvious in continuous enzymatic assays, but appear as IC50s which are not stable over time in other assays. Redox compounds have the dual ability to nobble fluophores (and alpha reagents), which may depend on the concentration of the fluorescent ligand, but generally does not follow cheng prusoff.
Other irreversible compounds can be screened out via this method, they will always appear non competitive by an equilibrium method (for enzymology steady state ~ equilibrium) as they are removing the some of the active enzyme but not affecting the affinity of the ligand for the remaining enzyme.
This does not mean irreversible compounds are not useful, there has been good precedent of using michael acceptors with cystine proteases to generate reversible, non-covalent inhibitors (see PR-619 to P22077, USP7). The trick is using kinetic methods to determine whether the inhibition is a one step reaction (no measurable affinity for the site) or two step (one initial binding step based on affinity), see work by petr kuzmic for more info.
Finally, a decent pharmacologist/etymologist will be able to check whether the above step reaction is competitive with the displaced ligand, and 1:1. Often bad actors which react with specific residues will hit most of the available residues on the protein leading to a high stoichiometry of binding.
This can be done in any assay technology, SPR, DSF, ITC and NMR all have well established methods for determining competition, kinetics and reaction stoichiometry.
Apologies for the long post, and my likely terrible spelling/grammar.
Thanks, Anonymous, for your very detailed response. The point that I’m getting at here is more about what we should (and should not) be calling PAINS. In the original article, PAINS were identified on the basis of hitting multiple assays but little or no evidence was presented for them acting by specific mechanisms. The results presented in the PAINS article are certainly of interest to people using HTS but it is not clear that we should apply the term PAINS to any compound that behave badly in an assay. One problem with calling something a PAIN is that I don’t know whether you mean a compound behaving badly in an assay, a compound which has been observed as a hit in multiple assays or a compound whose molecular structure matches one or more of the substructural PAINS filters in the original article. This was the basis of my original challenge to your assertion that, "Formal competition experiments would rule out the vast majority of PAINs".
ReplyDelete