Introducing the Chemical Probes Portal

Chemical probes can be incredibly powerful reagents for understanding biology. A potent, selective, and cell-active modulator of a specific protein can be invaluable for figuring out what that protein actually does. Fragment-based methods can be effective at identifying these tool compounds, as we've described here and here.

Unfortunately, good chemical probes are difficult to discover, and scientists are left struggling with suboptimal reagents that hit multiple targets, often through pathological mechanisms. This leads to "pollution of the scientific literature," in Jonathan Baell's memorable phrasing. Despite our occasional PAINS Shaming, high-profile articles in C&EN and Nature, and even a dedicated blog, the problem continues. What is to be done?

Yesterday, a team of 53 authors from 46 academic and industrial organizations published a Commentary in Nature Chemical Biology entitled "The promise and peril of chemical probes" (see here for excellent coverage in Nature, here for Science's take, and here for In the Pipeline). This provides a good working definition for a chemical probe. According to the Structural Genomics Consortium, a chemical probe for epigenetics targets must have:

• Potency < 100 nM against the desired target
• >30-fold selectivity vs related targets
• On-target cell activity < 1 µM

It should also be profiled against a larger panel of potential off-targets, and a related inactive compound (such as a stereoisomer) should be available as a control.

After discussing examples of high-quality probes, the researchers turn their attention to what they term – rather charitably – "probes of lesser value:"

The continued use of these probes poses a major problem: tens of thousands of publications each year use them to generate research of suspect conclusions, at great cost to the taxpayer and other funders, to scientific careers and to the reliability of the scientific literature.

The authors then go on to describe best-practices. For example, even high-quality probes can give spurious results when used at high concentrations. As Paracelsus recognized five centuries ago, the dose makes the poison.

All of this is important, but as the authors acknowledge, it's been said before. What really differentiates the Commentary is the simultaneous launch of a companion web site, the Chemical Probes Portal. Its creators hope that this will lead to vigorous community discussion around questions such as:

Is there a probe for my target protein?

Which ones should I use?

How should I use this probe properly?

Is this probe suitable for use in animal models?

Currently the Portal lists just seven probes with links to references and descriptions of selectivity, solubility, and the like. All of these are “good probes,” but hopefully this will expand: the paper itself discusses the shortcomings of molecules such as staurosporine, chaetocin, obatoclax, and gossypol, and including them in the portal with detailed warnings would be valuable for the scientific community.

I hope this takes off. Understanding the natural world is hard enough even with well-behaved reagents and carefully controlled experiments. Practical Fragments will check back in a year or so to see how the site is doing. In the meantime, probe cautiously!

Sloppy science

As regular readers may have discerned, I’m favorably disposed to most of the papers I highlight. They may have flaws or inconsistencies, but, with rare exceptions, I generally just ignore particularly problematic publications. Last year Teddy introduced the term PAINS-shaming to draw attention to – how shall we phrase it? – less salubrious specimens. Building on this alliterative theme, today’s post is about sloppy science. A fundamental tenant of sound science is to consider alternate explanations for results. Ignore this at your peril.

An example was published in J. Cancer Prev. The researchers were interested in a mutant of isocitrate dehydrogenase 1 (IDH1), a hot cancer metabolism target. They screened 500 fragments in a functional spectrophotometric assay, with each fragment present at the very low concentration of 5-10 µM. One of these inhibited the mutant protein by 80% – pretty impressive for a fragment. Until you look at the structure: 2-(3-trifluoromethylphenyl)isothioazol-3(2H)-one (shown below).

Fifty years ago, researchers showed that this chemical class (isothiazolinones, also called isothiazolones) could react with thiols, like this:

Isothiazolinones have been categorized as PAINS, though they do not show up in the original computational filters. However, Pete Kenny has (repeatedly) stated that having a dubious structure should not automatically disqualify a compound from further investigation, so what else do we know about isothiazolinones?

Well, there’s this paper, which concludes a discussion of isothiazolinones by stating:

We could not develop these into useful compounds and ultimately the structure–activity relationship (SAR) was uninterpretable. Most insidiously, there were encouraging aspects of sharp SAR as there always are with these PAINS, but this is eventually overwhelmed by flat and nonsensical SAR. Unpredictable nonspecific cytotoxicity was manifest. We found our compounds to be rapidly reactive with thiols under assay conditions.

Of course, one could argue that this is anecdotal. But then there’s this paper, with the unambigious title “Isothiazolones; thiol-reactive inhibitors of cysteine protease cathepsin B and histone acetyltransferase PCA”. The first line of the abstract states:

Isothiazolones and 5-chloroisothiazolones react chemoselectively with thiols by cleavage of the weak nitrogen-sulfur bond to form disulfides.

The researchers go on to demonstrate this using both small molecules and proteins, and some of the compounds they investigate are structurally quite similar to the hit here.

So in all likelihood the fragment described in the most recent paper reacts with one or more cysteine residues in IDH1, of which there are several. It is notable that the researchers conducted their assay in the absence of added thiol reducing agents, so modification of the cysteines would effectively be irreversible under their assay conditions.

What we have here is the re-identification of a known thiol-reactive molecule without any acknowledgement or apparent awareness that the molecule is reactive. I have no problem with covalent inhibitors, but I do have a problem with a generically reactive molecule being touted “for a future lead development”, as the researchers state in the abstract. It took me just minutes to track down the references above, and the fact that neither the researchers nor the reviewers did so is inexcusable.

Granted, this paper is not published in a high profile journal, and the easiest response would be to ignore it. It is certainly not the only one of its kind. Doing so, however, implicitly endorses sloppy science. This paper will undoubtedly pad the resumes of the authors. Highlighting its problems will hopefully make others wary of wasting time with this new "selective inhibitor."

New PAINS, and their painful mechanisms

Pan-assay interference compounds – PAINS – are a topic that has come up repeatedly at Practical Fragments (see here, here, and here for starters). Indeed, they form the basis of our occasional (and controversial) “PAINS shaming” series (see here, here, here, and here). In a paper recently published online in J. Med. Chem., HTSPains-master Mike Walters (University of Minnesota) and collaborators at the Mayo Clinic College of Medicine and AstraZeneca characterize several new classes of PAINS (also covered at In the Pipeline). I was honored with the invitation to write a Viewpoint on the topic. Since both papers are open-access I’ll just briefly touch on a few key points here.

First, one of the only critics of the PAINS concept is concerned that PAINS were originally defined on the basis of their over-representation as screening hits in one set of assays. The new paper goes beyond empiricism to characterize mechanisms, which involve non-specific reactions with thiols. (The “non-specific” aspect is key to their undesirability, as covalent drugs can be quite attractive.)

Second, just as “not every clam will hurt you”, not every molecule with a PAINS substructure will show activity in every assay. These “structure-interference relationships” (SIR) can be mistaken for “structure-activity relationships” (SAR), making PAINS all the more insidious. The researchers explore some of the reasons for the observed SIR.

Third, one of the saddest parts of the new paper is a list of dozens of references in the supplemental information in which PAINS were reported as screening hits or probes. It’s a safe bet that most – if not all – of these should be disregarded.

Finally, publishing this list of new PAINS will allow people to steer clear of them. To borrow from Hippocrates: chemical space is big, life is short. Why waste time working with chemotypes known to be pathological? 

Not Every Clam will Hurt You

I grew up on a wonderful little island called Jamestown (although technically it is Conanicut island and the town is Jamestown).  It was a great place to grow up, especially because in the summer we lived walking distance to the beach.  One of the very cool things about the beach is that it has an awesome sand bar that pokes up at low tide.  That was the most fun part of the beach to me.  One of the things we did was stand on the sand bar and dig our feet into it.  You scrunch your toes into the sand until you hit something hard.  Then you excavate with your toes around it.  If you got a foot or so down (this took some patience) and got lucky you would find a quahog.  Thems is good eating.  Many an hour was spent doing this and bringing home dinner.  Sometimes, you found something hard and started more excavation around it...and WHAM!  Not a quahog, but a razor clam instead! There goes your day as blood starts gushing out of your foot stuck in a foot of mud.  You can come up with a different approach to find the clam, but you still get hurt by razor clams.  Eventually, you give up digging for clams with your feet because you hit one too many razor clams and you get your clams from Zeek's Creek Bait Shop.

We here at Practical Fragments have a great job, we get to pontificate on fragment papers.  As most people know, its Good Cop (Dan)/Bad Cop (Me) by and large.  It works for us and the blog gets read by more than our mothers.  But this is an opinion blog, and as everyone knows (G-rated version): Opinions are like belly buttons, everyone has one.  We welcome contrary opinions, sometimes even try to provoke them.  Dan and I do very little coordinating for this blog beyond the "I will have something for Monday".  So, when we both find something that bothers us, well that's worth discussion.  One topic in particular Dan and I have found is PAINS (Pan Assay INterference Compounds).

PAINS are gaining traction as things to avoid in screening collections; there's even a Facebook page.  The literature is pretty clear as to what these are and why they are bad.  In my eyes, I am fine just removing them all from my screening deck and being done with them.  In fragment space, there is MORE than enough other compounds that I don't worry about missing whatever chemical space they live in.  However, as I have often said, a fragment is like pornography, the viewer knows it when they see it.  As you may know, I am not one for hard and fast rules.  If you want to keep PAINS-like compounds in your collection, fine by me.  BUT, you must be aware that they are PAINS.  You must know that they must be kept to a higher standard of evidence, you must do more controls, etc. And of course, if you are making tools then it doesn't matter if it is a PAIN (Dan and I disagree here.) 

So, Practical Fragments gadfly Pete Kenny has a post up at his (recently renamed blog) about PAINS.  In it, he takes Practical Fragments to the woodshed over PAINS, even though his main point is about how we make decisions on data.  He starts his commentary by pointing to this post and calling it a "vapid rant".  As noted in the comments to my post Pete immediately took exception to it believing the burden of proof should be on the blogger to demonstrate the guilt of the compound(s) in question.  He also cites this post as one that should be wary of calling something crap or pollution.  He then goes in to the ontogeny of PAINS and raises some points:

• PAINS study is irreproducible because structures and targets are not revealed
• Only 6 HTS campaigns were analyzed when 40+ were available
• All screens used Alpha-Screen, so this may not be very "PAN"

He then goes deep into the actual structure of rhodanines and how some are good, or less bad.  I think Pete has lost the forest for the trees, or shrubs.  Its not that there are probably some rhodanines that are NOT bad actors; but we know many are, and these require a higher level of confirmation than other compound classes.  Not every clam you dig for is going to slice your foot open, but when enough do, but after enough do, you change your approach.

PAINS Shaming, part deux

So, as regular readers know, we have declared war on PAINS on the blog.  As part of that effort, I (we?, not sure if Dan wants to be associated directly with it) introduced PAINS Shaming. Well, thanks to Angelo Pugliese and Duncan McArthur at the Beatson we have the latest paper to shame. 

The nice thing is that they come right out and call it like it is: a Rhodanine.  To reiterate, from the comment by Baell and Walters:

Rhodanines exemplify the extent of the problem. A literature search reveals 2,132 rhodanines reported as having biological activity in 410 papers, from some 290 organizations of which only 24 are commercial companies. The academic publications generally paint rhodanines as promising for therapeutic development. In a rare example of good practice, one of these publications (by the drug company Bristol-Myers Squibb) warns researchers that these types of compound undergo light-induced reactions that irreversibly modify proteins. It is hard to imagine how such a mechanism could be optimized to produce a drug or tool. Yet this paper is almost never cited by publications that assume that rhodanines are behaving in a drug-like manner.

And as predicted, this paper does not cite Voss et al.  They cite dose-depedent responses for their compounds.  Does it matter?  Not to me.  To their credit, they call these molecules tools, but also tout them for future therapeutic development.  A PAIN can be a useful tool or even lead to non-PAIN containing compounds, but it requires a higher level of proof.  I don't see that here. 

So, here is your PAINS Shaming (Holiday themed): 

PAINS in Nature

Practical Fragments has previously noted that many pan-assay interference compounds (PAINS) can be found in nature. Indeed, they’ve also found their way – unintentionally – into journals published by Nature Publishing Group. In an effort to educate the scientific community about these artifacts, Jonathan Baell (Monash University) and Mike Walters (University of Minnesota) have just published a Comment in Nature entitled “Chemical con artists foil drug discovery”. This is the clearest discussion I’ve yet seen of PAINS, and it deserves to be widely read.

Since the article is open-access I won’t go into depth here, other than to say that the researchers propose three steps to avoid PAINS.

1) Learn disreputable structures.

As a start, the paper provides a rogue’s gallery of some of the worst molecules, along with memorable interpretations by award-winning New Yorker cartoonist Roz Chast. It would be nice to see this posted in every academic screening center.

2) Check the literature.

This is even easier than having to learn structures, and should prevent people from embarrassing themselves by publishing research that is obviously flawed.

3) Assess assays.

Multiple orthogonal assays are useful for all science, not just FBLD!

Together with the recent C&ENstory and ACS symposium, this article ensures that PAINS are finally reaching the level of recognition such that scientists, reviewers, and editors will no longer be able to claim ignorance. Willful negligence may be another matter, but at least people will be able to recognize it as such.

PAINS in C&EN

Just a quick heads-up that Celia Arnaud has a nice story on pan-assay interference compounds (PAINS) in the latest issue of Chemical and Engineering News. Celia attended the PAINS symposium at the ACS meeting last month and spoke with several of the speakers for the piece.

As far as I know this is the first time C&EN has devoted an entire article to PAINS. It’s a fast read so I won’t summarize it here, other than to say that she does pick up on the concept of PAINS-shaming, which Teddy has also advocated. Although Practical Fragments has done some of this, most PAINS are not fragments, so it wouldn’t really be appropriate to do much of it here (though please visit HTSpains).

I do hope Celia’s article is widely read by practicing scientists, journal editors, and reviewers. The need for more PAINS recognition is amply illustrated by this article published in the most recent issue of J. Med. Chem. which reviews reported inhibitors of AP-1, many of them dubious. Let's hope that the C&EN piece cuts down on future pollution.

Who's Reviewing this Crap?

Dan and I teach a short course on FBDD; next chance to catch a version in October.  My favorite part of Dan's section is when he goes off on PAINS and the continuing pollution of the literature.  Rhodanines in particular get Dan's dander up.  I often come off as anti-academic because many of their "drug discovery" papers are crap.  Or they claim something is a lead without it being one.   I think it is time to start PAINS shaming these papers.  For those of you not intimately familiar with "shaming" it is very popular with dog shaming.  Well, here is our first PAINS Shaming.  This paper (pointed out by Matt Netherton at B-I, thanks Matt!) unabashedly points out the offensive molecule as a rhodanine. In the article, they point out:

What is particularly interesting about the most active species investigated here is that it has a structure that is very similar to that found in the drug epalrestat, an aldolase reductase inhibitor that is used to treat diabetic neuropathy, and is approved for clinical use in Japan, China, and India. This is encouraging because rhodanines as a class are known to often have activity in widely different assays, and indeed computer programs such as PAINS categorize, [our compounds] (as well as epalrestat) as possible “pan assayinterference compounds”. This can mean that the compounds cause false positives in assays, or that they may be multitarget inhibitors. In some cases multitargeting may be undesirable; however, in the context of anti-infective development, multitargeting is expected to increase efficacy as well as decrease the possibility of resistance development, both very desirable features.

  I am sorry, but this is exactly the Underpants Gnomes business plan. All I can say after reading this is who's reviewing this crap and saying it is all right?