The poll results are in, and it looks like most of the 46 respondents look askance at fragments that are larger than about 20 heavy atoms:
Since each non-hydrogen atom adds about 13 Da to a molecule, this means fragment sizes are limited to about 260 Da, well below the Rule of 3. And some folks are even more stringent – 30% of respondents set an upper limit of 16 non-hydrogen atoms, or about 210 Da.
By way of comparison, I looked at the size of fragments in a fairly large (albeit somewhat dated) review. Of the 42 fragments reported, 79% consist of 20 or fewer heavy atoms, so clearly this is a fruitful area.
Of course, as the graph above shows, larger fragments have been discovered and advanced, but perhaps it is generally better to avoid the more obese fragments.