Display MoreYour group's strategy seems to me to be among the sanest things I have heard on this site. But you are travelling the same road as the medical R&D community went down near the turn of the century. Large companies trying to industrialize scientific findings found out there was a big problem reproducing research published in even top-flight journals. The first time I heard of this was on the academic grapevine regarding a large drug company who decided that in order to push ahead in a particular field they wanted to establish the capability to reproduce 16 basic findings in-house. They tried, but in the end failed to replicate 15 of the 16 (I don't think I ever learned the name of the company). Since then, failure to replicate has become widely acknowledged and has been the subject of a series of editorials in Science and Nature. Current figures seem to be that 25-70% of peer-reviewed published findings cannot be replicated. The 25% figure comes from the physical sciences and the higher figures comes from biology and psychology. Reasons for failure to replicate are about the same as you outlined (analysis mistakes, self-delusion, etc), but if you ask the authors of the non-replicated works what is happening they say that the experimental conditions were not copied exactly. Sound familiar? .
What percentage of projects do you end up setting aside in your validation process?
Here is an example of a venture capitalist's experience in biology
So then, since the Pons-Fleischmann Anomalous Heat Effect has been replicated in 153 peer reviewed journals, across more than 180 labs and 14,700 instances, would you say that it is a well established scientific anomaly? Because there are people on this very forum who disagree with that.