Experimental design and reporting standards for improving the internal validity of pre-clinical studies in the field of pain: Consensus of the IMI-Europain consortium

K.L. Knopp 1 , C. Stenfors 2 , C. Baastrup 3 , A.W. Bannon 4 , M. Calvo 5 , O. Caspani 6 , G. Currie 7 , N.B. Finnerup 3 , W. Huang 8 , J.D. Kennedy 1 , I. Lefevre 9 , I. Machin 10 , M. Macleod 7 , H. Rees 11 , A.S.C. Rice 8 , K. Rutten 12 , M. Segerdahl 12 , J. Serra 13 , R. Wodarski 14 , O.-G. Berge 2  and R.-D. Treede 6
  • 1 Lilly Research Laboratories, Indianapolis, IN 46285, North America, USA
  • 2 AstraZeneca R&D, CNS & Pain IMed, Sodertalje, Sweden
  • 3 Danish Pain Research Center, Aarhus University, Aarhus, Denmark
  • 4 AbbVie Inc, IL 60064-6115, North Chicago, USA
  • 5 Neurorestoration Group, Wolfson Centre for Age-Related Diseases, King’s College London, Guy’s Campus, SE1 1UL, London, UK
  • 6 Center for Biomedicine and Medical Technology Mannheim, Medical Faculty Mannheim, Heidelberg University, Berlin, Germany
  • 7 Division of Clinical Neurosciences, Chancellor’s Building, University of Edinburgh, 49 Little France Crescent, Scotland EH16 4SB, London, UK
  • 8 Pain Research, Department of Surgery and Cancer, Imperial College, London, UK
  • 9 Sanofi Recherche, Montpellier, Paris, France
  • 10 In Vivo Pharmacology Consultant, Hertange, Deal Road, Kent CT14 0BG, London, UK
  • 11 Pfizer Neusentis, The Portway Building, Granta Park, Cambridge, UK
  • 12 Grünenthal GmbH, Global Biomedical Sciences, Department of Pharmacology Pain, Aachen, Germany
  • 13 Neuroscience Technologies, Baldiri Reixac, Parc Cientifique de Barcelona, 15-21, Barcelona, Spain
  • 14 Neuroscience Research Division, Lilly Research Centre, Eli Lilly & Co. Ltd, Erl Wood Manor, Windlesham, UK
K.L. Knopp, C. Stenfors, C. Baastrup, A.W. Bannon, M. Calvo
  • Neurorestoration Group, Wolfson Centre for Age-Related Diseases, King’s College London, Guy’s Campus, London, SE1 1UL, UK
  • Search for other articles:
  • degruyter.comGoogle Scholar
, O. Caspani
  • Center for Biomedicine and Medical Technology Mannheim, Medical Faculty Mannheim, Heidelberg University, Berlin, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar
, G. Currie
  • Division of Clinical Neurosciences, Chancellor’s Building, University of Edinburgh, 49 Little France Crescent, Scotland EH16 4SB, London, UK
  • Search for other articles:
  • degruyter.comGoogle Scholar
, N.B. Finnerup, W. Huang, J.D. Kennedy, I. Lefevre, I. Machin, M. Macleod
  • Division of Clinical Neurosciences, Chancellor’s Building, University of Edinburgh, 49 Little France Crescent, Scotland EH16 4SB, London, UK
  • Search for other articles:
  • degruyter.comGoogle Scholar
, H. Rees, A.S.C. Rice, K. Rutten
  • Grünenthal GmbH, Global Biomedical Sciences, Department of Pharmacology Pain, Aachen, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar
, M. Segerdahl
  • Grünenthal GmbH, Global Biomedical Sciences, Department of Pharmacology Pain, Aachen, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar
, J. Serra
  • Neuroscience Technologies, Baldiri Reixac, 15-21, Parc Cientifique de Barcelona, 08028, Barcelona, Spain
  • Search for other articles:
  • degruyter.comGoogle Scholar
, R. Wodarski
  • Neuroscience Research Division, Lilly Research Centre, Eli Lilly & Co. Ltd, Erl Wood Manor, Windlesham, Surrey GU20 6PH, UK
  • Search for other articles:
  • degruyter.comGoogle Scholar
, O.-G. Berge and R.-D. Treede
  • Center for Biomedicine and Medical Technology Mannheim, Medical Faculty Mannheim, Heidelberg University, Berlin, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

Background and aims

Pain is a subjective experience, and as such, pre-clinical models of human pain are highly simplified representations of clinical features. These models are nevertheless critical for the delivery of novel analgesics for human pain, providing pharmacodynamic measurements of activity and, where possible, on-target confirmation of that activity. It has, however, been suggested that at least 50% of all pre-clinical data, independent of discipline, cannot be replicated. Additionally, the paucity of “negative” data in the public domain indicates a publication bias, and significantly impacts the interpretation of failed attempts to replicate published findings. Evidence suggests that systematic biases in experimental design and conduct and insufficiencies in reporting play significant roles in poor reproducibility across pre-clinical studies. It then follows that recommendations on how to improve these factors are warranted.

Methods

Members of Europain, a pain research consortium funded by the European Innovative Medicines Initiative (IMI), developed internal recommendations on how to improve the reliability of pre-clinical studies between laboratories. This guidance is focused on two aspects: experimental design and conduct, and study reporting.

Results

Minimum requirements for experimental design and conduct were agreed upon across the dimensions of animal characteristics, sample size calculations, inclusion and exclusion criteria, random allocation to groups, allocation concealment, and blinded assessment of outcome. Building upon the Animals in Research: Reportingin vivo Experiments (ARRIVE) guidelines, reporting standards were developed for pre-clinical studies of pain. These include specific recommendations for reporting on ethical issues, experimental design and conduct, and data analysis and interpretation. Key principles such as sample size calculation, a priori definition of a primary efficacy measure, randomization, allocation concealments, and blinding are discussed. In addition, considerations of how stress and normal rodent physiology impact outcome of analgesic drug studies are considered. Flow diagrams are standard requirements in all clinical trials, and flow diagrams for preclinical trials, which describe number of animals included/excluded, and reasons for exclusion are proposed. Creation of a trial registry for pre-clinical studies focused on drug development in order to estimate possible publication bias is discussed.

Conclusions

More systematic research is needed to analyze how inadequate internal validity and/or experimental bias may impact reproducibility across pre-clinical pain studies. Addressing the potential threats to internal validity and the sources of experimental biases, as well as increasing the transparency in reporting, are likely to improve preclinical research broadly by ensuring relevant progress is made in advancing the knowledge of chronic pain pathophysiology and identifying novel analgesics.

Implications

We are now disseminating these Europain processes for discussion in the wider pain research community. Any benefit from these guidelines will be dependent on acceptance and disciplined implementation across pre-clinical laboratories, funding agencies and journal editors, but it is anticipated that these guidelines will be a first step towards improving scientific rigor across the field of pre-clinical pain research.

  • [1]

    International Association for the Study of Pain. Available at: http://www.iasp-pain.org

  • [2]

    Jensen TS, Baron R, Haanpää M, Kalso E, Loeser JD, Rice AS, Treede R-D. A new definition of neuropathic pain. Pain 2010;152:2204–5.

  • [3]

    Maier C, Baron R, Tölle TR, Binder A, Birbaumer N, Birklein F, Gierthmühlen J, Flor H, Geber C, Huge V, Krumova EK, Landwehrmeyer GB, Magerl W, Maihöfner C, Richter H, Rolke R, Scherens A, Schwarz A, Sommer C, Tronnier V, Uçeyler N, Valet M, Wasner G, Treede RD. Quantitative sensory testing in the German Research Network on Neuropathic Pain (DFNS): somatosensory abnormalities in 1236 patients with different neuropathic pain syndromes. Pain 2010;150:439–50.

    • Crossref
    • PubMed
    • Export Citation
  • [4]

    LoeserJD, Treede R-D. The Kyoto protocol of IASP basic pain terminology. Pain 2008;137:473–7.

    • Crossref
    • PubMed
    • Export Citation
  • [5]

    Whiteside GT, Adedoyin A, Leventhal L. Predictive validity of animal pain models? Acomparison of the pharmacokineticepharmacodynamic relationship for pain drugs in rats and humans. Neuropharmacology 2008;54:767–75.

    • Crossref
    • PubMed
    • Export Citation
  • [6]

    Berge O-G. Predictive validity of behavioural animal model for chronic pain. Br J Pharmacol 2011;164:1195–206.

    • Crossref
    • PubMed
    • Export Citation
  • [7]

    Moore A, Derry S, Eccleston C, Kalso E. Expect analgesic failure; pursue analgesic success. BMJ 2013;346:1–6.

  • [8]

    Mogil JS, Davis KD, Derbyshire SW. The necessity of animal models in pain research. Pain 2010;151:12–7.

    • Crossref
    • PubMed
    • Export Citation
  • [9]

    Begley CG, Ellis LM. Raise standards for pre-clinical cancer research. Nature 2012;483:531–3.

    • Crossref
    • Export Citation
  • [10]

    Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 2011;10:712.

    • Crossref
    • PubMed
    • Export Citation
  • [11]

    Macleod M. Why animal research needs to improve. Nature 2011;477:511.

  • [12]

    Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Hutton J, Altman DG. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS ONE 2009;4:e7824.

    • Crossref
    • PubMed
    • Export Citation
  • [13]

    Moher D, Simera I, Schulz KF, Hoey J, Altman DG. Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. BMC Med 2008;6:13.

    • Crossref
    • PubMed
    • Export Citation
  • [14]

    Henderson VC, Kimmelman J, Fergusson D, Grimshaw JM, Hackam DG. Threats to validity in the design and conduct of preclinical efficacy studies: a systematic review of guidelines for in vivo animal experiment. PLoS Med 2013;10: 1–14.

  • [15]

    Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, Finkelstein R, Fisher M, Gendelman HE, Golub RM, Goudreau JL, Gross RA, Gubitz AK, Hesterlee SE, Howells DW, Huguenrard J, Kelner K, Koroshetz W, Krinc D, Lazic SE, Levine MS, Macleod MR, McCall J M, Moxley 3rd RT, Narasimihan K, Noble LJ, Perrin S, Porter JD, Steward O, Unger E, UTz U, Silberberg SD. A call for transparent reporting to optimize the predictive value of pre-clinical research. Nature 2012;490:187–91.

    • Crossref
    • Export Citation
  • [16]

    Sena E, Whelbe P, Sandercock P, Macleod M. Systematic review and meta-analysis of the efficacy of tirilazad in experimental stroke. Stroke 2007;38:388–94.

    • Crossref
    • PubMed
    • Export Citation
  • [17]

    Gregory NS, Harris AL, Robinson CR, Dougherty PM, Fuchs PN, Sluka KA. An overview of animal models of pain: disease models and outcome measures. J Pain 2013;14:1255–69.

    • Crossref
    • PubMed
    • Export Citation
  • [18]

    Le Bars D, Hansson PT, Plaghki L. Current animal tests and models of pain. In: Beaulieu P, Lussier D, Porreca F, Dickenson AH, editors. Pharmacology of pain. Seattle: IASP Press; 2009. p. 1–30.

  • [19]

    Percie du Sert N, Rice AS. Improving the translation of analgesic drugs to the clinic: animal models of neuropathic pain. Br J Pharmacol 2014;171: 2951–63.

    • Crossref
    • PubMed
    • Export Citation
  • [20]

    Crossley NA, Sena E, Goehler J, Horn J, van derWorp B, Bath PMW. Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. Stroke 2008;39:929–34.

    • Crossref
    • Export Citation
  • [21]

    Steward O, Popovich PG, Dietric WD, Kleitman N. Replication and reproducibility in spinal cord injury research. Exp Neurol 2012;233:597–605.

    • Crossref
    • PubMed
    • Export Citation
  • [22]

    Kontinen VK, Meert TF. Predictive validity of neuropathic pain models in pharmacological studies with a behavioral outcome in the rat: a systematic review. In: Proceedings of the 10th World Congress on Pain (24th edition). Seattle: IASP Press; 2003. p. 489–98.

  • [23]

    Lindner MD, Frydel BR, Francis JM, Cain CK. Analgesic effects of adrenal chromaffin allografts: contingent on special procedures or due to experimenter bias?J Pain 2003;4:64–73.

    • Crossref
    • PubMed
    • Export Citation
  • [24]

    Eisenach JC, Lindner MD. Did experimenter bias conceal the efficacy of spinal opioids in previous studies with the spinal nerve ligation model of neuropathic pain? Anesthesiology 2004;100:765–7.

    • Crossref
    • Export Citation
  • [25]

    Rice ASC, Cimino-Brown D, Eisenach JC, Kontinen VK, Lacroix-Fralish ML, Machin I, Preclinical Pain Consortium, Mogil JS, Stöhr T. Animal models and the prediction of efficacy in clinical trials of analgesic drugs: a critical appraisal and call for uniform reporting standards. Pain 2008;139:243–7.

    • Crossref
    • PubMed
    • Export Citation
  • [26]

    Rice ASC. Predicting analgesic efficacy from animal models of peripheral neuropathy and nerve injury: a critical view from the clinic. In: Mogil JS, editor. Pain 2010 - an updated review: refresher course syllabus. Seattle: IASP Press; 2010. p. 415–26.

  • [27]

    Currie GL, Delaney A, Bennett MI, Dickenson AH, Egan KJ, Vesterinen HM, Sena ES, Macleod MR, Colvin LA, Fallon MT. Animal models of bone cancer pain: systematic review and meta-analyses. Pain 2013;154:917–26.

    • Crossref
    • PubMed
    • Export Citation
  • [28]

    Sena ES, van der Worp HB, Bath PMW, Howells DW, Macleod MR. Publication bias inreports ofanimal stroke studies leads to major over statement of efficacy. PLoS Biol 2010;8:e1000344.

    • Crossref
    • Export Citation
  • [29]

    Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF. Improving the quality of reporting of randomized controlled trials. The CONSORT statement.JAMA 1996;276:637–9.

  • [30]

    Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191–4.

    • Crossref
    • Export Citation
  • [31]

    Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med 2001;134:657–62.

    • Crossref
    • Export Citation
  • [32]

    MoherD, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987–91.

    • Crossref
    • Export Citation
  • [33]

    Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med 2010;8:18.

    • Crossref
    • PubMed
    • Export Citation
  • [34]

    Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 2010;8:e1000412.

    • Crossref
    • PubMed
    • Export Citation
  • [35]

    Altman DG. Endorsement of the CONSORT statement by high impact medical journals: surveyofinstructions forauthors. BMJ 2005;330:1056–7.

    • Crossref
    • Export Citation
  • [36]

    Hopewell S, Altman DG, Moher D, Schulz KF. Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journal ‘instructions to Authors’. Trials 2008;9:20.

    • Crossref
    • Export Citation
  • [37]

    Danos O, Davies K, Lehn P, Mulligan R. The ARRIVE guidelines, a welcome improvement to standards for reporting animal research. J Gene Med 2010;12:559–60.

    • Crossref
    • PubMed
    • Export Citation
  • [38]

    Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA 2001;285:1992–5.

    • Crossref
    • Export Citation
  • [39]

    Plint AC, Moher D, Schulz K, Altman DG, Morrison A. Does the CONSORT check list improve the quality of reports of randomized controlled trials?A systematic review. In: Fifth International Congress of Peer Review and Biomedical Publication. 2005.

  • [40]

    Nature Medicine Editorial. Raising standards. Nat Med 2013;19:508.

  • [41]

    Macleod MR, Fisher M, O’Collins V, Sena ES, Dirnagl U, Bath PM, Buchan A, van der Worp HB, Traystman R, Minematsu K, Donnan GA, Howells DW. Good laboratory practice. preventing introduction of bias at the bench. Stroke 2009;40:e50–2.

    • PubMed
    • Export Citation
  • [42]

    Sena E, van der Worp HB, Howells D, Macleod M. How can we improve the pre-clinical development ofdrugs forstroke? Trends Neurosci 2007;30:433–9.

    • Crossref
    • Export Citation
  • [43]

    van der Worp HB, Howells DW, Sena ES, Porritt MJ, Rewell S, O’Collins V, Macleod MR. Can animal models of disease reliably inform human studies? PLoS Med 2010;7:e1000245.

    • Crossref
    • PubMed
    • Export Citation
  • [44]

    Organisation for the Economic Co-operation and Development. Available at: http://www.oecd.org/env/chemicalsafetyandbiosafety/testingofchemicals/linkstonationalwebsitesongoodlaboratorypractice.htm

  • [45]

    MogilJS, BaileyAL. Sexandgenderdifferences in pain and analgesia. Prog Brain Res 2010;186:141–57.

    • PubMed
    • Export Citation
  • [46]

    Fillingim RB, Gear RW. Sex differences in opioid analgesia: clinical and experimental findings. Eur J Pain 2004;8:413–25.

    • Crossref
    • PubMed
    • Export Citation
  • [47]

    Butler RK, Finn DP. Stress-induced analgesia. Prog Neurobiol 2009;88:184–202.

    • Crossref
    • PubMed
    • Export Citation
  • [48]

    Hart PC, Bergner CL, SmolinskyAN, Dufour BD, Egan RJ, Laporte JL, KalueffAV. Experimental models of anxiety for drug discovery and brain research. Methods Mol Biol 2010;602:299–321.

    • Crossref
    • Export Citation
  • [49]

    Carrive P, Churyukanov M, LeBars D. Areassessment of stress-induced analgesia in the rat using an unbiased method. Pain 2011;152:676–86.

    • Crossref
    • PubMed
    • Export Citation
  • [50]

    Animal Welfare Information Center: Pain and distress references. Available at: http://awic.nal.usda.gov/naldisplay/index.php?info_center=3&taxJevel=1&tax_subject=310

  • [51]

    National Research Council. Recognition and alleviation of pain in laboratory animals. Washington: National Academies Press; 2009.

  • [52]

    Miaoulis G, Michener RD. An introduction to sampling. Dubuque, IA: Kendall/Hunt Publishing Company; 1976.

  • [53]

    EFSA Scientific Committee. Statistical significance and biological relevance. EFSA J 2011;9:2372.

  • [54]

    National Centre for the Replacement Refinement & Reduction of Animals in Research. Available at http://www.nc3rs.org.uk/

  • [55]

    Meller ST, Dykstra C, Pechman PS, Maves TJ, Gebhart GF. Ethanol dose-dependently attenuates NMDA-mediated thermal hyperalgesia in the rat. Neurosci Lett 1993;154:137–40.

    • Crossref
    • PubMed
    • Export Citation
  • [56]

    Colucci M, Maione F, Bonito MC, Piscopo A, Di Giannuario A, Pieretti S. New insights of dimethyl sulphoxide effects (DMSO) on experimental in vivo models of nociception and inflammation. Pharmacol Res 2008;57:419–25.

    • Crossref
    • PubMed
    • Export Citation
  • [57]

    Morton DB, Jennings M, Buckwell A, Ewbank R, Godfrey C, Holgate B, Inglis I, James R, Page C, Sharman I, Verschoyle R, Westall L, Wilson AB, Joint Working Group on Refinement. Refining procedures for the administration of substances: report of the BVAAWF/FRAME/RSPCA/UFAW Joint Working Group on Refinement. Lab Anim 2000;35:1–41.

  • [58]

    Ellison SLR, Barwick V, Farrant TJ. Practicalstatistics forthe analytical scientist: a bench guide. 2nd ed. RSC Publishing; 2009.

  • [59]

    Efron B. Forcing a sequential experiment to be balanced. Biometrika 1971;58:403.

    • Crossref
    • Export Citation
  • [60]

    OlejnikSF. Variance heterogeneity: an outcome to explain or a nuisance factor to control. J Exp Educ 1988;56:193–7.

    • Crossref
    • Export Citation
  • [61]

    Forder PM, Gebski VJ, Keech AC. Allocation concealment and blinding: when ignorance is bliss. Med J Aust 2005;182:87–9.

    • Crossref
    • PubMed
    • Export Citation
  • [62]

    Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet 2002;359:614–8.

    • Crossref
    • PubMed
    • Export Citation
  • [63]

    Meert TF, Vermeirsch HA. A pre-clinical comparison between different opioids: antinociceptive versus adverse effect. Pharmacol Biochem Behav 2005;80:309–26.

    • Crossref
    • Export Citation
  • [64]

    Vonsy JL, Ghandehari J, Dickenson AH. Differential analgesic effects of morphine and gabapentin on behavioural measures of pain and disability in a model of osteoarthritis pain in rats. Eur J Pain 2009;13:786–93.

    • Crossref
    • Export Citation
  • [65]

    Zimmermann M. Ethical guidelines for investigations of experimental pain in conscious animals. Pain 1983;16:109–10.

    • Crossref
    • PubMed
    • Export Citation
  • [66]

    Finnerup NB, Sindrup SH, Jensen TS. The evidence for pharmacological treatment of neuropathic pain. Pain 2010;150:573–81.

    • Crossref
    • PubMed
    • Export Citation
  • [67]

    Clinical Trials.org. Available at: http://clinicaltrials.gov/

  • [68]

    Crabbe JC, Wahlsten D, Dudek BC. Genetics of mouse behavior: interactions with laboratory environment. Science 1999;284:1670–2.

    • Crossref
    • PubMed
    • Export Citation
Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Search