Gould, ElliotElliotGouldFraser, Hannah SHannah SFraserParker, Timothy HTimothy HParkerNakagawa, ShinichiShinichiNakagawaGriffith, Simon CSimon CGriffithVesk, Peter APeter AVeskFidler, FionaFionaFidlerHamilton, Daniel GDaniel GHamiltonAbbey-Lee, Robin NRobin NAbbey-LeeAbbott, Jessica KJessica KAbbottAguirre, Luis ALuis AAguirreAlcaraz, CarlesCarlesAlcarazAloni, IrithIrithAloniAltschul, DrewDrewAltschulArekar, KunalKunalArekarAtkins, Jeff WJeff WAtkinsAtkinson, JoeJoeAtkinsonBaker, Christopher MChristopher MBakerBarrett, MeghanMeghanBarrettBell, KristianKristianBellBello, Suleiman KehindeSuleiman KehindeBelloBeltrán, IvánIvánBeltránBerauer, Bernd JBernd JBerauerBertram, Michael GrantMichael GrantBertramBillman, Peter DPeter DBillmanBlake, Charlie KCharlie KBlakeBlake, ShannonShannonBlakeBliard, LouisLouisBliardBonisoli-Alquati, AndreaAndreaBonisoli-AlquatiBonnet, TimothéeTimothéeBonnetBordes, Camille Nina MarionCamille Nina MarionBordesBose, Aneesh P HAneesh P HBoseBotterill-James, ThomasThomasBotterill-JamesBoyd, Melissa AnnaMelissa AnnaBoydBoyle, Sarah ASarah ABoyleBradfer-Lawrence, TomTomBradfer-LawrenceBradham, JenniferJenniferBradhamBrand, Jack AJack ABrandBrengdahl, Martin IMartin IBrengdahlBulla, MartinMartinBullaBussière, LucLucBussièreCamerlenghi, EttoreEttoreCamerlenghiCampbell, Sara ESara ECampbellCampos, Leonardo L FLeonardo L FCamposCaravaggi, AnthonyAnthonyCaravaggiCardoso, PedroPedroCardosoCarroll, Charles J WCharles J WCarrollCatanach, Therese ATherese ACatanachChen, XuanXuanChenChik, Heung Ying JanetHeung Ying JanetChikChoy, Emily SarahEmily SarahChoyChristie, Alec PhilipAlec PhilipChristieChuang, AngelaAngelaChuangChunco, Amanda JAmanda JChuncoClark, Bethany LBethany LClarkContina, AndreaAndreaContinaCovernton, Garth AGarth ACoverntonCox, Murray PMurray PCoxCressman, Kimberly AKimberly ACressmanCrotti, MarcoMarcoCrottiCrouch, Connor DavidsonConnor DavidsonCrouchD'Amelio, Pietro BPietro BD'Ameliode Sousa, Alexandra AllisonAlexandra Allisonde SousaDöbert, Timm FabianTimm FabianDöbertDobler, RalphRalphDoblerDobson, Adam JAdam JDobsonDoherty, Tim STim SDohertyDrobniak, Szymon MarianSzymon MarianDrobniakDuffy, Alexandra GraceAlexandra GraceDuffyDuncan, Alison BAlison BDuncanDunn, Robert PRobert PDunnDunning, JamieJamieDunningDutta, TrishnaTrishnaDuttaEberhart-Hertel, LukeLukeEberhart-HertelElmore, Jared AlanJared AlanElmoreElsherif, Mahmoud MedhatMahmoud MedhatElsherifEnglish, Holly MHolly MEnglishEnsminger, David CDavid CEnsmingerErnst, Ulrich RainerUlrich RainerErnstFerguson, Stephen MStephen MFergusonFernandez-Juricic, EstebanEstebanFernandez-JuricicFerreira-Arruda, ThalitaThalitaFerreira-ArrudaFieberg, JohnJohnFiebergFinch, Elizabeth AElizabeth AFinchFiorenza, Evan AEvan AFiorenzaFisher, David NDavid NFisherFontaine, AmélieAmélieFontaineForstmeier, WolfgangWolfgangForstmeierFourcade, YoanYoanFourcadeFrank, Graham SGraham SFrankFreund, Cathryn ACathryn AFreundFuentes-Lillo, EduardoEduardoFuentes-LilloGandy, Sara LSara LGandyGannon, Dustin GDustin GGannonGarcía-Cervigón, Ana IAna IGarcía-CervigónGarretson, Alexis CAlexis CGarretsonGe, XuezhenXuezhenGeGeary, William LWilliam LGearyGéron, CharlyCharlyGéronGilles, MarcMarcGillesGirndt, AntjeAntjeGirndtGliksman, DanielDanielGliksmanGoldspiel, Harrison BHarrison BGoldspielGomes, Dylan G EDylan G EGomesGood, Megan KateMegan KateGoodGoslee, Sarah CSarah CGosleeGosnell, J StephenJ StephenGosnellGrames, Eliza MEliza MGramesGratton, PaoloPaoloGrattonGrebe, Nicholas MNicholas MGrebeGreenler, Skye MSkye MGreenlerGriffioen, MaaikeMaaikeGriffioenGriffith, Daniel MDaniel MGriffithGriffith, Frances JFrances JGriffithGrossman, Jake JJake JGrossmanGüncan, AliAliGüncanHaesen, StefStefHaesenHagan, James GJames GHaganHager, Heather AHeather AHagerHarris, Jonathan PhiloJonathan PhiloHarrisHarrison, Natasha DeanNatasha DeanHarrisonHasnain, Sarah SyediaSarah SyediaHasnainHavird, Justin ChaseJustin ChaseHavirdHeaton, Andrew JAndrew JHeatonHerrera-Chaustre, María LauraMaría LauraHerrera-ChaustreHoward, Tanner JTanner JHowardHsu, Bin-YanBin-YanHsuIannarilli, FabiolaFabiolaIannarilliIranzo, Esperanza CEsperanza CIranzoIverson, Erik N KErik N KIversonJimoh, Saheed OlaideSaheed OlaideJimohJohnson, Douglas HDouglas HJohnsonJohnsson, MartinMartinJohnssonJorna, JesseJesseJornaJucker, TommasoTommasoJuckerJung, MartinMartinJungKačergytė, InetaInetaKačergytėKaltz, OliverOliverKaltzKe, AlisonAlisonKeKelly, Clint DClint DKellyKeogan, KatharineKatharineKeoganKeppeler, Friedrich WolfgangFriedrich WolfgangKeppelerKillion, Alexander KAlexander KKillionKim, DongminDongminKimKochan, David PDavid PKochanKorsten, PeterPeterKorstenKothari, ShanShanKothariKuppler, JonasJonasKupplerKusch, Jillian MJillian MKuschLagisz, MalgorzataMalgorzataLagiszLalla, Kristen MarianneKristen MarianneLallaLarkin, Daniel JDaniel JLarkinLarson, Courtney LCourtney LLarsonLauck, Katherine SKatherine SLauckLauterbur, M EliseM EliseLauterburLaw, AlanAlanLawLéandri-Breton, Don-JeanDon-JeanLéandri-BretonLembrechts, Jonas JJonas JLembrechtsL'Herpiniere, KiaraKiaraL'HerpiniereLievens, Eva J PEva J PLievensde Lima, Daniela OliveiraDaniela Oliveirade LimaLindsay, ShaneShaneLindsayLuquet, MartinMartinLuquetMacLeod, RossRossMacLeodMacphie, Kirsty HKirsty HMacphieMagellan, KitKitMagellanMair, Magdalena MMagdalena MMairMalm, Lisa ELisa EMalmMammola, StefanoStefanoMammolaMandeville, Caitlin PCaitlin PMandevilleManhart, MichaelMichaelManhartManrique-Garzon, Laura MilenaLaura MilenaManrique-GarzonMäntylä, ElinaElinaMäntyläMarchand, PhilippePhilippeMarchandMarshall, Benjamin MichaelBenjamin MichaelMarshallMartin, Charles ACharles AMartinMartin, Dominic AndreasDominic AndreasMartin0000-0001-7197-2278Martin, Jake MitchellJake MitchellMartinMartinig, April RobinApril RobinMartinigMcCallum, Erin SErin SMcCallumMcCauley, MarkMarkMcCauleyMcNew, Sabrina MSabrina MMcNewMeiners, Scott JScott JMeinersMerkling, ThomasThomasMerklingMichelangeli, MarcusMarcusMichelangeliMoiron, MariaMariaMoironMoreira, BrunoBrunoMoreiraMortensen, JenniferJenniferMortensenMos, BenjaminBenjaminMosMuraina, Taofeek OlatunbosunTaofeek OlatunbosunMurainaMurphy, Penelope WrennPenelope WrennMurphyNelli, LucaLucaNelliNiemelä, PetriPetriNiemeläNightingale, JoshJoshNightingaleNilsonne, GustavGustavNilsonneNolazco, SergioSergioNolazcoNooten, Sabine SSabine SNootenNovotny, Jessie LantermanJessie LantermanNovotnyOlin, Agnes BirgittaAgnes BirgittaOlinOrgan, Chris LChris LOrganOstevik, Kate LKate LOstevikPalacio, Facundo XavierFacundo XavierPalacioPaquet, MatthieuMatthieuPaquetParker, Darren JamesDarren JamesParkerPascall, David JDavid JPascallPasquarella, Valerie JValerie JPasquarellaPaterson, John HaroldJohn HaroldPatersonPayo-Payo, AnaAnaPayo-PayoPedersen, Karen MarieKaren MariePedersenPerez, GrégoireGrégoirePerezPerry, Kayla IKayla IPerryPottier, PatricePatricePottierProulx, Michael JMichael JProulxProulx, RaphaëlRaphaëlProulxPruett, Jessica LJessica LPruettRamananjato, VeronarindraVeronarindraRamananjatoRandimbiarison, Finaritra TolotraFinaritra TolotraRandimbiarisonRazafindratsima, Onja HOnja HRazafindratsimaRennison, Diana JDiana JRennisonRiva, FedericoFedericoRivaRiyahi, SepandSepandRiyahiRoast, Michael JamesMichael JamesRoastRocha, Felipe PereiraFelipe PereiraRochaRoche, Dominique GDominique GRocheRomán-Palacios, CristianCristianRomán-PalaciosRosenberg, Michael SMichael SRosenbergRoss, JessicaJessicaRossRowland, Freya EFreya ERowlandRugemalila, DeusdedithDeusdedithRugemalilaRussell, Avery LAvery LRussellRuuskanen, SuviSuviRuuskanenSaccone, PatrickPatrickSacconeSadeh, AsafAsafSadehSalazar, Stephen MStephen MSalazarSales, KrisKrisSalesSalmón, PabloPabloSalmónSánchez-Tójar, AlfredoAlfredoSánchez-TójarSantos, Leticia PereiraLeticia PereiraSantosSantostefano, FrancescaFrancescaSantostefanoSchilling, Hayden THayden TSchillingSchmidt, MarcusMarcusSchmidtSchmoll, TimTimSchmollSchneider, Adam CAdam CSchneiderSchrock, Allie EAllie ESchrockSchroeder, JuliaJuliaSchroederSchtickzelle, NicolasNicolasSchtickzelleSchultz, Nick LNick LSchultzScott, Drew ADrew AScottScroggie, Michael PeterMichael PeterScroggieShapiro, Julie TeresaJulie TeresaShapiroSharma, NitikaNitikaSharmaShearer, Caroline LCaroline LShearerSimón, DiegoDiegoSimónSitvarin, Michael IMichael ISitvarinSkupien, Fabrício LuizFabrício LuizSkupienSlinn, Heather LeaHeather LeaSlinnSmith, Grania PollyGrania PollySmithSmith, Jeremy AJeremy ASmithSollmann, RahelRahelSollmannWhitney, Kaitlin StackKaitlin StackWhitneyStill, Shannon MichaelShannon MichaelStillStuber, Erica FErica FStuberSutton, Guy FGuy FSuttonSwallow, BenBenSwallowTaff, Conor ClaverieConor ClaverieTaffTakola, ElinaElinaTakolaTanentzap, Andrew JAndrew JTanentzapTarjuelo, RocíoRocíoTarjueloTelford, Richard JRichard JTelfordThawley, Christopher JChristopher JThawleyThierry, HugoHugoThierryThomson, JacquelineJacquelineThomsonTidau, SvenjaSvenjaTidauTompkins, Emily MEmily MTompkinsTortorelli, Claire MarieClaire MarieTortorelliTrlica, AndrewAndrewTrlicaTurnell, Biz RBiz RTurnellUrban, LaraLaraUrbanVan de Vondel, StijnStijnVan de Vondelvan der Wal, Jessica Eva MeganJessica Eva Meganvan der WalVan Eeckhoven, JensJensVan Eeckhovenvan Oordt, FrancisFrancisvan OordtVanderwel, K MichelleK MichelleVanderwelVanderwel, Mark CMark CVanderwelVanderwolf, Karen JKaren JVanderwolfVélez, JulianaJulianaVélezVergara-Florez, Diana CarolinaDiana CarolinaVergara-FlorezVerrelli, Brian CBrian CVerrelliVieira, Marcus ViníciusMarcus ViníciusVieiraVillamil, NoraNoraVillamilVitali, ValerioValerioVitaliVollering, JulienJulienVolleringWalker, JeffreyJeffreyWalkerWalker, Xanthe JXanthe JWalkerWalter, Jonathan AJonathan AWalterWaryszak, PawelPawelWaryszakWeaver, Ryan JRyan JWeaverWedegärtner, Ronja E MRonja E MWedegärtnerWeller, Daniel LDaniel LWellerWhelan, ShannonShannonWhelanWhite, Rachel LouiseRachel LouiseWhiteWolfson, David WilliamDavid WilliamWolfsonWood, AndrewAndrewWoodYanco, Scott WScott WYancoYen, Jian D LJian D LYenYoungflesh, CaseyCaseyYoungfleshZilio, GiacomoGiacomoZilioZimmer, CédricCédricZimmerZimmerman, Gregory MarkGregory MarkZimmermanZitomer, Rachel ARachel AZitomer2025-02-262025-02-262025-02-06https://boris-portal.unibe.ch/handle/20.500.12422/205094Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small "many analyst" study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future.enAnalytical heterogeneityMany-analystMetascienceReplication crisisReproducibility500 - Science::580 - Plants (Botany)500 - Science::570 - Life sciences; biologyarticle10.48620/855453991577110.1186/s12915-024-02101-x