Kurate by Kivira
Research quality assurance at portfolio scaleWellcome already judges the rigor a project proposes. Now it can measure the rigor it delivered.
Wellcome selects the best proposals through expert peer review and, having embedded DORA, judges research on its content rather than on journal metrics. Methodological rigor — whether a funded study used a valid comparator, held to its prespecified endpoints, and drew conclusions its design could support — is a distinct axis, and one no funder has been able to measure at scale, because appraising it took expert-hours per study. When Kurate graded a corpus of trials drawn from higher-quality domains, the spread across that axis was wide.
Even in a sample drawn from higher-quality domains, grades span the full range — the same wide spread found across the published literature generally, not a property of any one funder's choices. What's new isn't the spread; it's that it's now measurable. Once it is, a funder can see which work clears the bar for confident interpretation (A–C) and direct more of the portfolio toward it.
Kurate's premise doesn't rest on our numbers alone. A large, independent metascience literature has documented that questionable research practices are common — outcomes switched between registration and publication, selective reporting, underpowered designs, p-hacking and HARKing — leaving a public record in which a study can look authoritative and still not support its own conclusions. Public and philanthropic funding flows into that record. Kurate turns decades of this evidence into a check that runs at portfolio scale.
Kurate is built by researchers who publish in statistics, research methodology and metascience — including in Nature, NeurIPS and ICLR. Members of the team have separately built the peer-reviewed precursors to Kurate's core method — automated comparison of trial registrations against published results. That background shapes how the tool handles fragile claims, uncertain comparisons and noisy literature.
Research in causal inference, deep generative modelling and multimodal ML; 50+ peer-reviewed publications including ICLR and NeurIPS.
Two doctorates — Engineering (Vision, Speech & Signal Processing), University of Surrey; and Applied Mathematics for the Human & Social Sciences, University of Lausanne. Affiliations: University of Lausanne, University of Surrey, and the Sense Center for Innovation and Research.
Contributes to Kurate's evidence-evaluation methodology and clinical rubric development; 50+ peer-reviewed publications, including three metascience papers in Nature.
Expertise in research-integrity assessment and LLM-workflow evaluation; author of published registration-versus-report comparison tools.
How Kurate works
Kurate reconstructs each study's intended design from its own paper trail, then measures the published report against it — the work a methodologist and a statistician would do together, applied uniformly at scale. Two documents in; one auditable verdict out.
One rubric, applied identically to every study — reproducible, auditable, and fast enough to run across a whole portfolio.
Where Kurate fits
Wellcome already scores rigor at review, judges output on merit under DORA, and requires every trial it funds to be registered and reported. The one question none of these answers: did the funded work deliver the rigor it proposed? Kurate is built for exactly that gap — complementing the mechanisms already in place, not replacing them.
Is the proposed science rigorous?
Wellcome's expert peer review assesses the scientific quality and feasibility of each proposal, and — having embedded DORA in its evaluation — judges research on its content, not journal metrics. Assessed on the proposal, before any data exists.
Is the work registered and reported?
Wellcome's Clinical Trials Policy requires every funded trial to be registered, with its protocol and analysis plan published and linked to the registry, and results reported — and Wellcome monitors compliance every year. Availability, not verified method.
Did the funded work deliver the rigor it proposed?
No portfolio-scale measure closes this loop today. Kurate compares each published study against its own pre-registration, across the whole portfolio — the delivered-rigor counterpart to the rigor Wellcome scores at review, and DORA-aligned by construction: it reads method, not metrics.
Wellcome's Clinical Trials Policy already requires the protocol and analysis plan to be published and linked to the trial registry — so the pre-specification is on file, and compliance is monitored annually. What no one checks at scale is whether the published report matches it. That comparison is exactly what Kurate performs — automatically, across the whole portfolio.
What it lets Wellcome do
Across a research portfolio of about £1.6 billion a year (£16 billion committed to 2032), spanning thousands of awards and researchers, Kurate grades every study on one rubric in a single pass, turning rigor from something you could only sample by hand into a portfolio-wide layer alongside the evaluation Wellcome already runs. What that puts within reach:
See which schemes and calls actually deliver rigor
Rank funding schemes, programmes and calls by the methodological quality of the work they produce — so Wellcome can direct more funding toward the ones that consistently deliver, and give the ones that lag the evidence to sharpen their guidance or evaluation criteria. The same lens resolves to the researcher level where that's useful — surfacing where rigor support would help most, not a blacklist. Grounded in method, not reputation, and defensible because every grade is auditable.
Track portfolio quality over time
Watch methodological quality across a funding scheme, a discipline, or a decade — and test whether a given call raises or lowers the rigor of the work it produces.
Strengthen the schemes that need it
See which funding schemes and evaluation panels produce the most rigorous work — and where adjusted guidance would raise quality the most.
Report defensible rigor metrics
Give Wellcome's leadership, its Board of Governors, and the public a reproducible measure of research quality that complements existing evaluation instead of competing with it.
See a field's quality evolve
Track whether rigor in a domain is improving or eroding over years — the kind of question that shapes where a new call or initiative should be aimed.
How to read the rigor
- Registration-anchored. Quality is judged against the study's own pre-data-collection commitments, not a reviewer's taste.
- GRADE-like, without pool contamination. Comparable in spirit to a Cochrane/GRADE appraisal, but dealbreaker studies are excluded rather than downweighted — so one flawed study can't quietly dilute a synthesis.
- Cross-domain in one pass. Clinical, methodological and statistical checks applied uniformly — precisely where human review is scarce, slow, and inconsistent.
- Reproducible & auditable. The same rubric, applied the same way, every time — a property no distributed panel of reviewers can guarantee, and the basis for defensible portfolio metrics.
- Scope-honest. Strongest in medication, treatment and RCT-relevant domains, where the registration-versus-report comparison is most decisive.
See Kurate for yourself
The figures in this brief are illustrative. To see the real thing, ask us for a demo — we'll walk you through Kurate on live published research: the methodological grade, the dealbreaker flags, and the detailed analysis behind each grade.
About the figures. The grade distribution is Kurate's own result on a corpus drawn from higher-quality research domains; it is presented as a floor, not a random portfolio sample, and does not yet establish how Wellcome-funded output is distributed. Wellcome's research spend (£16 billion committed for the decade to 2032, ~£1.6 billion a year) is from Wellcome reporting. Wellcome mechanisms referenced — expert peer-review evaluation, the adoption of DORA / responsible research assessment, the Open Access policy, and the Clinical Trials Policy requiring registration, protocol and analysis-plan publication and results reporting with annual compliance monitoring — are per Wellcome public documentation. The savings floor scales the ~£1.6 billion base by an illustratively small share and is a lower-bound sketch, not a projection. Kurate is decision-support for research quality assessment and does not replace clinical, regulatory, or institutional review. Kurate is built by Kivira Health — kivira.health. Kurate is built by Kivira Health — kivira.health.
Selected related publications
Methodological and meta-scientific work associated with the Kurate team.
- Vowels, M. J. (2023). Misspecification and unreliable interpretations in psychology and social science. Psychological Methods, 28(3), 507–526. DOI
- Vowels, M. J., Vowels, L. M., & Wood, N. D. (2023). Spectral and cross-spectral analysis: A tutorial for psychologists and social scientists. Psychological Methods, 28(3), 631–650. DOI
- Vowels, M. J. (2023). Prespecification of structure for the optimization of data collection and analysis. Collabra: Psychology, 9(1), Article 71300. DOI
- Vowels, M. J. (2024). Trying to outrun causality with machine learning: Limitations of model explainability techniques for exploratory research. Psychological Methods. DOI
- Vowels, M. J. (2025). A causal research pipeline and tutorial for psychologists and social scientists. Psychological Methods. DOI
- Vowels, M. J. (2024). Typical yet unlikely and normally abnormal: The intuition behind high-dimensional statistics. Statistics, Politics and Policy, 15(1), 87–113. DOI
- Aczel, B., Szaszi, B., Clelland, H. T., Kovacs, M., Holzmeister, F., et al. (2026). Investigating the analytical robustness of the social and behavioural sciences. Nature, 652(8108), 135–142. DOI
- Higgins, W. C., Clarke, B., Elson, M., & Cummins, J. (2026). Recommendations for incorporating LLMs into psychological research: A commentary on Austin and colleagues (2026). PsyArXiv. DOI
- Ahnström, L., Bruckner, T., Aspromonti, D. A., Caquelin, L., Cummins, J., et al. (2026). TrialScout links published results to trial registrations using a large language model. medRxiv. DOI
- Elson, M., Hussey, I., Clarke, B., Norwood, S. F., Grinschgl, S., Arslan, R. C., et al. (2026). Against anonymising meta-scientific data. PsyArXiv. DOI
- Cummins, J., Clarke, B., Hussey, I., & Elson, M. (2026). RegCheck: A tool for automating comparisons between study registrations and papers. arXiv. DOI
- Miske, O., Abatayo, A. L., Daley, M., Dirzo, M., Fox, N., Haber, N., Hahn, K. M., et al. (2026). Investigating the reproducibility of the social and behavioural sciences. Nature, 652(8108), 126–134. DOI
- Cummins, J. (2025, September 1). Psychology needs… an AI revolution. The Psychologist. Article
- Röseler, L., Kaiser, L., Doetsch, C., Klett, N., Seida, C., Schütz, A., Aczel, B., et al. (2024). The Replication Database: Documenting the replicability of psychological science. Journal of Open Psychology Data, 12(1), Article 8. DOI
- Tierney, W., Hardy, J. H., III, Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., Hiring Decisions Forecasting Collaboration, & Uhlmann, E. L. (2020). Creative destruction in science. Organizational Behavior and Human Decision Processes, 161, 291–309. DOI
- Van Dessel, P., Cummins, J., Hughes, S. J., Kasran, S., Cathelyn, F., & Moran Yorovich, T. (2020). Reflecting on twenty-five years of research using implicit measures: Recommendations for their future use. Social Cognition, 38(Supplement), S223–S242. DOI