Title
Agentic AI for Research Evaluation Systems: A Decision-Sciences Framework with Ownership, Responsibility, and Organisational Strength
Authors
Abstract
Purpose — Universities continue to reward journal prestige and publication counts, although responsible-assessment movements have shown that these proxies mismeasure individual contribution and render translation, infrastructure, and societal engagement structurally invisible. This paper reconceptualises research evaluation as a decision-sciences design problem and asks whether agentic artificial intelligence (AI) can make broader, evidence-centred evaluation operationally feasible without weakening accountability. The central claim is conditional: earlier adoption strengthens the organisation only when governance quality and ownership clarity exceed identifiable thresholds; below them, earlier adoption scales confusion and bias.
Design/methodology/approach — The paper develops a conceptual framework with a formalised propositional model. Responsible-assessment principles (DORA, the Leiden Manifesto, CoARA, and the Singapore Statement on Research Integrity) are synthesised with a five-agent system architecture (narrative parsing, multi-label classification, knowledge-graph construction, reviewer recommendation, and explanation over a shared evidence registry). Twelve equations define organisational evaluation strength as a bounded [0,1] composite of decision quality, fairness, strategic alignment, and organisational learning. An explicit adoption-stock bridge connects adoption timing and intensity to model capability and evidence richness, so the adoption effect is derived as a total derivative rather than asserted. Proposition 1 and Hypotheses H1–H5 state the testable implications, with full operationalisation, identification, validity, and ethics provisions.
Findings — Organisational evaluation strength is a joint function of AI capability, evidence richness, governance quality, ownership clarity, and explainability. The sign of the adoption effect is determined by a race between a capability-and-learning channel (positive) and a governance-erosion channel (negative when adoption dilutes accountability or amplifies bias). Thresholds G* and O* are therefore not arbitrary constants but sign-change conditions solvable from the model. Under stated assumptions, advancing the adoption date (holding the investment path fixed) weakly increases learning and strength at every horizon — a corollary that gives the familiar slogan ‘the sooner, the stronger’ a precise, falsifiable meaning.
Research limitations/implications — The model is formal and propositional; coefficients and thresholds require empirical calibration. The paper specifies a four-stage validation programme linked hypothesis-by-hypothesis to the model (extraction benchmarking, graph verification, committee human-factors experiments, and adversarial bias audits), together with difference-in-differences and instrumental-variable strategies for the self-selection problem that threatens any naive ‘early adopters perform better’ comparison. Field heterogeneity and cross-institutional portability beyond the Singapore deployment context remain open.
Practical implications — Institutions can redesign evaluation workflows around inspectable evidence packets, assign five accountable human roles (system owner, decision owner, model steward, audit lead, domain expert reviewer), and benchmark decision-system quality with the ownership-clarity and traceability indices defined here. Singapore’s RIE2030 (S$37 billion, 2026–2030) provides a policy-aligned deployment context whose four strategic domains supply the mission taxonomy for the relevance construct.
Social implications — Appropriately governed AI-augmented evaluation can reduce prestige-proxy dependence, surface overlooked contributions (software, data, standards, mentoring, policy influence), and realign academic incentives toward genuine knowledge creation and translational impact, while preserving due process, contestability, and appeal rights for those evaluated.
Originality/value — The paper offers (i) a decision-sciences formalisation of agentic-AI adoption in academic evaluation with bounded, internally consistent dynamics; (ii) an explicit adoption bridge that repairs the missing link between adoption timing and organisational strength; (iii) a governance architecture that operationalises the Singapore Statement as auditable checkpoints aligned to NIST AI RMF and ISO/IEC 42001; and (iv) a hypothesis-linked empirical roadmap. It reframes evaluation reform as governance engineering, not merely a normative aspiration.
Keywords
agentic AI; research evaluation; decision sciences; narrative CV; knowledge graph; explainable AI; responsible research assessment; organisational learning
Classification-JEL
I23, O33, M15, D83
Pages
30-54
How to Cite
Chan, L. C. W. J., & Pan, S. H. (2026). Agentic AI for Research Evaluation Systems: A Decision-Sciences Framework with Ownership, Responsibility, and Organisational Strength. Annals of Artificial Intelligence, 1(1), 30-54.