Text

Background: Digital public health interventions (DiPHI) are proliferating globally, over 350,000 health apps are now available. Systematic evaluation is essential to safeguard quality, safety, and equity. The DigiPHrame framework (Leibniz ScienceCampus Digital Public Health) covers 13 domains and 207 criteria spanning technical, usability, ethical, legal, data security, equity, sustainability, and AI aspects. Manual application requires multidisciplinary expertise and significant time, limiting scalability.

Aim/scientific question: To develop and assess the feasibility of DigiPhrame AI, an evidence-grounded AI prototype for automated, structured evaluation of DiPHI using the DigiPHrame framework.

Material & Methods: DigiPhrame AI combines a FastAPI backend with a React/TypeScript frontend. Intervention documents and application source code are semantically indexed in a ChromaDB vector store, enabling evidence retrieval from both technical and narrative sources. A RAG pipeline retrieves criterion-relevant passages and passes them to 13 domain-specific LLM personas grounded in 39 best-practice frameworks (WHO, ISO, GDPR, PROGRESS-Plus) that translate evidence gaps into actionable recommendations. Multiple LLM providers are supported, including offline-capable local models for GDPR-compliant deployment. For each of the 207 criteria, the system produces a four-category score, source-linked evidence citations, score justification, and a prioritised recommendation.

Results: DigiPhrame AI was applied to two digital public health interventions, a cancer prevention app and an ANC/PNC maternity app, each receiving a complete report across all 13 domains and 207 criteria, with source-linked evidence and prioritised recommendations per criterion. Full evaluation completed in under 30 minutes per intervention. A systematic study comparing DigiPhrame AI outputs against independent expert assessments is currently underway.

Discussion: These initial evaluations confirm that comprehensive AI-assisted DiPHI evaluation is technically feasible. RAG-based grounding ensures evidence traceability and reduces hallucination risk. Dependence on documentation completeness remains a key limitation. The ongoing expert comparison study will establish reliability across a broader range of intervention types.

Take Home Message: DigiPhrame AI is a scalable, transparent, and privacy-adaptable tool enabling researchers and developers to evaluate digital interventions against the full DigiPHrame framework, reducing the time and expertise burden of systematic quality assessment.

Competing interests: None