Project
AI-Assisted Network Construction Methods
This project studies the usefulness of AI-based approaches in constructing network structures. Generative neural nets such as GPT and other Large Language Models (LLMs) can be used to construct at candidate network structures by iteratively probing the LLM to identify plausible links between pairs of symptoms and other variables. Essentially, this puts the LLM in the role traditionally taken by experts in the causal loop diagram method (CLD; Crielaard et al., 2022). In the CLD methodology, experts are queried to construct a tentative causal model of a system, which typically concerns the general structure of causal effects, that is, effects that are expected to generalize across individuals, such as, for example, an edge between the nodes of sleep quality and mood. So, the idea would be to complement the individual network structure as estimated from person-specific Experience Sampling Methodology (ESM) data by adding edges between nodes returned by the LLM that are expected to generalize over individuals. This would allow for the construction of larger networks than could be realistically estimated in a person-specific data analysis.
In this project, we aim to develop this technology and study its methodological properties, such as reliability and validity. In addition, we study how LLMs can be combined with person-specific data. First, we develop an R-package that can construct psychopathology networks using LLMs. Second, we study the reliability and validity of the resulting networks by testing how well they match with network structures derived from different sources of information: a) networks based on expert interviews (Crielaard et al., 2022), b) networks based on self-reports using PECAN methodology in which patients construct networks based on their understanding of their own system (Klintwall et al., 2023), and c) networks based on statistical analysis of (time series) data, Third, taking an explainable AI approach, we aim to tackle a downside of LLM-based approaches, namely that the evidence LLMs base their responses on is not transparent; this problem is addressed by developing extensions where the links between symptoms are identified in a transparent, evidence-based manner based on identifiable scientific papers. This can be done if the LLM has access to a body of relevant scientific literature by using vector stores to search specific documents in these vector stores and enriching prompts with the relevant data (e.g., using customised LLMs; Topsakal & Akinci, 2023). This would inform a therapist with information concerning the relevant edges to address for an individual person, and may lead to suggestions on what mechanism to target.
So, this project aims to investigate the potential of employing an artificial intelligence (AI) approach for building networks to assist clinicians in developing evidence-based and individually tailored case conceptualisations. The ultimate goal is to develop approaches that not only support but also enhance clinical practices, thereby mitigating challenges and alleviating the experienced workload burden by clinicians and patients alike (Roefs et al., 2022).
The current project is well suited as an IOPS project as it aims to create a new psychometric method that incorporates artificial intelligence. This will be achieved by developing an R-package that will enable the creation and validation of psychopathology networks. The project will address transparency concerns by implementing an explainable AI approach and will enhance evidence-based, individually tailored case conceptualizations by combining AI technologies with fundamental psychometric principles.
Supervisors:
Prof. dr. D. Borsboom,
Prof. dr. A. Roefs,
Dr. H. Rosenbusch,
Dr. C. van Lissa,
Dr. B. Verkuil
Period
16 October 2023 – 16 October 2027