Epistemology

Machine learning

Psychology

Programming language

Data science

Proxy (statistics)

Crowdsourcing

Social psychology

Psychiatry

Philosophy

Environmental health

Population

Operationalization

External validity

Social media

Applied psychology

Remedial education

Mathematics education

Cognitive psychology

Computer science

Internal validity

Mental health

World Wide Web

Pathology

Construct (python library)

Medicine

A growing body of research is combining social media data with machine learning to predict mental health states of individuals. An implication of this research lies in informing evidence-based diagnosis and treatment. However, obtaining clinically valid diagnostic information from sensitive patient populations is challenging. Consequently, researchers have operationalized characteristic online behaviors as "proxy diagnostic signals" for building these models. This paper posits a challenge in using these diagnostic signals, purported to support clinical decision-making. Focusing on three commonly used proxy diagnostic signals derived from social media, we find that predictive models built on these data, although offer strong internal validity, suffer from poor external validity when tested on mental health patients. A deeper dive reveals issues of population and sampling bias, as well as of uncertainty in construct validity inherent in these proxies. We discuss the methodological and clinical implications of these gaps and provide remedial guidelines for future research.

Methodological Gaps in Predicting Mental Health States from Social Media