Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience

Autores: Coles, Nicholas A.; Perz, Bartosz; Behnke, Maciej; Eichstaedt, Johannes C.; Kim, Soo Hyung; Vu, Tu N.; Raman, Chirag; Tejada, Julian; Huynh, Van Thong; Zhang, Guangyi; Cui, Tanming; Podder, Sharanyak; Chavda, Rushi; Pandey, Shubham; Upadhyay, Arpit; Padilla Buritica, Jorge I.; Barrera Causil, Carlos J.; Ji, Linying; Dollack, Felix; Kiyokawa, Kiyoshi; Liu, Huakun; Tagliazucchi, Enzo Rodolfo; Bugnon, Leandro Ariel; Bruno, Nicolás Marcelo; D Amelio, Tomás Ariel; Hinduja, Saurabh; Marmolejo Ramos, Fernando; Canavan, Shaun; Jivnani, Liza; Saganowski, Stanisław
Año de publicación: 2025
Idioma: inglés
Tipo de recurso: artículo
Estado: versión publicada
Descripción: Researchers are increasingly using machine learning to study physiological markers of emotion. We evaluated the promises and limitations of this approach via a big team science competition. Twelve teams competed to predict self-reported affective experiences using a multi-modal set of peripheral nervous system measures. Models were trained and tested in multiple ways: with data divided by participants, targeted emotion, inductions, and time. In 100% of tests, teams outperformed baseline models that made random predictions. In 46% of tests, teams also outperformed baseline models that relied on the simple average of ratings from training datasets. More notably, results uncovered a methodological challenge: multiplicative constraints on generalizability. Inferences about the accuracy and theoretical implications of machine learning efforts depended not only on their architecture, but also how they were trained, tested, and evaluated. For example, some teams performed better when tested on observations from the same (vs. different) subjects seen during training. Such results could be interpreted as evidence against claims of universality. However, such conclusions would be premature because other teams exhibited the opposite pattern. Taken together, results illustrate how big team science can be leveraged to understand the promises and limitations of machine learning methods in affective science and beyond.
Fil: Coles, Nicholas A.. University of Florida; Estados Unidos
Fil: Perz, Bartosz. Wrocław University of Science and Technology; Polonia
Fil: Behnke, Maciej. Adam Mickiewicz University; Polonia
Fil: Eichstaedt, Johannes C.. University of Stanford; Estados Unidos
Fil: Kim, Soo Hyung. Harvard Medical School; Estados Unidos
Fil: Vu, Tu N.. Chonnam National University; Corea del Sur
Fil: Raman, Chirag. Delft University of Technology; Países Bajos
Fil: Tejada, Julian. Universidade Federal de Sergipe; Brasil
Fil: Huynh, Van Thong. FPT University; Vietnam
Fil: Zhang, Guangyi. Harvard Medical School; Estados Unidos
Fil: Cui, Tanming. No especifíca;
Fil: Podder, Sharanyak. Indian Institute of Science Education and Research Bhopal; India
Fil: Chavda, Rushi. Indian Institute of Technology Bombay; India
Fil: Pandey, Shubham. Indian Institute of Technology Bombay; India
Fil: Upadhyay, Arpit. Indian Institute of Technology Bombay; India
Fil: Padilla Buritica, Jorge I.. Instituto Tecnológico Metropolitano.; Colombia
Fil: Barrera Causil, Carlos J.. Instituto Tecnológico Metropolitano.; Colombia
Fil: Ji, Linying. Montana State University; Estados Unidos
Fil: Dollack, Felix. Nara Institute of Science and Technology; Japón
Fil: Kiyokawa, Kiyoshi. Nara Institute of Science and Technology; Japón
Fil: Liu, Huakun. Nara Institute of Science and Technology; Japón
Fil: Tagliazucchi, Enzo Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina
Fil: Bugnon, Leandro Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina
Fil: Bruno, Nicolás Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina
Fil: D Amelio, Tomás Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina
Fil: Hinduja, Saurabh. University of South Florida; Estados Unidos
Fil: Marmolejo Ramos, Fernando. Flinders University.; Australia
Fil: Canavan, Shaun. University of South Florida; Estados Unidos
Fil: Jivnani, Liza. University of South Florida; Estados Unidos
Fil: Saganowski, Stanisław. Wrocław University of Science and Technology; Polonia
Materia: machine learning
affective computing
physiological signals
Nivel de accesibilidad: acceso abierto
Condiciones de uso: https://creativecommons.org/licenses/by/2.5/ar/
Repositorio
Institución: Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador: oai:ri.conicet.gov.ar:11336/275082

Acceder

id	CONICETDig_f2d36a851235be7c65ba6e9654a90e11
oai_identifier_str	oai:ri.conicet.gov.ar:11336/275082
network_acronym_str	CONICETDig
repository_id_str	3498
network_name_str	CONICET Digital (CONICET)
spelling	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experienceColes, Nicholas A.Perz, BartoszBehnke, MaciejEichstaedt, Johannes C.Kim, Soo HyungVu, Tu N.Raman, ChiragTejada, JulianHuynh, Van ThongZhang, GuangyiCui, TanmingPodder, SharanyakChavda, RushiPandey, ShubhamUpadhyay, ArpitPadilla Buritica, Jorge I.Barrera Causil, Carlos J.Ji, LinyingDollack, FelixKiyokawa, KiyoshiLiu, HuakunTagliazucchi, Enzo RodolfoBugnon, Leandro ArielBruno, Nicolás MarceloD Amelio, Tomás ArielHinduja, SaurabhMarmolejo Ramos, FernandoCanavan, ShaunJivnani, LizaSaganowski, Stanisławmachine learningaffective computingphysiological signalshttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1Researchers are increasingly using machine learning to study physiological markers of emotion. We evaluated the promises and limitations of this approach via a big team science competition. Twelve teams competed to predict self-reported affective experiences using a multi-modal set of peripheral nervous system measures. Models were trained and tested in multiple ways: with data divided by participants, targeted emotion, inductions, and time. In 100% of tests, teams outperformed baseline models that made random predictions. In 46% of tests, teams also outperformed baseline models that relied on the simple average of ratings from training datasets. More notably, results uncovered a methodological challenge: multiplicative constraints on generalizability. Inferences about the accuracy and theoretical implications of machine learning efforts depended not only on their architecture, but also how they were trained, tested, and evaluated. For example, some teams performed better when tested on observations from the same (vs. different) subjects seen during training. Such results could be interpreted as evidence against claims of universality. However, such conclusions would be premature because other teams exhibited the opposite pattern. Taken together, results illustrate how big team science can be leveraged to understand the promises and limitations of machine learning methods in affective science and beyond.Fil: Coles, Nicholas A.. University of Florida; Estados UnidosFil: Perz, Bartosz. Wrocław University of Science and Technology; PoloniaFil: Behnke, Maciej. Adam Mickiewicz University; PoloniaFil: Eichstaedt, Johannes C.. University of Stanford; Estados UnidosFil: Kim, Soo Hyung. Harvard Medical School; Estados UnidosFil: Vu, Tu N.. Chonnam National University; Corea del SurFil: Raman, Chirag. Delft University of Technology; Países BajosFil: Tejada, Julian. Universidade Federal de Sergipe; BrasilFil: Huynh, Van Thong. FPT University; VietnamFil: Zhang, Guangyi. Harvard Medical School; Estados UnidosFil: Cui, Tanming. No especifíca;Fil: Podder, Sharanyak. Indian Institute of Science Education and Research Bhopal; IndiaFil: Chavda, Rushi. Indian Institute of Technology Bombay; IndiaFil: Pandey, Shubham. Indian Institute of Technology Bombay; IndiaFil: Upadhyay, Arpit. Indian Institute of Technology Bombay; IndiaFil: Padilla Buritica, Jorge I.. Instituto Tecnológico Metropolitano.; ColombiaFil: Barrera Causil, Carlos J.. Instituto Tecnológico Metropolitano.; ColombiaFil: Ji, Linying. Montana State University; Estados UnidosFil: Dollack, Felix. Nara Institute of Science and Technology; JapónFil: Kiyokawa, Kiyoshi. Nara Institute of Science and Technology; JapónFil: Liu, Huakun. Nara Institute of Science and Technology; JapónFil: Tagliazucchi, Enzo Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; ArgentinaFil: Bugnon, Leandro Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; ArgentinaFil: Bruno, Nicolás Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; ArgentinaFil: D Amelio, Tomás Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; ArgentinaFil: Hinduja, Saurabh. University of South Florida; Estados UnidosFil: Marmolejo Ramos, Fernando. Flinders University.; AustraliaFil: Canavan, Shaun. University of South Florida; Estados UnidosFil: Jivnani, Liza. University of South Florida; Estados UnidosFil: Saganowski, Stanisław. Wrocław University of Science and Technology; PoloniaRoyal Society of Chemistry2025-06info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/275082Coles, Nicholas A.; Perz, Bartosz; Behnke, Maciej; Eichstaedt, Johannes C.; Kim, Soo Hyung; et al.; Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience; Royal Society of Chemistry; Royal Society Open Science; 12; 6; 6-2025; 1-132054-5703CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://royalsocietypublishing.org/doi/10.1098/rsos.241778info:eu-repo/semantics/altIdentifier/doi/10.1098/rsos.241778info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-12-03T08:52:06Zoai:ri.conicet.gov.ar:11336/275082instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-12-03 08:52:06.402CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
title	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
spellingShingle	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience Coles, Nicholas A. machine learning affective computing physiological signals
title_short	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
title_full	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
title_fullStr	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
title_full_unstemmed	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
title_sort	Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience
dc.creator.none.fl_str_mv	Coles, Nicholas A. Perz, Bartosz Behnke, Maciej Eichstaedt, Johannes C. Kim, Soo Hyung Vu, Tu N. Raman, Chirag Tejada, Julian Huynh, Van Thong Zhang, Guangyi Cui, Tanming Podder, Sharanyak Chavda, Rushi Pandey, Shubham Upadhyay, Arpit Padilla Buritica, Jorge I. Barrera Causil, Carlos J. Ji, Linying Dollack, Felix Kiyokawa, Kiyoshi Liu, Huakun Tagliazucchi, Enzo Rodolfo Bugnon, Leandro Ariel Bruno, Nicolás Marcelo D Amelio, Tomás Ariel Hinduja, Saurabh Marmolejo Ramos, Fernando Canavan, Shaun Jivnani, Liza Saganowski, Stanisław
author	Coles, Nicholas A.
author_facet	Coles, Nicholas A. Perz, Bartosz Behnke, Maciej Eichstaedt, Johannes C. Kim, Soo Hyung Vu, Tu N. Raman, Chirag Tejada, Julian Huynh, Van Thong Zhang, Guangyi Cui, Tanming Podder, Sharanyak Chavda, Rushi Pandey, Shubham Upadhyay, Arpit Padilla Buritica, Jorge I. Barrera Causil, Carlos J. Ji, Linying Dollack, Felix Kiyokawa, Kiyoshi Liu, Huakun Tagliazucchi, Enzo Rodolfo Bugnon, Leandro Ariel Bruno, Nicolás Marcelo D Amelio, Tomás Ariel Hinduja, Saurabh Marmolejo Ramos, Fernando Canavan, Shaun Jivnani, Liza Saganowski, Stanisław
author_role	author
author2	Perz, Bartosz Behnke, Maciej Eichstaedt, Johannes C. Kim, Soo Hyung Vu, Tu N. Raman, Chirag Tejada, Julian Huynh, Van Thong Zhang, Guangyi Cui, Tanming Podder, Sharanyak Chavda, Rushi Pandey, Shubham Upadhyay, Arpit Padilla Buritica, Jorge I. Barrera Causil, Carlos J. Ji, Linying Dollack, Felix Kiyokawa, Kiyoshi Liu, Huakun Tagliazucchi, Enzo Rodolfo Bugnon, Leandro Ariel Bruno, Nicolás Marcelo D Amelio, Tomás Ariel Hinduja, Saurabh Marmolejo Ramos, Fernando Canavan, Shaun Jivnani, Liza Saganowski, Stanisław
author2_role	author author author author author author author author author author author author author author author author author author author author author author author author author author author author author
dc.subject.none.fl_str_mv	machine learning affective computing physiological signals
topic	machine learning affective computing physiological signals
purl_subject.fl_str_mv	https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv	Researchers are increasingly using machine learning to study physiological markers of emotion. We evaluated the promises and limitations of this approach via a big team science competition. Twelve teams competed to predict self-reported affective experiences using a multi-modal set of peripheral nervous system measures. Models were trained and tested in multiple ways: with data divided by participants, targeted emotion, inductions, and time. In 100% of tests, teams outperformed baseline models that made random predictions. In 46% of tests, teams also outperformed baseline models that relied on the simple average of ratings from training datasets. More notably, results uncovered a methodological challenge: multiplicative constraints on generalizability. Inferences about the accuracy and theoretical implications of machine learning efforts depended not only on their architecture, but also how they were trained, tested, and evaluated. For example, some teams performed better when tested on observations from the same (vs. different) subjects seen during training. Such results could be interpreted as evidence against claims of universality. However, such conclusions would be premature because other teams exhibited the opposite pattern. Taken together, results illustrate how big team science can be leveraged to understand the promises and limitations of machine learning methods in affective science and beyond. Fil: Coles, Nicholas A.. University of Florida; Estados Unidos Fil: Perz, Bartosz. Wrocław University of Science and Technology; Polonia Fil: Behnke, Maciej. Adam Mickiewicz University; Polonia Fil: Eichstaedt, Johannes C.. University of Stanford; Estados Unidos Fil: Kim, Soo Hyung. Harvard Medical School; Estados Unidos Fil: Vu, Tu N.. Chonnam National University; Corea del Sur Fil: Raman, Chirag. Delft University of Technology; Países Bajos Fil: Tejada, Julian. Universidade Federal de Sergipe; Brasil Fil: Huynh, Van Thong. FPT University; Vietnam Fil: Zhang, Guangyi. Harvard Medical School; Estados Unidos Fil: Cui, Tanming. No especifíca; Fil: Podder, Sharanyak. Indian Institute of Science Education and Research Bhopal; India Fil: Chavda, Rushi. Indian Institute of Technology Bombay; India Fil: Pandey, Shubham. Indian Institute of Technology Bombay; India Fil: Upadhyay, Arpit. Indian Institute of Technology Bombay; India Fil: Padilla Buritica, Jorge I.. Instituto Tecnológico Metropolitano.; Colombia Fil: Barrera Causil, Carlos J.. Instituto Tecnológico Metropolitano.; Colombia Fil: Ji, Linying. Montana State University; Estados Unidos Fil: Dollack, Felix. Nara Institute of Science and Technology; Japón Fil: Kiyokawa, Kiyoshi. Nara Institute of Science and Technology; Japón Fil: Liu, Huakun. Nara Institute of Science and Technology; Japón Fil: Tagliazucchi, Enzo Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina Fil: Bugnon, Leandro Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina Fil: Bruno, Nicolás Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina Fil: D Amelio, Tomás Ariel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física del Plasma. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física del Plasma; Argentina Fil: Hinduja, Saurabh. University of South Florida; Estados Unidos Fil: Marmolejo Ramos, Fernando. Flinders University.; Australia Fil: Canavan, Shaun. University of South Florida; Estados Unidos Fil: Jivnani, Liza. University of South Florida; Estados Unidos Fil: Saganowski, Stanisław. Wrocław University of Science and Technology; Polonia
description	Researchers are increasingly using machine learning to study physiological markers of emotion. We evaluated the promises and limitations of this approach via a big team science competition. Twelve teams competed to predict self-reported affective experiences using a multi-modal set of peripheral nervous system measures. Models were trained and tested in multiple ways: with data divided by participants, targeted emotion, inductions, and time. In 100% of tests, teams outperformed baseline models that made random predictions. In 46% of tests, teams also outperformed baseline models that relied on the simple average of ratings from training datasets. More notably, results uncovered a methodological challenge: multiplicative constraints on generalizability. Inferences about the accuracy and theoretical implications of machine learning efforts depended not only on their architecture, but also how they were trained, tested, and evaluated. For example, some teams performed better when tested on observations from the same (vs. different) subjects seen during training. Such results could be interpreted as evidence against claims of universality. However, such conclusions would be premature because other teams exhibited the opposite pattern. Taken together, results illustrate how big team science can be leveraged to understand the promises and limitations of machine learning methods in affective science and beyond.
publishDate	2025
dc.date.none.fl_str_mv	2025-06
dc.type.none.fl_str_mv	info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo
format	article
status_str	publishedVersion
dc.identifier.none.fl_str_mv	http://hdl.handle.net/11336/275082 Coles, Nicholas A.; Perz, Bartosz; Behnke, Maciej; Eichstaedt, Johannes C.; Kim, Soo Hyung; et al.; Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience; Royal Society of Chemistry; Royal Society Open Science; 12; 6; 6-2025; 1-13 2054-5703 CONICET Digital CONICET
url	http://hdl.handle.net/11336/275082
identifier_str_mv	Coles, Nicholas A.; Perz, Bartosz; Behnke, Maciej; Eichstaedt, Johannes C.; Kim, Soo Hyung; et al.; Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience; Royal Society of Chemistry; Royal Society Open Science; 12; 6; 6-2025; 1-13 2054-5703 CONICET Digital CONICET
dc.language.none.fl_str_mv	eng
language	eng
dc.relation.none.fl_str_mv	info:eu-repo/semantics/altIdentifier/url/https://royalsocietypublishing.org/doi/10.1098/rsos.241778 info:eu-repo/semantics/altIdentifier/doi/10.1098/rsos.241778
dc.rights.none.fl_str_mv	info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by/2.5/ar/
eu_rights_str_mv	openAccess
rights_invalid_str_mv	https://creativecommons.org/licenses/by/2.5/ar/
dc.format.none.fl_str_mv	application/pdf application/pdf
dc.publisher.none.fl_str_mv	Royal Society of Chemistry
publisher.none.fl_str_mv	Royal Society of Chemistry
dc.source.none.fl_str_mv	reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str	CONICET Digital (CONICET)
collection	CONICET Digital (CONICET)
instname_str	Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv	CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv	dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_	1850504984906432512
score	13.275514

Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience

Publicaciones similares