A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America

Autores
Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; Martínez, Lautaro; Sánchez, Jorge; Schilman, Mauro; Ivetta, Guido; Halvorsen, Alexia; Mata Rojo, Amanda; Bordon, Matías; Busaniche, Beatriz
Año de publicación
2023
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.
Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; Argentina
Fil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Halvorsen, Alexia. Fundación Via Libre; Argentina
Fil: Mata Rojo, Amanda. Fundación Via Libre; Argentina
Fil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Busaniche, Beatriz. Fundación Via Libre; Argentina
Materia
Natural Language Processing
Language models
Bias
Stereotypes and Discrimination
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/218993
Acceder
id CONICETDig_d54f959f09b0db4b2c4be5532e6cecb5
oai_identifier_str oai:ri.conicet.gov.ar:11336/218993
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling A methodology to characterize bias and harmful stereotypes in natural language processing in Latin AmericaAlemany, Laura AlonsoBenotti, LucianaMaina, Hernán JavierGonzalez, LucíaRajngewerc, MarielaMartínez, LautaroSánchez, JorgeSchilman, MauroIvetta, GuidoHalvorsen, AlexiaMata Rojo, AmandaBordon, MatíasBusaniche, BeatrizNatural Language ProcessingLanguage modelsBiasStereotypes and Discriminationhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; ArgentinaFil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Halvorsen, Alexia. Fundación Via Libre; ArgentinaFil: Mata Rojo, Amanda. Fundación Via Libre; ArgentinaFil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Busaniche, Beatriz. Fundación Via Libre; ArgentinaCornell University2023-03info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/218993Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-242331-8422CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://arxiv.org/abs/2207.06591v3info:eu-repo/semantics/altIdentifier/doi/10.48550/arXiv.2207.06591info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-10-22T11:37:40Zoai:ri.conicet.gov.ar:11336/218993instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-10-22 11:37:41.08CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
title A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
spellingShingle A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
Alemany, Laura Alonso
Natural Language Processing
Language models
Bias
Stereotypes and Discrimination
title_short A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
title_full A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
title_fullStr A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
title_full_unstemmed A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
title_sort A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
dc.creator.none.fl_str_mv Alemany, Laura Alonso
Benotti, Luciana
Maina, Hernán Javier
Gonzalez, Lucía
Rajngewerc, Mariela
Martínez, Lautaro
Sánchez, Jorge
Schilman, Mauro
Ivetta, Guido
Halvorsen, Alexia
Mata Rojo, Amanda
Bordon, Matías
Busaniche, Beatriz
author Alemany, Laura Alonso
author_facet Alemany, Laura Alonso
Benotti, Luciana
Maina, Hernán Javier
Gonzalez, Lucía
Rajngewerc, Mariela
Martínez, Lautaro
Sánchez, Jorge
Schilman, Mauro
Ivetta, Guido
Halvorsen, Alexia
Mata Rojo, Amanda
Bordon, Matías
Busaniche, Beatriz
author_role author
author2 Benotti, Luciana
Maina, Hernán Javier
Gonzalez, Lucía
Rajngewerc, Mariela
Martínez, Lautaro
Sánchez, Jorge
Schilman, Mauro
Ivetta, Guido
Halvorsen, Alexia
Mata Rojo, Amanda
Bordon, Matías
Busaniche, Beatriz
author2_role author
author
author
author
author
author
author
author
author
author
author
author
dc.subject.none.fl_str_mv Natural Language Processing
Language models
Bias
Stereotypes and Discrimination
topic Natural Language Processing
Language models
Bias
Stereotypes and Discrimination
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.
Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; Argentina
Fil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Halvorsen, Alexia. Fundación Via Libre; Argentina
Fil: Mata Rojo, Amanda. Fundación Via Libre; Argentina
Fil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Busaniche, Beatriz. Fundación Via Libre; Argentina
description Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.
publishDate 2023
dc.date.none.fl_str_mv 2023-03
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/218993
Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-24
2331-8422
CONICET Digital
CONICET
url http://hdl.handle.net/11336/218993
identifier_str_mv Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-24
2331-8422
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://arxiv.org/abs/2207.06591v3
info:eu-repo/semantics/altIdentifier/doi/10.48550/arXiv.2207.06591
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Cornell University
publisher.none.fl_str_mv Cornell University
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1846782027479646208
score 12.982451

Publicaciones similares