A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America
- Autores
- Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; Martínez, Lautaro; Sánchez, Jorge; Schilman, Mauro; Ivetta, Guido; Halvorsen, Alexia; Mata Rojo, Amanda; Bordon, Matías; Busaniche, Beatriz
- Año de publicación
- 2023
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.
Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; Argentina
Fil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Halvorsen, Alexia. Fundación Via Libre; Argentina
Fil: Mata Rojo, Amanda. Fundación Via Libre; Argentina
Fil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Busaniche, Beatriz. Fundación Via Libre; Argentina - Materia
-
Natural Language Processing
Language models
Bias
Stereotypes and Discrimination - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/218993
Ver los metadatos del registro completo
id |
CONICETDig_d54f959f09b0db4b2c4be5532e6cecb5 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/218993 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin AmericaAlemany, Laura AlonsoBenotti, LucianaMaina, Hernán JavierGonzalez, LucíaRajngewerc, MarielaMartínez, LautaroSánchez, JorgeSchilman, MauroIvetta, GuidoHalvorsen, AlexiaMata Rojo, AmandaBordon, MatíasBusaniche, BeatrizNatural Language ProcessingLanguage modelsBiasStereotypes and Discriminationhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought.Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; ArgentinaFil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Halvorsen, Alexia. Fundación Via Libre; ArgentinaFil: Mata Rojo, Amanda. Fundación Via Libre; ArgentinaFil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Busaniche, Beatriz. Fundación Via Libre; ArgentinaCornell University2023-03info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/218993Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-242331-8422CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://arxiv.org/abs/2207.06591v3info:eu-repo/semantics/altIdentifier/doi/10.48550/arXiv.2207.06591info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:07:41Zoai:ri.conicet.gov.ar:11336/218993instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:07:41.876CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
title |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
spellingShingle |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America Alemany, Laura Alonso Natural Language Processing Language models Bias Stereotypes and Discrimination |
title_short |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
title_full |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
title_fullStr |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
title_full_unstemmed |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
title_sort |
A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America |
dc.creator.none.fl_str_mv |
Alemany, Laura Alonso Benotti, Luciana Maina, Hernán Javier Gonzalez, Lucía Rajngewerc, Mariela Martínez, Lautaro Sánchez, Jorge Schilman, Mauro Ivetta, Guido Halvorsen, Alexia Mata Rojo, Amanda Bordon, Matías Busaniche, Beatriz |
author |
Alemany, Laura Alonso |
author_facet |
Alemany, Laura Alonso Benotti, Luciana Maina, Hernán Javier Gonzalez, Lucía Rajngewerc, Mariela Martínez, Lautaro Sánchez, Jorge Schilman, Mauro Ivetta, Guido Halvorsen, Alexia Mata Rojo, Amanda Bordon, Matías Busaniche, Beatriz |
author_role |
author |
author2 |
Benotti, Luciana Maina, Hernán Javier Gonzalez, Lucía Rajngewerc, Mariela Martínez, Lautaro Sánchez, Jorge Schilman, Mauro Ivetta, Guido Halvorsen, Alexia Mata Rojo, Amanda Bordon, Matías Busaniche, Beatriz |
author2_role |
author author author author author author author author author author author author |
dc.subject.none.fl_str_mv |
Natural Language Processing Language models Bias Stereotypes and Discrimination |
topic |
Natural Language Processing Language models Bias Stereotypes and Discrimination |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought. Fil: Alemany, Laura Alonso. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Benotti, Luciana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Física; Argentina Fil: Maina, Hernán Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Gonzalez, Lucía. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Rajngewerc, Mariela. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física. Sección Ciencias de la Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina Fil: Martínez, Lautaro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Sánchez, Jorge. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Schilman, Mauro. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Ivetta, Guido. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Halvorsen, Alexia. Fundación Via Libre; Argentina Fil: Mata Rojo, Amanda. Fundación Via Libre; Argentina Fil: Bordon, Matías. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Busaniche, Beatriz. Fundación Via Libre; Argentina |
description |
Automated decision-making systems, specially those based on natural language processing, are pervasive in our lives. They are not only behind the internet search engines we use daily, but also take more critical roles: selecting candidates for a job, determining suspects of a crime, diagnosing autism and more. Such automated systems make errors, which may be harmful in many ways, be it because of the severity of the consequences (as in health issues) or because of the sheer number of people they affect. When errors made by an automated system affect a population more than other, we call the system biased.Most modern natural language technologies are based on artifacts obtained from enormous volumes of text using machine learning, namely language models and word embeddings. Since they are created applying subsymbolic machine learning, mostly artificial neural networks, they are opaque and practically uninterpretable by direct inspection, thus making it very difficult to audit them.In this paper we present a methodology that spells out how social scientists, domain experts, and machine learning experts can collaboratively explore biases and harmful stereotypes in word embeddings and large language models. Our methodology is based on the following principles:1. focus on the linguistic manifestations of discrimination on word embeddings and language models, not on the mathematical properties of the models2. reduce the technical barrier for discrimination experts3. characterize through a qualitative exploratory process in addition to ametric-based approach4. address mitigation as part of the training process, not as an after thought. |
publishDate |
2023 |
dc.date.none.fl_str_mv |
2023-03 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/218993 Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-24 2331-8422 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/218993 |
identifier_str_mv |
Alemany, Laura Alonso; Benotti, Luciana; Maina, Hernán Javier; Gonzalez, Lucía; Rajngewerc, Mariela; et al.; A methodology to characterize bias and harmful stereotypes in natural language processing in Latin America; Cornell University; arXiv; 3-2023; 1-24 2331-8422 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/https://arxiv.org/abs/2207.06591v3 info:eu-repo/semantics/altIdentifier/doi/10.48550/arXiv.2207.06591 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
Cornell University |
publisher.none.fl_str_mv |
Cornell University |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1844613939990626304 |
score |
13.070432 |