Abstraction-Aware Inference of Metamorphic Relations

Autores
Nolasco, Franco Agustín; Molina, Facundo Joaquín; Degiovanni, Renzo Gastón; Gorla, Alessandra; Garbervetsky, Diego David; Papadakis, Mike; Uchitel, Sebastian; Aguirre, Nazareno Matias; Frias, Marcelo Fabian
Año de publicación
2024
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called metamorphic relations, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.
Fil: Nolasco, Franco Agustín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina
Fil: Molina, Facundo Joaquín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina. IMDEA Software Institute; España
Fil: Degiovanni, Renzo Gastón. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Luxembourg Institute Of Science And Technology; Luxemburgo
Fil: Gorla, Alessandra. IMDEA Software Institute; España
Fil: Garbervetsky, Diego David. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; Argentina
Fil: Papadakis, Mike. University of Luxembourg; Luxemburgo
Fil: Uchitel, Sebastian. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; Argentina. Imperial College London; Reino Unido
Fil: Aguirre, Nazareno Matias. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina
Fil: Frias, Marcelo Fabian. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. University of Texas at El Paso; Estados Unidos
Materia
Specification inference
Metamorphic testing
Fuzzing
SAT solving
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/256849
Acceder
id CONICETDig_2773ef7c35f908c9dae9408011d8f0da
oai_identifier_str oai:ri.conicet.gov.ar:11336/256849
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Abstraction-Aware Inference of Metamorphic RelationsNolasco, Franco AgustínMolina, Facundo JoaquínDegiovanni, Renzo GastónGorla, AlessandraGarbervetsky, Diego DavidPapadakis, MikeUchitel, SebastianAguirre, Nazareno MatiasFrias, Marcelo FabianSpecification inferenceMetamorphic testingFuzzingSAT solvinghttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called metamorphic relations, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.Fil: Nolasco, Franco Agustín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; ArgentinaFil: Molina, Facundo Joaquín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina. IMDEA Software Institute; EspañaFil: Degiovanni, Renzo Gastón. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Luxembourg Institute Of Science And Technology; LuxemburgoFil: Gorla, Alessandra. IMDEA Software Institute; EspañaFil: Garbervetsky, Diego David. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; ArgentinaFil: Papadakis, Mike. University of Luxembourg; LuxemburgoFil: Uchitel, Sebastian. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; Argentina. Imperial College London; Reino UnidoFil: Aguirre, Nazareno Matias. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; ArgentinaFil: Frias, Marcelo Fabian. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. University of Texas at El Paso; Estados UnidosAssociation for Computing Machinery2024-07info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/256849Nolasco, Franco Agustín; Molina, Facundo Joaquín; Degiovanni, Renzo Gastón; Gorla, Alessandra; Garbervetsky, Diego David; et al.; Abstraction-Aware Inference of Metamorphic Relations; Association for Computing Machinery; Proceedings of the ACM on Software Engineering; 1; FSE; 7-2024; 450-4722994-970XCONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://dl.acm.org/doi/10.1145/3643747info:eu-repo/semantics/altIdentifier/doi/10.1145/3643747info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:27:03Zoai:ri.conicet.gov.ar:11336/256849instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:27:04.181CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Abstraction-Aware Inference of Metamorphic Relations
title Abstraction-Aware Inference of Metamorphic Relations
spellingShingle Abstraction-Aware Inference of Metamorphic Relations
Nolasco, Franco Agustín
Specification inference
Metamorphic testing
Fuzzing
SAT solving
title_short Abstraction-Aware Inference of Metamorphic Relations
title_full Abstraction-Aware Inference of Metamorphic Relations
title_fullStr Abstraction-Aware Inference of Metamorphic Relations
title_full_unstemmed Abstraction-Aware Inference of Metamorphic Relations
title_sort Abstraction-Aware Inference of Metamorphic Relations
dc.creator.none.fl_str_mv Nolasco, Franco Agustín
Molina, Facundo Joaquín
Degiovanni, Renzo Gastón
Gorla, Alessandra
Garbervetsky, Diego David
Papadakis, Mike
Uchitel, Sebastian
Aguirre, Nazareno Matias
Frias, Marcelo Fabian
author Nolasco, Franco Agustín
author_facet Nolasco, Franco Agustín
Molina, Facundo Joaquín
Degiovanni, Renzo Gastón
Gorla, Alessandra
Garbervetsky, Diego David
Papadakis, Mike
Uchitel, Sebastian
Aguirre, Nazareno Matias
Frias, Marcelo Fabian
author_role author
author2 Molina, Facundo Joaquín
Degiovanni, Renzo Gastón
Gorla, Alessandra
Garbervetsky, Diego David
Papadakis, Mike
Uchitel, Sebastian
Aguirre, Nazareno Matias
Frias, Marcelo Fabian
author2_role author
author
author
author
author
author
author
author
dc.subject.none.fl_str_mv Specification inference
Metamorphic testing
Fuzzing
SAT solving
topic Specification inference
Metamorphic testing
Fuzzing
SAT solving
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called metamorphic relations, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.
Fil: Nolasco, Franco Agustín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina
Fil: Molina, Facundo Joaquín. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina. IMDEA Software Institute; España
Fil: Degiovanni, Renzo Gastón. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Luxembourg Institute Of Science And Technology; Luxemburgo
Fil: Gorla, Alessandra. IMDEA Software Institute; España
Fil: Garbervetsky, Diego David. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; Argentina
Fil: Papadakis, Mike. University of Luxembourg; Luxemburgo
Fil: Uchitel, Sebastian. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; Argentina. Imperial College London; Reino Unido
Fil: Aguirre, Nazareno Matias. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. Universidad Nacional de Río Cuarto. Facultad de Ciencias Exactas Fisicoquímicas y Naturales. Departamento de Computación; Argentina
Fil: Frias, Marcelo Fabian. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina. University of Texas at El Paso; Estados Unidos
description Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called metamorphic relations, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.
publishDate 2024
dc.date.none.fl_str_mv 2024-07
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/256849
Nolasco, Franco Agustín; Molina, Facundo Joaquín; Degiovanni, Renzo Gastón; Gorla, Alessandra; Garbervetsky, Diego David; et al.; Abstraction-Aware Inference of Metamorphic Relations; Association for Computing Machinery; Proceedings of the ACM on Software Engineering; 1; FSE; 7-2024; 450-472
2994-970X
CONICET Digital
CONICET
url http://hdl.handle.net/11336/256849
identifier_str_mv Nolasco, Franco Agustín; Molina, Facundo Joaquín; Degiovanni, Renzo Gastón; Gorla, Alessandra; Garbervetsky, Diego David; et al.; Abstraction-Aware Inference of Metamorphic Relations; Association for Computing Machinery; Proceedings of the ACM on Software Engineering; 1; FSE; 7-2024; 450-472
2994-970X
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://dl.acm.org/doi/10.1145/3643747
info:eu-repo/semantics/altIdentifier/doi/10.1145/3643747
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Association for Computing Machinery
publisher.none.fl_str_mv Association for Computing Machinery
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1844614272623050752
score 13.070432

Publicaciones similares