Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system

Autores
Paul, Sinu; Croft, Nathan P.; Purcell, Anthony W.; Tscharke, David C.; Sette, Alessandro; Nielsen, Morten; Peters, Bjoern
Año de publicación
2020
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
T cell epitope candidates are commonly identified using computational prediction tools in order to enable applications such as vaccine design, cancer neoantigen identification, development of diagnostics and removal of unwanted immune responses against protein therapeutics. Most T cell epitope prediction tools are based on machine learning algorithms trained on MHC binding or naturally processed MHC ligand elution data. The ability of currently available tools to predict T cell epitopes has not been comprehensively evaluated. In this study, we used a recently published dataset that systematically defined T cell epitopes recognized in vaccinia virus (VACV) infected C57BL/6 mice (expressing H-2Db and H-2Kb), considering both peptides predicted to bind MHC or experimentally eluted from infected cells, making this the most comprehensive dataset of T cell epitopes mapped in a complex pathogen. We evaluated the performance of all currently publicly available computational T cell epitope prediction tools to identify these major epitopes from all peptides encoded in the VACV proteome. We found that all methods were able to improve epitope identification above random, with the best performance achieved by neural network-based predictions trained on both MHC binding and MHC ligand elution data (NetMHCPan-4.0 and MHCFlurry). Impressively, these methods were able to capture more than half of the major epitopes in the top N = 277 predictions within the N = 767,788 predictions made for distinct peptides of relevant lengths that can theoretically be encoded in the VACV proteome. These performance metrics provide guidance for immunologists as to which prediction methods to use, and what success rates are possible for epitope predictions when considering a highly controlled system of administered immunizations to inbred mice. In addition, this benchmark was implemented in an open and easy to reproduce format, providing developers with a framework for future comparisons against new tools.
Fil: Paul, Sinu. La Jolla Institute for Allergy and Immunology; Estados Unidos
Fil: Croft, Nathan P.. Monash University; Australia
Fil: Purcell, Anthony W.. Monash University; Australia
Fil: Tscharke, David C.. The Australian National University; Australia
Fil: Sette, Alessandro. La Jolla Institute for Allergy and Immunology; Estados Unidos
Fil: Nielsen, Morten. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Investigaciones Biotecnológicas. Universidad Nacional de San Martín. Instituto de Investigaciones Biotecnológicas; Argentina. Technical University of Denmark; Dinamarca
Fil: Peters, Bjoern. La Jolla Institute for Allergy and Immunology; Estados Unidos
Materia
MHC
Benchmark
Immunoinformatics
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/112153

id CONICETDig_5cd44dda8d1aea49bd4674fe71930b65
oai_identifier_str oai:ri.conicet.gov.ar:11336/112153
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model systemPaul, SinuCroft, Nathan P.Purcell, Anthony W.Tscharke, David C.Sette, AlessandroNielsen, MortenPeters, BjoernMHCBenchmarkImmunoinformaticshttps://purl.org/becyt/ford/3.4https://purl.org/becyt/ford/3T cell epitope candidates are commonly identified using computational prediction tools in order to enable applications such as vaccine design, cancer neoantigen identification, development of diagnostics and removal of unwanted immune responses against protein therapeutics. Most T cell epitope prediction tools are based on machine learning algorithms trained on MHC binding or naturally processed MHC ligand elution data. The ability of currently available tools to predict T cell epitopes has not been comprehensively evaluated. In this study, we used a recently published dataset that systematically defined T cell epitopes recognized in vaccinia virus (VACV) infected C57BL/6 mice (expressing H-2Db and H-2Kb), considering both peptides predicted to bind MHC or experimentally eluted from infected cells, making this the most comprehensive dataset of T cell epitopes mapped in a complex pathogen. We evaluated the performance of all currently publicly available computational T cell epitope prediction tools to identify these major epitopes from all peptides encoded in the VACV proteome. We found that all methods were able to improve epitope identification above random, with the best performance achieved by neural network-based predictions trained on both MHC binding and MHC ligand elution data (NetMHCPan-4.0 and MHCFlurry). Impressively, these methods were able to capture more than half of the major epitopes in the top N = 277 predictions within the N = 767,788 predictions made for distinct peptides of relevant lengths that can theoretically be encoded in the VACV proteome. These performance metrics provide guidance for immunologists as to which prediction methods to use, and what success rates are possible for epitope predictions when considering a highly controlled system of administered immunizations to inbred mice. In addition, this benchmark was implemented in an open and easy to reproduce format, providing developers with a framework for future comparisons against new tools.Fil: Paul, Sinu. La Jolla Institute for Allergy and Immunology; Estados UnidosFil: Croft, Nathan P.. Monash University; AustraliaFil: Purcell, Anthony W.. Monash University; AustraliaFil: Tscharke, David C.. The Australian National University; AustraliaFil: Sette, Alessandro. La Jolla Institute for Allergy and Immunology; Estados UnidosFil: Nielsen, Morten. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Investigaciones Biotecnológicas. Universidad Nacional de San Martín. Instituto de Investigaciones Biotecnológicas; Argentina. Technical University of Denmark; DinamarcaFil: Peters, Bjoern. La Jolla Institute for Allergy and Immunology; Estados UnidosPublic Library of Science2020-05info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/112153Paul, Sinu; Croft, Nathan P.; Purcell, Anthony W.; Tscharke, David C.; Sette, Alessandro; et al.; Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system; Public Library of Science; Plos Computational Biology; 16; 5; 5-2020; 1-181553-7358CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://dx.plos.org/10.1371/journal.pcbi.1007757info:eu-repo/semantics/altIdentifier/doi/10.1371/journal.pcbi.1007757info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-10T13:06:36Zoai:ri.conicet.gov.ar:11336/112153instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-10 13:06:36.765CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
title Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
spellingShingle Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
Paul, Sinu
MHC
Benchmark
Immunoinformatics
title_short Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
title_full Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
title_fullStr Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
title_full_unstemmed Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
title_sort Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system
dc.creator.none.fl_str_mv Paul, Sinu
Croft, Nathan P.
Purcell, Anthony W.
Tscharke, David C.
Sette, Alessandro
Nielsen, Morten
Peters, Bjoern
author Paul, Sinu
author_facet Paul, Sinu
Croft, Nathan P.
Purcell, Anthony W.
Tscharke, David C.
Sette, Alessandro
Nielsen, Morten
Peters, Bjoern
author_role author
author2 Croft, Nathan P.
Purcell, Anthony W.
Tscharke, David C.
Sette, Alessandro
Nielsen, Morten
Peters, Bjoern
author2_role author
author
author
author
author
author
dc.subject.none.fl_str_mv MHC
Benchmark
Immunoinformatics
topic MHC
Benchmark
Immunoinformatics
purl_subject.fl_str_mv https://purl.org/becyt/ford/3.4
https://purl.org/becyt/ford/3
dc.description.none.fl_txt_mv T cell epitope candidates are commonly identified using computational prediction tools in order to enable applications such as vaccine design, cancer neoantigen identification, development of diagnostics and removal of unwanted immune responses against protein therapeutics. Most T cell epitope prediction tools are based on machine learning algorithms trained on MHC binding or naturally processed MHC ligand elution data. The ability of currently available tools to predict T cell epitopes has not been comprehensively evaluated. In this study, we used a recently published dataset that systematically defined T cell epitopes recognized in vaccinia virus (VACV) infected C57BL/6 mice (expressing H-2Db and H-2Kb), considering both peptides predicted to bind MHC or experimentally eluted from infected cells, making this the most comprehensive dataset of T cell epitopes mapped in a complex pathogen. We evaluated the performance of all currently publicly available computational T cell epitope prediction tools to identify these major epitopes from all peptides encoded in the VACV proteome. We found that all methods were able to improve epitope identification above random, with the best performance achieved by neural network-based predictions trained on both MHC binding and MHC ligand elution data (NetMHCPan-4.0 and MHCFlurry). Impressively, these methods were able to capture more than half of the major epitopes in the top N = 277 predictions within the N = 767,788 predictions made for distinct peptides of relevant lengths that can theoretically be encoded in the VACV proteome. These performance metrics provide guidance for immunologists as to which prediction methods to use, and what success rates are possible for epitope predictions when considering a highly controlled system of administered immunizations to inbred mice. In addition, this benchmark was implemented in an open and easy to reproduce format, providing developers with a framework for future comparisons against new tools.
Fil: Paul, Sinu. La Jolla Institute for Allergy and Immunology; Estados Unidos
Fil: Croft, Nathan P.. Monash University; Australia
Fil: Purcell, Anthony W.. Monash University; Australia
Fil: Tscharke, David C.. The Australian National University; Australia
Fil: Sette, Alessandro. La Jolla Institute for Allergy and Immunology; Estados Unidos
Fil: Nielsen, Morten. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Investigaciones Biotecnológicas. Universidad Nacional de San Martín. Instituto de Investigaciones Biotecnológicas; Argentina. Technical University of Denmark; Dinamarca
Fil: Peters, Bjoern. La Jolla Institute for Allergy and Immunology; Estados Unidos
description T cell epitope candidates are commonly identified using computational prediction tools in order to enable applications such as vaccine design, cancer neoantigen identification, development of diagnostics and removal of unwanted immune responses against protein therapeutics. Most T cell epitope prediction tools are based on machine learning algorithms trained on MHC binding or naturally processed MHC ligand elution data. The ability of currently available tools to predict T cell epitopes has not been comprehensively evaluated. In this study, we used a recently published dataset that systematically defined T cell epitopes recognized in vaccinia virus (VACV) infected C57BL/6 mice (expressing H-2Db and H-2Kb), considering both peptides predicted to bind MHC or experimentally eluted from infected cells, making this the most comprehensive dataset of T cell epitopes mapped in a complex pathogen. We evaluated the performance of all currently publicly available computational T cell epitope prediction tools to identify these major epitopes from all peptides encoded in the VACV proteome. We found that all methods were able to improve epitope identification above random, with the best performance achieved by neural network-based predictions trained on both MHC binding and MHC ligand elution data (NetMHCPan-4.0 and MHCFlurry). Impressively, these methods were able to capture more than half of the major epitopes in the top N = 277 predictions within the N = 767,788 predictions made for distinct peptides of relevant lengths that can theoretically be encoded in the VACV proteome. These performance metrics provide guidance for immunologists as to which prediction methods to use, and what success rates are possible for epitope predictions when considering a highly controlled system of administered immunizations to inbred mice. In addition, this benchmark was implemented in an open and easy to reproduce format, providing developers with a framework for future comparisons against new tools.
publishDate 2020
dc.date.none.fl_str_mv 2020-05
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/112153
Paul, Sinu; Croft, Nathan P.; Purcell, Anthony W.; Tscharke, David C.; Sette, Alessandro; et al.; Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system; Public Library of Science; Plos Computational Biology; 16; 5; 5-2020; 1-18
1553-7358
CONICET Digital
CONICET
url http://hdl.handle.net/11336/112153
identifier_str_mv Paul, Sinu; Croft, Nathan P.; Purcell, Anthony W.; Tscharke, David C.; Sette, Alessandro; et al.; Benchmarking predictions of MHC class I restricted T cell epitopes in a comprehensively studied model system; Public Library of Science; Plos Computational Biology; 16; 5; 5-2020; 1-18
1553-7358
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://dx.plos.org/10.1371/journal.pcbi.1007757
info:eu-repo/semantics/altIdentifier/doi/10.1371/journal.pcbi.1007757
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Public Library of Science
publisher.none.fl_str_mv Public Library of Science
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1842980277701312512
score 12.993085