The overview of the BioRED (Biomedical Relation Extraction Dataset) track at BioCreative VIII.

Journal: Database : the journal of biological databases and curation
Published Date: Aug 8, 2024

Abstract

The BioRED track at BioCreative VIII calls for a community effort to identify, semantically categorize, and highlight the novelty factor of the relationships between biomedical entities in unstructured text. Relation extraction is crucial for many biomedical natural language processing (NLP) applications, from drug discovery to custom medical solutions. The BioRED track simulates a real-world application of biomedical relationship extraction, and as such, considers multiple biomedical entity types, normalized to their specific corresponding database identifiers, as well as defines relationships between them in the documents. The challenge consisted of two subtasks: (i) in Subtask 1, participants were given the article text and human expert annotated entities, and were asked to extract the relation pairs, identify their semantic type and the novelty factor, and (ii) in Subtask 2, participants were given only the article text, and were asked to build an end-to-end system that could identify and categorize the relationships and their novelty. We received a total of 94 submissions from 14 teams worldwide. The highest F-score performances achieved for the Subtask 1 were: 77.17% for relation pair identification, 58.95% for relation type identification, 59.22% for novelty identification, and 44.55% when evaluating all of the above aspects of the comprehensive relation extraction. The highest F-score performances achieved for the Subtask 2 were: 55.84% for relation pair, 43.03% for relation type, 42.74% for novelty, and 32.75% for comprehensive relation extraction. The entire BioRED track dataset and other challenge materials are available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/ and https://codalab.lisn.upsaclay.fr/competitions/13377 and https://codalab.lisn.upsaclay.fr/competitions/13378. Database URL: https://ftp.ncbi.nlm.nih.gov/pub/lu/BC8-BioRED-track/https://codalab.lisn.upsaclay.fr/competitions/13377https://codalab.lisn.upsaclay.fr/competitions/13378.

Authors

  • Rezarta Islamaj
    National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, United States.
  • Po-Ting Lai
    National Library of Medicine, National Institutes of Health, Bethesda, MD, USA.
  • Chih-Hsuan Wei
    National Center for Biotechnology Information, Bethesda, MD 20894 USA.
  • Ling Luo
    Department of Epidemiology and Medical Statistics School of Public Health, Guangdong Medical University, Dongguan, Guangdong, China.
  • Tiago Almeida
    Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, Aveiro, Portugal.
  • Richard A A Jonker
    IEETA/DETI, LASI, University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal.
  • Sofia I R Conceição
    Departamento de Informática, Faculdade de Ciências da Universidade de Lisboa, Edifício C6 Campo Grande, Lisbon 1749-016, Portugal.
  • Diana F Sousa
    Departamento de Informática, Faculdade de Ciências, Universidade de Lisboa, Lisboa 1749-016, Portugal.
  • Cong-Phuoc Phan
    Department of Computer Science and Information Engineering, National Cheng Kung University, No.1, University Road, Tainan City 701, Taiwan, Republic of China.
  • Jung-Hsien Chiang
    Department of Computer Science and Information Engineering, National Cheng Kung University, 1, University Road, Tainan City, Taiwan.
  • Jiru Li
    School of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.
  • Dinghao Pan
    School of Computer Science and Technology, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China.
  • Wilailack Meesawad
    Department of Computer Science and Information Engineering, National Central University, No. 300, Zhongda Rd., Zhongli District, Taoyuan City 32001, Taiwan, Republic of China.
  • Richard Tzong-Han Tsai
    Department of Computer Science and Information Engineering, National Central University, Taiwan.
  • M Janina Sarol
    Informatics Programs, University of Illinois Urbana-Champaign, 614 E Daniel Street, Champaign, IL 61820, United States.
  • Gibong Hong
    School of Information Sciences, University of Illinois Urbana-Champaign, Champaign, IL 61820, United States.
  • Airat Valiev
    Higher School of Economics University, 20 Myasnitskaya st, Moscow 101000, Russia.
  • Elena Tutubalina
    Kazan (Volga Region) Federal University, Kazan, Russia.
  • Shao-Man Lee
    Miin Wu School of Computing, National Cheng Kung University, No. 1, University Road, Tainan 701, Taiwan, Republic of China.
  • Yi-Yu Hsu
    National Center for Biotechnology Information, Bethesda, MD, USA.
  • Mingjie Li
    Key Laboratory of Ministry of Education for Genetics, Breeding and Multiple Utilization of Crops, College of Agriculture, Fujian Agriculture and Forestry University, Fuzhou, China.
  • Karin Verspoor
    Dept of Computing and Information Systems, School of Engineering, University of Melbourne, Melbourne, Australia.
  • Zhiyong Lu
    National Center for Biotechnology Information, Bethesda, MD 20894 USA.

Keywords

External Resources

Popular Topics
Recent Journals