1st CFP MedProcNER track: Automatic detection of Medical procedures in text (BioASQ/CLEF2023)

MedProcNER (Medical Procedure Named Entity Recognition & Linking)

TRAINING SET + ANNOTATION GUIDELINES RELEASE



URLs:
- Web: https://temu.bsc.es/medprocner/
- Corpus: https://doi.org/10.5281/zenodo.7817745
- Guidelines: https://doi.org/10.5281/zenodo.7817666
- Registration: https://temu.bsc.es/medprocner/registration/


Description:

The recent use and exploitation of large language models (LLMs) and transformer-based technologies have resulted in considerable improvements in clinical NLP & health data processing systems, in particular for content in English, but also increasingly for other languages. Efficient semantic annotation strategies/named entity recognition of key clinical concepts, such as diseases, medications or adverse events are critical for medical text mining applications, including QA, information extraction, predictive modeling or even generative AI.

Despite being at the core of clinical practice and healthcare, only limited attempts focussed on the automatic detection and normalization of medical procedures from texts. These were often restricted to very specialized types of procedures (e.g. medical imaging). Some efforts like the Merlot corpus included medical procedures, but were not exclusively focused on this key clinical concept type.

Thus, there is a clear need to foster the development of concept recognition systems for medical procedures, being essential to characterize existing treatment options, diagnostic procedures of patients as well as analyzing key aspects of therapeutic or preventive techniques associated with patient care. Procedures are also relevant for clinical coding efforts of electronic health records.

To promote the development, evaluation and use of clinical concept recognition systems of medical procedure from clinical texts we organize the MedProcNER task as part of the BioASQ/CLEF 2023 evaluation initiative.

For this track a large Gold Standard corpus of clinical case texts in Spanish manually annotated through a collaborative effort between clinicians, clinical coding experts and linguists will be released. Additionally a multilingual Silver Standard version of the MedProcNER corpus in English and other languages including French, Italian or Portuguese will be published as well as cross-mappings to ICD-10 and MeSH codes.

Aligned with different practical end user scenarios three subtasks will be posed:

1. Procedure-NER (Clinical Procedure Recognition) subtask: requires the automatic detection of all clinical procedures mentioned in clinical case documents (their start and end character positions).

2. Procedure-Norm (Clinical Procedure Normalization) subtask: requires the automatic detection of all clinical procedures mentioned in clinical case documents (their start and end character positions) together with their corresponding SNOMED CT concept code.

3. Procedure-Indexing (Clinical Procedure-based Document Indexing) subtask: similar to semantic indexing or clinical coding tasks, this subtask requires teams to automatically assign to each document their corresponding list of clinical procedure SNOMED CT codes.


SCHEDULE:
- Train set release Procedure-NER: April 11th, 2023
- Train set release Procedure-Norm/Indexing & : April 21st, 2023
- Test set release (start of evaluation period): April 24th, 2023
- End of evaluation period (system submissions): May 15th, 2023
- Working papers submission: June 5th, 2023
- Notification of acceptance (peer-reviews): June 23rd, 2023
- Camera-ready system descriptions: July 7th, 2023
- BioASQ workshop at CLEF 2023: Thessaloniki, Greece, September 18th-21st, 2023 (https://clef2023.clef-initiative.eu)


Publications & conference

Following previous BioASQ/CLEF efforts, participating teams will be invited to contribute a short systems description paper for the CLEF 2023 proceedings, and to give a short presentation of their approach at the BioASQ workshop at the CLEF 2023 conference (18-21 September, Thessaloniki, Greece)

Organizers:

Martin Krallinger, Barcelona Supercomputing Center, Spain
Salvador Lima, Barcelona Supercomputing Center, Spain
Eulàlia Farré, Barcelona Supercomputing Center, Spain
Luis Gascó, Barcelona Supercomputing Center, Spain
Anastasios Nentidis, National Center for Scientific Research Demokritos, Greece
Anastasia Krithara, National Center for Scientific Research Demokritos, Greece