Do you have an image de-identification algorithm that you’d like to test and evaluate against a published benchmark? Take advantage of the Medical Image De-Identification Benchmark (MIDI-B), which in collaboration with Sage Bionetworks, provides a platform for testing your tool against a well-defined benchmark whose data sets, answer keys, and a validation script are open and available.

This benchmark uses images from the MIDI data set, a large and diverse collection of multisite and multimodality clinical DICOM images selected from public collections in NCI’s The Cancer Imaging Archive, which have personal data replaced with corresponding synthetic identifiers.

The data and submission procedures are the same as for the MIDI-B Challenge hosted in 2024. As a team (individual or group), you are allowed to submit de-identified data once per day for both the validation and test queues. Additionally, there have been some changes to the original answer keys, which may result in slightly different performance outcomes than the original MIDI-B Challenge.

Submission queues for the MIDI-B:

• “MIDI-B De-identification - Validation - Continuous Benchmarking”
• “MIDI-B De-identification - Test - Continuous Benchmarking”

To assess the performance of your de-identification algorithm, the MIDI-B platform will automatically verify the removal of protected health information and personally identifiable information from the DICOM header and the pixel data. It does this through reports that compare executed actions against required actions, the number of errors, and compliance against the DICOM Validator.