HEADLINE: Benchmarking VLMs in Head CT Reporting¶

You can find more detailed information about the challenge at headline.ccibonn.ai.

1. Dataset Access¶

Please also sign up at the following link to get access to the dataset, if you have not already done so: HEADLINE Registration.

2. Submission on Grand Challenge¶

Method submission will be handled through this page on Grand Challenge. Please sign up for a Grand Challenge account and then click 'Join' at the top right for this challenge. The requests will be approved manually, so please be patient.

An overview of the phases can be found here. Details on the submission formats will be available on the submission page for each phase.

• Validation phase: open now. Submit your generated reports for the 500 validation studies.
• Sanity check phase: open now. Submit your algorithm container to check that all runs smoothly. See here for sample code for handling the required inputs and outputs.
• Test phase: planned to open soon.

Important: Your Grand Challenge account must be verified to submit a solution to the sanity check & test phases. Please find more details here. Verification is handled by the Grand Challenge team (not by challenge organisers) and can take a couple of days. In the meantime, though, you should already be able to submit to the validation phase, which is a "predictions-only" phase.

3. Short Paper on OpenReview¶

Finally, when submitting your final method during the test phase, you should also submit a short paper with the method description - this will be through OpenReview here.