SHINRA2021-ML: Results Submission

For each target language, a group can submit up to three runs based on different methods. (Please contact us if you need our special consideration.) The difference between the methods should be described clearly in your system description report.
Run result files should be grouped by method and compressed in a zip file. Use the submission form to upload the zip file.
If you might have some trouble and submitted more than one versions for a method, the final version will be used for evaluation.

Each run result file for a target language should be named in the format:
[language code].json　ex： en.json

The zip file to upload should be named in the format: [Group ID]_[Method ID]_[Version ID].zip

　ex：’RIKEN_BERT_001.zip’

Group ID: The group ID used for the task registration.
Method ID: The ID to distinguish between the methods used for your runs. Specify the ID using any combination of ASCII alphanumeric characters other than ‘_’. ex：’BERT’ .
Version ID: The version number used to determine the latest version when you resubmit results based on the same method for some reason. The ID should be specified as 3 digits with leading zeros and the incremental count of submission. ex: ‘001’, ‘002’, ‘003’, …

As for submission format, please see the data format page.

Please submit the run result files using the following submission form. Read the How to fill in the submission form beforehand.

Group ID: Fill in your Group ID used for registration.
E-mail: Fill in the e-mail address of your representative (principal investigator) or the contact person.
Brief description of the method: Fill in the brief description of the method.
Submission file: Specify the zip file to submit.
Data used for the run: All the task data are listed with checkboxes. Check all that apply.
Other external info used for the run: Make a list of all the external info used for the run including the pre-trained model.
Confirmation: Check that your run files include prediction for all the target data.

If you have any questions, contact the followings:

Slack among the participants and organizers:
http://shinra2021-ml.slack.com