NTCIR-15: SHINRA2020-ML System Data Download

You need an account to get any of the data for the task. Please create your SHINRA account at SHINRA: Sign in page.

Evaluation Results:

The SHINRA2020-ML evaluation results are shown in:

Please use Notes on SHINRA2020-ML Evaluation Report as a reference.

Participant Runs (merged):

The run results submitted by participant groups are merged into a file for each language.

In the following example, single lines are split over multiple lines and white space characters are inserted for human readability.
See SHINRA2020-ML: Data Formats for the details of the submission format.

example

{
   "pageid":"00001",
   "results":[
      {
         "team":"GROUP1",
         "method":"Method1",
         "ENEs":[
            {
               "ENE_id":"1.4.5.1",
               "score":0.9245855808258057
            }
         ]
      },
      {
         "team":"GROUP2",
         "method":"Method1",
         "ENEs":[
            {
               "ENE_id":"1.4.5.2",
               "score":0.9245855808258057
            }
         ]
      },
      {
         "team":"GROUP3",
         "method":"Method1",
         "ENEs":[
            {
               "ENE_id":"1.4.5.3",
               "score":0.9245855808258057
            }
         ]
      }
   ]
}

Reference

SHINRA2020-ML Task Overview :
Satoshi Sekine, Masako Nomoto, Kouta Nakayama, Asuka Sumida, Koji Matsuda, and Maya Ando. 2020.
Overview of SHINRA2020-ML Task. In Proceedings of the NTCIR-15 Conference. (to be published)
SHINRA2020-ML website : SHINRA2020-ML