How to Participate


  • January 2020             Data release
  • July 31, 2020              Registration & Result submission deadline
  • August 20, 2020       Evaluation results due back to participants
  • Nov 1, 2020                 System description report Deadline
  • December 2020        NTCIR-15 Conference (NII, Tokyo)
  • May 2020                    Proceedings will be published

    Just wanna peek: Willing to know if it’s interesting

    If you want to see what it would be like, you can download training data (Trial Datasets“),  which includes 30 language Wikipedia entities classified into categories.

    SHINRA2020-ML aims to categorize 30 language Wikipedia entities into ENE (Extended Named Entity) categories. When you participate in the task, you can choose at least one target language. The training data will be used to categorize all the remaining Wikipedia entities of the language. 

    Procedure of Full-Participation

    1. Registration through NTCIR-15
    2. Account Creation and Data download
    3. Result Submission
    4. Conference (NTCIR-15)

    1.  Registration through NTCIR-15

    SHINRA2020-ML is a task in NTCIR-15 of the NTCIR evaluation workshop series. If you would like to participate in the task, please register through the following page .

        NTCIR-15 Registration page

    2.  Account Creation and Data Download

    You need an account to get any of the data provided for the task. Please create your SHINRA account at SHINRA: Sign in page. When you download the data, you have to login using the account.  

    The training data (already categorized Wikipedia entities) and target data (the Wikipedia entities you will categorize) are available from Shinra2020-ml Data download page. 

    3. Result Submission

    Please submit the data through the specified page (TBA). 

    4.  Conference (NTCIR-15)

    The results can be presented at the NTCIT-15 conference. Further details will be announced later.