This open access book summarizes the first two decades of the NII Testbeds and Community for
Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global
team of researchers and hosted by the National Institute of Informatics (NII) Japan. The book
is unique in that it discusses not just what was done at NTCIR but also how it was done and
the impact it has achieved. For example in some chapters the reader sees the early seeds of
what eventually grew to be the search engines that provide access to content on the World Wide
Web today's smartphones that can tailor what they show to the needs of their owners and the
smart speakers that enrich our lives at home and on the move. We also get glimpses into how new
search engines can be built for mathematical formulae or for the digital record of a lived
human life. Key to the success of the NTCIR endeavor was early recognition that information
access research is an empirical discipline and that evaluation therefore lay at the core of the
enterprise. Evaluation is thus at the heart of each chapter in this book. They show for
example how the recognition that some documents are more important than others has shaped
thinking about evaluation design. The thirty-three contributors to this volume speak for the
many hundreds of researchers from dozens of countries around the world who together shaped
NTCIR as organizers and participants.This book is suitable for researchers practitioners and
students-anyone who wants to learn about past and present evaluation efforts in information
retrieval information access and natural language processing as well as those who want to
participate in an evaluation task or even to design and organize one.
