QA4MRE at CLEF2012The main objective of this exercise is to develop a methodology for evaluating Machine Reading systems through Question Answering and Reading Comprehension Tests. Systems should be able to extract knowledge from large volumes of text and use this knowledge to answer questions. This methodology should allow the comparison of systems' performance and the study of the best approaches.
TASK OVERVIEW The Machine Reading task addresses the problem of building a bridge between knowledge encoded as natural text and the formal reasoning systems that need such knowledge. The knowledge contained in naturally occurring texts should be made available in forms that machines can use to perform some kind of reasoning and expand the system's inference capabilities. In contrast to text mining (or text harvesting, sometimes also called macro-reading), where the system reads and combines evidence from hundreds or thousands of texts, MR is the task of obtaining an in-depth understanding of just one, or a small number, of texts. In fact, the task will focus on the reading of single documents, where correct answers require some inference and the consideration of previously acquired background knowledge.
PILOT TASK Beside the Main Task, also two pilot tasks are offered this year at QA4MRE
Both pilot tasks will be offered in English only and will be coordinated by the University of Antwerp, Belgium. |
