Selecting Actions for Resource-bounded Information Extraction using Reinforcement Learning

Given a database with missing or uncertain content, our goal is to correct and ll the database by extracting specic in- formation from a large corpus such as the Web, and to do so under resource limitations. We formulate the informa- tion gathering task as a series of choices among alternative, resource-consuming actions and use reinforcement learning to select the best action at each time step. We use tempo- ral dierence q-learning method to train the function that selects these actions, and compare it to an online, error- driven algorithm called SampleRank. We present a system that nds information such as email, job title and depart- ment aliation for the faculty at our university, and show that the learning-based approach accomplishes this task e- ciently under a limited action budget. Our evaluations show that we can obtain 92.4% of the nal F1, by only using 14.3% of all possible actions.