Goals

• Learn how to customize information seeking systems (IR and QA) for optimal performance on any queries.
• Initially focus on automatic expansion techniques for relevance feedback and pseudo-relevance feedback.
• Extend this to other tools and apply the results to QA in different ways, such as including support for recall oriented QA.

Areas of Focus:

1. Individual Topic Failure Analysis - Intensive manual examination of retrieval runs (1-2 topics per day) to understand why systems fail, and what information systems need to succeed.
2. Retrieval/QA Multi-system Experiments - Run multiple systems in a controlled fashion to isolate system factors for later manual and statistical analysis.
3. Analyze Multi-system Experiments - Discover how system factors affect reliability of system performance,
4. Statistical Analysis of Multi-System Experiments - Statistically isolate system and topic performance issues. Discover if we can statistically categorize topics in a fashion that matches the failure causes of the individual topic failure analysis.
5. Develop publically available tools in support of all 4 areas above.