Analyzing User Information Seeking Strategies in Interactive Information Retrieval Systems A Mixed-Methods Evaluation Framework

Abstract

Abstract

Understanding how users adapt their ISS across task types is critical for evaluating and designing effective IIR systems. While traditional evaluations focus on performance metrics like relevance or precision, they often fail to capture user behavior's cognitive and strategic dimensions. This study proposes a mixed-methods evaluation framework to analyze ISS across factual, exploratory, and comparative tasks. A user study involving 30 participants used a custom academic search interface. Data were collected through interaction logs, satisfaction surveys, and post-task reflections.

The findings show that strategy use is highly task-dependent. Factual tasks encouraged direct lookup behaviors, whereas exploratory and comparative tasks involved iterative query refinement, multi-document synthesis, and opportunistic exploration. Satisfaction correlated positively with strategy diversity; expert users demonstrated greater adaptability in complex tasks. Five dominant strategy types were identified and linked to task complexity and user expertise using quantitative log data and qualitative user narratives. This study advances IIR evaluation by offering a replicable, strategy-sensitive framework beyond traditional click-based metrics. The framework provides actionable insights for designing adaptive, user-centered search systems that support a range of strategic behaviors across varying information needs.

Keywords: Interactive Information Retrieval, Information Seeking Strategies, Mixed-Methods Evaluation, User Behavior, Task-Based Search