Analyzing User Information Seeking Strategies in Interactive Information Retrieval Systems A Mixed-Methods Evaluation Framework
Keywords:
Interactive Information Retrieval, Information Seeking Strategies, Mixed-Methods Evaluation, User Behavior, Task-Based Search.Abstract
Abstract
Understanding how users adapt their ISS across task types is critical for evaluating and designing effective IIR systems. While traditional evaluations focus on performance metrics like relevance or precision, they often fail to capture user behavior's cognitive and strategic dimensions. This study proposes a mixed-methods evaluation framework to analyze ISS across factual, exploratory, and comparative tasks. A user study involving 30 participants used a custom academic search interface. Data were collected through interaction logs, satisfaction surveys, and post-task reflections.
The findings show that strategy use is highly task-dependent. Factual tasks encouraged direct lookup behaviors, whereas exploratory and comparative tasks involved iterative query refinement, multi-document synthesis, and opportunistic exploration. Satisfaction correlated positively with strategy diversity; expert users demonstrated greater adaptability in complex tasks. Five dominant strategy types were identified and linked to task complexity and user expertise using quantitative log data and qualitative user narratives. This study advances IIR evaluation by offering a replicable, strategy-sensitive framework beyond traditional click-based metrics. The framework provides actionable insights for designing adaptive, user-centered search systems that support a range of strategic behaviors across varying information needs.
Keywords: Interactive Information Retrieval, Information Seeking Strategies, Mixed-Methods Evaluation, User Behavior, Task-Based Search
References
. Adhav, H., & Singh, V. (2024). Modeling Topic Evolution to Steer Interactive Information Search (pp. 586–596). https://doi.org/10.1007/978-3-031-12700-7_60
. Aloteibi, S. (2020). A user-centred approach to information retrieval. Apollo - University of Cambridge Repository.
. Aqle, A., Al-Thani, D., & Jaoua, A. (2022). Can search result summaries enhance the web search efficiency and experiences of the visually impaired users? Universal Access in the Information Society, 21(1), 171–192. https://doi.org/10.1007/s10209-020-00777-w
. Athukorala, K., G?owacka, D., Jacucci, G., Oulasvirta, A., & Vreeken, J. (2016). Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks. Journal of the Association for Information Science and Technology, 67(11), 2635–2651. https://doi.org/10.1002/asi.23617
. Aula, A., Khan, R. M., & Guan, Z. (2010). How does search behavior change as search becomes more difficult? Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 35–44. https://doi.org/10.1145/1753326.1753333
. Azzopardi, L., Breuer, T., Engelmann, B., Kreutz, C., MacAvaney, S., Maxwell, D., Parry, A., Roegiest, A., Wang, X., & Zerhoudi, S. (2024). SimIIR 3: A Framework for the Simulation of Interactive and Conversational Information Retrieval. Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, 197–202. https://doi.org/10.1145/3673791.3698427
. Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407–424. https://doi.org/10.1108/eb024320
. Belkin, N. (1995). Cases, Scripts, and Information-Seeking Strategies: On the Design of Interactive Information Retrieval Systems. Expert Systems with Applications, 9(3), 379–395. https://doi.org/10.1016/0957-4174(95)00011-W
. Chen, O., Paas, F., & Sweller, J. (2023). A Cognitive Load Theory Approach to Defining and Measuring Task Complexity Through Element Interactivity. Educational Psychology Review, 35(2), 63. https://doi.org/10.1007/s10648-023-09782-w
. Creswell, J., & Plano, C. (2017). Designing and conducting mixed methods research. (3rd Edition) SAGE Publications.
. Engelmann, B., Breuer, T., Friese, J. I., Schaer, P., & Fuhr, N. (2023). Context-Driven Interactive Query Simulations Based on Generative Large Language Models.
. Ingwersen, P., & Järvelin, K. (2005). The Turn: Integration of information seeking and retrieval in context (Vol. 18). Springer-Verlag. https://doi.org/10.1007/1-4020-3851-8
. Ji, K., Hettiachchi, D., Salim, F. D., Scholer, F., & Spina, D. (2024). Characterizing Information Seeking Processes with Multiple Physiological Signals. https://doi.org/10.1145/3626772.3657793
. Kelly, D. (2009). Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, 3(1–2), 1–224. https://doi.org/10.1561/1500000012
. Kelly, D., Arguello, J., Edwards, A., & Wu, W. (2015). Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework. Proceedings of the 2015 International Conference on The Theory of Information Retrieval, 101–110. https://doi.org/10.1145/2808194.2809465
. Kiesel, J., Gohsen, M., Mirzakhmedova, N., Hagen, M., & Stein, B. (2024). Simulating Follow-Up Questions in Conversational Search (pp. 382–398). https://doi.org/10.1007/978-3-031-56060-6_25
. Kuhlthau, C. (1993). Seeking meaning: A process approach to library and information services. Libraries Unlimited.
. Li, Y., & Belkin, N. J. (2008). A faceted approach to conceptualizing tasks in information seeking. Information Processing & Management, 44(6), 1822–1837. https://doi.org/10.1016/j.ipm.2008.07.005
. Maxwell, D., & Azzopardi, L. (2016). Simulating Interactive Information Retrieval. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1141–1144. https://doi.org/10.1145/2911451.2911469
. Pradhan, D., & Maharana, B. (2022). Faceted Search Interface for Library Portal to Support Scholarly Information Seeking from E-Resource Collection: A Case of NIT Rourkela. International Conference on Libraries of the Future: Emerging Trends, KIIT Bhubaneswar and MANLIBNET.
. Reinanda, R., Meij, E., & de Rijke, M. (2020). Knowledge Graphs: An Information Retrieval Perspective. Foundations and Trends® in Information Retrieval, 14(4), 289–444. https://doi.org/10.1561/1500000063
. Sanderson, M. (2010). Test Collection Based Evaluation of Information Retrieval Systems. Foundations and Trends® in Information Retrieval, 4(4), 247–375. https://doi.org/10.1561/1500000009
Downloads
Published
Versions
- 2025-09-01 (3)
- 2025-12-26 (2)
- 2025-12-21 (1)
How to Cite
Issue
Section
License
Copyright (c) 2025 University of Sindh Journal of Information and Communication Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
University of Sindh Journal of Information and Communication Technology (USJICT) follows an Open Access Policy under Attribution-NonCommercial CC-BY-NC license. Researchers can copy and redistribute the material in any medium or format, for any purpose. Authors can self-archive publisher's version of the accepted article in digital repositories and archives.
Upon acceptance, the author must transfer the copyright of this manuscript to the Journal for publication on paper, on data storage media and online with distribution rights to USJICT, University of sindh, Jamshoro, Pakistan. Kindly download the copyright for below and attach as a supplimentry file during article submission
