Unifying Online and Counterfactual Learning to Rank (Extended Abstract)

State-of-the-art Learning to Rank (LTR) methods for optimizing ranking systems based on user interactions are divided into online approaches – that learn by direct interaction – and counterfactual approaches – that learn from historical interactions. We propose a novel intervention-aware estimator to bridge this online/counterfactual division. The estimator corrects for the effect of position bias, trust bias, and item-selection bias by using corrections based on the behavior of the logging policy and on online interventions: changes to the logging policy made during the gathering of click data. Our experimental results show that, unlike existing counterfactual LTR methods, the intervention-aware estimator can greatly benefit from online interventions. To the best of our knowledge, this is the first method that is shown to be highly effective in both online and counterfactual scenarios.

The paper on which this an abstract is based can be found here.

Download the paper here.

Download the presentation slides here.

View the poster for this publication here.

Code is available here.

Recommended citation:

H. Oosterhuis and M. de Rijke. "Unifying Online and Counterfactual Learning to Rank (Extended Abstract)." In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence, 2021.