Benchmarking News Recommendation in the Era of Green AI

Mar. 6^th, 2024: Got accepted by TheWebConf ‘24. It is a revised and condensed version of the previous work titled Only Encode Once: Making Content-based News Recommender Greener. While the core ideas and results remain consistent, the presentation scope have been modified for brevity and clarity. For the full details and extended discussions, please refer to the original long paper at here.

Qijiong LIU^*, Jieming ZHU^*, Quanyu DAI, Xiao-Ming WU^#
*Equal contribution (co-first authors).
[Code] [Paper]

Abstract

Over recent years, news recommender systems have gained significant attention in both academia and industry, emphasizing the need for a standardized benchmark to evaluate and compare the performance of these systems. Concurrently, Green AI advocates for reducing the energy consumption and environmental impact of machine learning. To address these concerns, we introduce the first Green AI benchmarking framework for news recommendation, known as GreenRec, and propose a metric for assessing the tradeoff between recommendation accuracy and efficiency. Our benchmark encompasses 30 base models and their variants, covering traditional end-to-end training paradigms as well as our proposed efficient only-encode-once (OLEO) paradigm. Through experiments consuming 2000 GPU hours, we observe that the OLEO paradigm achieves competitive accuracy compared to state-of-the-art end-to-end paradigms and delivers up to a 2992% improvement in sustainability metrics. We have released the source code and data for reproducibility.

Citation

@misc{liu2024benchmarking,
      title={Benchmarking News Recommendation in the Era of Green AI}, 
      author={Qijiong Liu and Jieming Zhu and Quanyu Dai and Xiao-Ming Wu},
      booktitle = {Proceedings of the ACM Web Conference 2024},
      month = {may},
      year = {2024},
      address = {Singapore}
}