PREC: Boosting Deep CTR Prediction with a Plug-and-Play Pre-trainer for News Recommendation

Aug. 17^th, 2022: Got accepted by COLING ‘22! This marks a significant milestone in my research career as it represents the first paper accepted with me as the primary author.

Qijiong LIU, Jieming ZHU, Quanyu DAI, Xiao-Ming WU^#
[Code] [Paper]

Abstract

Understanding news content is critical to improving the quality of news recommendation. To achieve this goal, recent studies have attempted to apply pre-trained language models (PLMs) such as BERT for semantic-enhanced news recommendation. Despite their great success in offline evaluation, it is still a challenge to apply such large PLMs in real-time ranking model due to the stringent requirement in inference and updating time. To bridge this gap, we propose a plug-and-play pre-trainer, namely PREC, to learn both user and news encoders through multi-task pre-training. Instead of directly leveraging sophisticated PLMs for end-to-end inference, we focus on how to use the derived user and item representations to boost the performance of conventional lightweight models for click-through-rate prediction. This enables efficient online inference as well as compatibility to conventional models, which would significantly ease the practical deployment. We validate the effectiveness of PREC through both offline evaluation on public datasets and online A/B testing in an industrial application.

Citation

@inproceedings{liu2022prec,
    title = "Boosting Deep {CTR} Prediction with a Plug-and-Play Pre-trainer for News Recommendation",
    author = "Liu, Qijiong  and
      Zhu, Jieming  and
      Dai, Quanyu  and
      Wu, Xiaoming",
    booktitle = "Proceedings of the 29th International Conference on Computational Linguistics",
    month = oct,
    year = "2022",
    address = "Gyeongju, Republic of Korea",
    publisher = "International Committee on Computational Linguistics",
    url = "https://aclanthology.org/2022.coling-1.249",
    pages = "2823--2833"
}