In the context of the escalating use of social media in Arabic-speaking countries, driven by improved internet access, affordable smartphones, and a growing digital connectivity trend, this study addresses a significant challenge: the widespread dissemination of fake news. The ease and rapidity of spreading information on social media, coupled with a lack of stringent fact-checking measures, exacerbate the issue of misinformation. Our study examines how language features, especially Named Entity Recognition (NER) features, play a role in detecting fake news. We built two models: an AraBERT Multi-task Learning (MTL) based one for classifying Arabic fake news, and a token classification model that focuses on fake news NER features. The study combines embedding vectors from these models using an embedding fusion technique and applies machine learning algorithms for fake news detection in Arabic. We also introduced a feature selection algorithm named RLTTAO based on improving the Triangulation Topology Aggregation Optimizer (TTAO) performance using Reinforcement Learning and random opposition-based learning to enhance the performance by selecting relevant features, thereby improving the fusion process. Our results show that incorporating NER features enhances the accuracy of fake news detection in 5 out of 7 datasets, with an average improvement of 1.62%.