Human Activity Recognition (HAR) and fall detection, as applications within the field of biomedical signal processing, are increasingly pivotal in enhancing patient care, preventive healthcare, and rehabilitation. Fall therapy is one of the most expensive treatments, usually taking a long time to complete. A single fall accident might result in serious injuries, long-term incapacity, or even death. As a result, a reliable and cost-effective fall detection system is essential. Wearable sensors have received wide attention due to their availability and capability to capture different human motions. Thus, in the current study, we develop a comprehensive HAR system for multi-classification tasks to recognize several human actions, such as walking, sitting, standing, falling, and others. At the same time, a binary classification of this model is developed to recognize fall and non-fall actions, which can be used to track elderly actions and send an alert in case of falling to do necessary rescue actions. The developed system is built using a Parallel Convolutional Neural Network and Transformer-based architecture (PCNN-Transformer). PCNN-Transformer benefits from the parallel architecture and the residual mapping mechanism to learn temporal feature representations from the sensors’ data. The CNN blocks are aligned in parallel alongside several Transformer-based encoders, followed by a concatenation operation to sum up the extracted features from the input data ( sensors data). Moreover, the CNN blocks implement a residual mapping mechanism to reduce the model complexity and training time. The proposed model is tested using several open-source datasets: SisFall, UniMib-SHAR, and MobiAct. It recorded high accuracy rates compared to several deep learning models. For instance, in the binary classification (fall detection), the proposed model achieved an average accuracy of 99.95%, 98.68%, and 99.71% for SisFall, UniMib-SHAR, and MobiAct, respectively.