The emergence of wearable medical devices and wireless body area networks (WBANs) has enabled continuous, real-time patient monitoring. These systems generate large volumes of health data, requiring low-latency and reliable processing for timely interventions. However, local processing is often inefficient due to the energy and computational limitations of mobile devices. Offloading tasks to edge computing and cloud resources offers a promising alternative. Nonetheless, optimizing offloading decisions in dynamic healthcare scenarios remains challenging due to heterogeneous task requirements and varying computational resources. This paper presents a hierarchical actor-critic task offloading approach (HACTO), a deep-reinforcement-learning framework designed to enhance the efficiency and adaptability of task offloading in healthcare scenarios. By introducing a hierarchical decision structure, HACTO reduces complexity and improves learning performance. The problem is modeled as a Markov decision process and solved using the deep deterministic policy gradient algorithm. HACTO jointly optimizes task offloading with respect to three objectives: meeting task deadlines, minimizing the energy consumption of mobile devices, and reducing resource usage costs. Our experimental results show that HACTO outperforms traditional and deep-reinforcement-learning-based offloading strategies, making it a promising solution for intelligent task offloading in resource-constrained WBAN environments.