Image super-resolution method based on the interactive fusion of transformer and CNN features

Faculty Computer Science Year: 2023
Type of Publication: ZU Hosted Pages:
Authors:
Journal: The Visual Computer Springer Nature Volume:
Keywords : Image super-resolution method based , , interactive fusion    
Abstract:
Recently, Transformer has achieved outstanding performance in the field of computer vision, where the ability to capture global context is crucial for image super-resolution (SR) reconstruction. Unlike convolutional neural networks (CNNs), Transformers lack a local mechanism for information exchange within local regions. To address this problem, we propose a U-Net network (TCSR) based on Transformer and CNN feature interaction fusion, which has skip-connections for local and global semantic feature learning. The TCSR takes the transformer blocks as the basic module of the U-Net architecture and gradually extracts multi-scale feature information while modeling global long-range dependencies. First, we propose an efficient multi-head shift transposed attention to improve the internal structure of the transformer and thus recover sufficient texture details. In addition, a feature enhancement module is inserted in the skip-connection positions to capture local structural information at different levels. Finally, to further exploit the contextual information from features, we use a locally enhanced feed-forward layer to replace the feed-forward network in each Transformer, which incorporates local feature representation into the global context. Powered by these designs, TCSR has the ability to capture both local and global dependencies for image HR reconstruction. Extensive experiments showed that compared with other state-of-the-art SR algorithms, our proposed method could effectively recover the details of the image, producing significant improvements in both visual effect and image quality.
   
     
 
       

Author Related Publications

  • Wael Said AbdelMageed Mohamed, "A big data approach to sentiment analysis using greedy feature selection with cat swarm optimization-based long short-term memory neural networks", Springer Nature, 2018 More
  • Wael Said AbdelMageed Mohamed, "High-Precision Brain Tumor Diagnosis Using SECNN-MNet Framework and Explainable AI", Springer Nature Link, 2025 More
  • Wael Said AbdelMageed Mohamed, "Deception and cloud integration: A multi-layered approach for DDoS detection, mitigation, and attack surface minimization in SD-IoT networks", .Elsevier Ltd, 2025 More
  • Wael Said AbdelMageed Mohamed, "Reinforcement Learning for Industrial Automation: A Comprehensive Review of Adaptive Control and Decision-Making in Smart Factories", MDPI, 2025 More
  • Wael Said AbdelMageed Mohamed, "RAUM-GANs: A Multi-Layer GAN-Enhanced Framework for Accurate Multiple Sclerosis Lesion Segmentation in MRI", Nature Portfolio, 2025 More

Department Related Publications

  • Ahmed Salah Mohamed Mostafa, "Cluster-Distribute-Align-Merge: A General Algorithm to Speed Up Multiple Sequence Alignment on Multi-Core Computers", Journal of Computational and Theoretical Nanoscience, 2014 More
  • Zaher Awad Aboelenieen Elhendy, "NEW APPROACH TO IMAGE EDGE DETECTION BASED ON QUANTUM ENTROPY", JOURNAL OF RUSSIAN LASER RESEARCH, 2016 More
  • Sarah AbdelRazek Ahmed AbdulHameid, "Cloud Storage Forensics: Survey", International Journal of Engineering Trends and Technology (IJETT), 2017 More
  • Doaa El-Shahat Barakat Mohammed, "A modified hybrid whale optimization algorithm for the scheduling problem in multimedia data objects", Wiley online library, 2019 More
  • Abdallah Gamal abdallah mahmoud, "A novel model for evaluation Hospital medical care systems based on plithogenic sets", Elsevier B.V., 2019 More
Tweet