Image super-resolution method based on the interactive fusion of transformer and CNN features

Faculty Computer Science Year: 2023
Type of Publication: ZU Hosted Pages:
Authors:
Journal: The Visual Computer Springer Nature Volume:
Keywords : Image super-resolution method based , , interactive fusion    
Abstract:
Recently, Transformer has achieved outstanding performance in the field of computer vision, where the ability to capture global context is crucial for image super-resolution (SR) reconstruction. Unlike convolutional neural networks (CNNs), Transformers lack a local mechanism for information exchange within local regions. To address this problem, we propose a U-Net network (TCSR) based on Transformer and CNN feature interaction fusion, which has skip-connections for local and global semantic feature learning. The TCSR takes the transformer blocks as the basic module of the U-Net architecture and gradually extracts multi-scale feature information while modeling global long-range dependencies. First, we propose an efficient multi-head shift transposed attention to improve the internal structure of the transformer and thus recover sufficient texture details. In addition, a feature enhancement module is inserted in the skip-connection positions to capture local structural information at different levels. Finally, to further exploit the contextual information from features, we use a locally enhanced feed-forward layer to replace the feed-forward network in each Transformer, which incorporates local feature representation into the global context. Powered by these designs, TCSR has the ability to capture both local and global dependencies for image HR reconstruction. Extensive experiments showed that compared with other state-of-the-art SR algorithms, our proposed method could effectively recover the details of the image, producing significant improvements in both visual effect and image quality.
   
     
 
       

Author Related Publications

  • Wael Said AbdelMageed Mohamed, "A big data approach to sentiment analysis using greedy feature selection with cat swarm optimization-based long short-term memory neural networks", Springer Nature, 2018 More
  • Wael Said AbdelMageed Mohamed, "High-Precision Brain Tumor Diagnosis Using SECNN-MNet Framework and Explainable AI", Springer Nature Link, 2025 More
  • Wael Said AbdelMageed Mohamed, "Deception and cloud integration: A multi-layered approach for DDoS detection, mitigation, and attack surface minimization in SD-IoT networks", .Elsevier Ltd, 2025 More
  • Wael Said AbdelMageed Mohamed, "Reinforcement Learning for Industrial Automation: A Comprehensive Review of Adaptive Control and Decision-Making in Smart Factories", MDPI, 2025 More
  • Wael Said AbdelMageed Mohamed, "RAUM-GANs: A Multi-Layer GAN-Enhanced Framework for Accurate Multiple Sclerosis Lesion Segmentation in MRI", Nature Portfolio, 2025 More

Department Related Publications

  • Ibrahiem Mahmoud Mohamed Elhenawy, "BERT-CNN: A Deep Learning Model for Detecting Emotions from Text", Tech Science Press, 2021 More
  • Ahmed Raafat Abass Mohamed Saliem, "BERT-CNN: A Deep Learning Model for Detecting Emotions from Text", Tech Science Press, 2021 More
  • Ahmed Raafat Abass Mohamed Saliem, "Using General Regression with Local Tuning for Learning Mixture Models from Incomplete Data Sets", ScienceDirect, 2010 More
  • Ahmed Raafat Abass Mohamed Saliem, "On determining efficient finite mixture models with compact and essential components for clustering data", ScienceDirect, 2013 More
  • Ahmed Raafat Abass Mohamed Saliem, "Unsupervised learning of mixture models based on swarm intelligence and neural networks with optimal completion using incomplete data", ScienceDirect, 2012 More
Tweet