Image super-resolution method based on the interactive fusion of transformer and CNN features

Faculty Computer Science Year: 2023
Type of Publication: ZU Hosted Pages:
Authors:
Journal: The Visual Computer Springer Nature Volume:
Keywords : Image super-resolution method based , , interactive fusion    
Abstract:
Recently, Transformer has achieved outstanding performance in the field of computer vision, where the ability to capture global context is crucial for image super-resolution (SR) reconstruction. Unlike convolutional neural networks (CNNs), Transformers lack a local mechanism for information exchange within local regions. To address this problem, we propose a U-Net network (TCSR) based on Transformer and CNN feature interaction fusion, which has skip-connections for local and global semantic feature learning. The TCSR takes the transformer blocks as the basic module of the U-Net architecture and gradually extracts multi-scale feature information while modeling global long-range dependencies. First, we propose an efficient multi-head shift transposed attention to improve the internal structure of the transformer and thus recover sufficient texture details. In addition, a feature enhancement module is inserted in the skip-connection positions to capture local structural information at different levels. Finally, to further exploit the contextual information from features, we use a locally enhanced feed-forward layer to replace the feed-forward network in each Transformer, which incorporates local feature representation into the global context. Powered by these designs, TCSR has the ability to capture both local and global dependencies for image HR reconstruction. Extensive experiments showed that compared with other state-of-the-art SR algorithms, our proposed method could effectively recover the details of the image, producing significant improvements in both visual effect and image quality.
   
     
 
       

Author Related Publications

  • Wael Said AbdelMageed Mohamed, "A big data approach to sentiment analysis using greedy feature selection with cat swarm optimization-based long short-term memory neural networks", Springer Nature, 2018 More
  • Wael Said AbdelMageed Mohamed, "Improving the reconstruction of dental occlusion using a reconstructed‑based identical matrix point technique", Springer Nature Switzerland AG, 2021 More
  • Wael Said AbdelMageed Mohamed, "Connection-Adjustable Network Slicing Process for Heterogeneous Service Handling in Real-Time Applications", American Scientific Publishers, 2022 More
  • Wael Said AbdelMageed Mohamed, "Space Division Multiple Access for Cellular V2X Communications", Tech Science Press, 2022 More
  • Wael Said AbdelMageed Mohamed, "A Multi-Factor Authentication-Based Framework for Identity Management in Cloud Applications", Tech Science Press, 2021 More

Department Related Publications

  • Ahmed Salah Mohamed Mostafa, "A Parallel Chemical Reaction Optimization for Multiple Choice Knapsack Problem", Springer Berlin Heidelberg, 2014 More
  • Abdul Wahid Ibrahim Mahmoud Khamis, "The design and implementation of mobile Arabic fingerspelling recognition system", International Journal of Computer Science and Network Security, 2014 More
  • Ibrahiem Mahmoud Mohamed Elhenawy, "Feature and Intensity Based Medical Image Registration Using Particle Swarm Optimization", Springer, 2017 More
  • Doaa El-Shahat Barakat Mohammed, "A Novel Whale Optimization Algorithm for Cryptanalysis in Merkle-Hellman Cryptosystem", Springer, 2018 More
  • Doaa El-Shahat Barakat Mohammed, "Integrating the whale algorithm with Tabu search for quadratic assignment problem: A new approach for locating hospital departments", Elsevier, 2018 More
Tweet