This paper proposes a new method for encryption of RGB color images by combining two encryption approaches: the spatial approach and the transformation approach. The proposed method uses the 3D fractional modified Henon map (3D FrMHM) and the discrete fractional Krawtchouk moments (FrDKM). We have also proposed a new hybrid optimization algorithm (H-SSAOA) to optimize the parameters of the proposed Henon map and the parameters of the Krawtchouk fractional moments. This algorithm is based on the hybridization of two metaheuristic algorithms: the "Salp Swarm Algorithm" (SSA) and the "Arithmetic Optimization Algorithm" (AOA). The simulation results reveal the optimization efficiency of the proposed hybrid algorithm H-SSAOA compared to other meta-heuristic algorithms and the efficiency of the suggested encryption method for encrypting RGB color images in terms of sensitivity to the security key and resistance to different attacks.