Over the past few years, billions of unsecured Internet of Things (IoT) devices have been produced and released, and that number will only grow as wireless technology advances. As a result of their susceptibility to malware, effective methods have become necessary for identifying IoT malware. However, the low generalizability and the nonindependently and identically distributed data (non-IID) still pose a major challenge to achieving this goal. In this work, a new federated malware detection paradigm, termed FED-MAL, is introduced to collaboratively train multiple distributed edge devices to detect malware. In FED-MAL, the malware binaries are transformed into an image format to lessen the impact on non-IID, and then a compact convolutional model, named AM-NET, is proposed to learn the malware patterns as an image recognition task. The compact nature of AM-NET makes it an appropriate choice for deployment on resource-constrained IoT devices. Following, a refined edge-based adversarial training is given in FED-MAL to empower generalizability and resistibility by generating adversarial samples from various participating clients. Experimental evaluation on publicly available malware data sets shows that the FED-MAL is efficacious, reliable, expandable, generalizable, and communication efficient.