Objective To evaluate diagnostic performance and inter-reviewer agreement (IRA) of the Gynecologic Imaging Reporting and Data System (GI-RADS) for diagnosis of adnexal masses (AMs) by pelvic ultrasound (US). Patients and methods A prospective multicenter study included 308 women (mean age, 41 ± 12.5 years; range, 15–73 years) with 325 AMs detected by US. All US examinations were analyzed, and AMs were categorized into five categories according to the GI-RADS classification. We used histopathology and US follow-up as the reference standards for calculating diagnostic performance of GI-RADS for detecting malignant AMs. The Fleiss kappa (κ) tests were applied to evaluate the IRA of GI-RADS scoring results for predicting malignant AMs. Results A total of 325 AMs were evaluated: 127 (39.1%) were malignant and 198 (60.9%) were benign. Of 95 AMs categorized as GI-RADS 2 (GR2), none was malignant; of 94 AMs categorized as GR3, three were malignant; of 13 AMs categorized as GR4, six were malignant; and of 123 AMs categorized as GR5, 118 were malignant. On a lesion-based analysis, the GI-RADS had a sensitivity, a specificity, and an accuracy of 92.9%, 97.5%, and 95.7%, respectively, when regarding only those AMs classified as GR5 for predicting malignancy. Considering combined GR4 and GR5 as a predictor for malignancy, the sensitivity, specificity, and accuracy of GI-RADS were 97.6%, 93.9%, and 95.4%, respectively. The IRA of the GI-RADS category was very good (κ = 0.896). The best cutoff value for predicting malignant AMs was >GR3. Conclusions The GI-RADS is very valuable for improving US structural reports.