Long-term cropping systems require balanced phosphorus (P) management for better yield and environmental sustainability. However, the soil P transformations under fallow rotations with and without long-term nitrogen (N) and P fertilization largely remained unknown. This study evaluated the status of P forms in loess soils in response to varied combined rates of N and P fertilizers, tillage management practices, fallow land systems (natural fallow (NF), and bare fallow (BF)). Four NP treatments (N0P0, control; N0P100, 100 kg P ha−1; N160P0, 160 kg N ha−1; and N160P100), and two treatments with no fertilizer application and crops (NF and BF) were conducted. The treatments N0P100 and N160P100 significantly increased soil total P, inorganic P (Pi), organic P (Po), and Olsen P concentrations compared to the control, NF, and BF treatments. Labile P fractions (NaHCO3-Po and NaHCO3-Pi) were 7.30% and 11.8–12.4% higher in fertilized treatments than in control, NF, and BF treatments. The moderately labile NaOH-Pi was stable in all treatments, but NaOH-Po significantly decreased in the NF (2.60%) and BF (1.40%) treatments compared to the control and fertilized treatments; however, HClD-Pi was 59.1–66.0% higher in NF and BF compared to the control and fertilized treatments. Non-labile P (HClC-Pi and HClc-Po) fractions showed no significant difference between the fertilized and unfertilized treatments. Residual P levels were substantially greater in the P fertilized (N0P100) treatment than in the fallow treatments. The conceptual framework and redundancy (RDA) analysis revealed that the labile (NaHCO3-Pi and NaHCO3-Po) and moderately labile P fractions (NaOH-Po, NaOH-Pi, and HClD-Pi) were substantially associated with Olsen P contents, grain yield, and P uptake. Higher moderate fraction concentrations in fallows and their positive correlation with yield, P uptake, and Olsen P predict the importance of reserved P in these soils upon long-term fertilization, suggesting the utilization of P legacy and optimizing fertilizer applications.