Olive productivity should be improved through stimulating nutrition, particularly under poor fertility soils. Consequently, the objective of this study was to assess the efficacy of applying organic and bio-fertilizers on the physiological growth, yield and fruit quality of olive trees under newly reclaimed poor-fertility sandy soil in an arid environment. During a field experiment carried out at El-Qantara, North Sinai, Egypt over two consecutive seasons (2019–2020 and 2020–2021), olive Kalamata trees were evaluated under three organic fertilizer treatments alone or in combination with three bio-fertilizers treatments. Organic fertilizer was applied as goat manure (16.8 kg/tree/year), or olive pomace (8.5 kg/tree/year) in mid-December of each season vs. untreated trees. The bio-fertilizers were applied as N-fixing bacteria (150 g/tree) was inculated in early March of each season, or amino acid mixture (1.5%) was applied three times, at 70% of full bloom, 21 days after full bloom, and a month later in comparison to a non-fertilized tree (control). The cultivar used was Kalamata, a dual-purpose cultivar for oil and table olives whose value increases when processed as table olives. The results indicated that the goat manure followed by olive pomace significantly enhanced photosynthetic pigments (chlorophyll a, b, and carotenoids), leaf mineral contents (N, P, K, Ca, Mg and Fe), tree canopy volume, number of flowers per inflorescence, number of inflorescences per shoot, initial fruit set, fruit retention. For fruit quality, fruit length and width, fruit weight, and total fruit yield was increased compared to the non-fertilized control. Likewise, the bio-fertilizer N-fixing bacteria followed by the amino acid mixture significantly improved all of the aforementioned parameters. Accordingly, it is recommended, both environmentally and economically to utilize organic and bio-fertizers, particularly goat manure combined with N-fixing bacteria, in low-fertility soil to sustain olive production as well as reducing mineral fertilization.