Recent advances in natural language processing have seen remarkable improvements in text classification tasks, largely driven by transformer-based architecture such as BERT. However, deploying these models in real-world applications demands a focus on responsible artificial intelligence (AI) for emphasizing transparency, uncertainty quantification, and interpretability to foster trustworthiness. This study proposes an approach combining classical ensemble classifiers (Logistic Regression, Random Forest, and Support Vector Classification) calibrated for probabilistic outputs with BERT fine-tuning on the DBpedia 14-class dataset. Furthermore, we incorporate neutrosophic logic to quantify the uncertainty in predictions by calculating Truth (T), Indeterminacy (I), and Falsity (F) measures from model output probabilities. Our classical ensemble achieves an average accuracy of approximately 97.3%, while BERT fine-tuning attains near-perfect accuracy (~99.8%) across the 14 balanced classes. The neutrosophic uncertainty analysis reveals high confidence (high T, low I and F), with indeterminacy effectively identifying ambiguous samples, highlighting areas for human review. These results underscore the utility of combining classical and deep learning methods within a responsible AI framework, providing both state-of-the-art performance and interpretable uncertainty quantification, crucial for trustworthy deployment in sensitive applications.