In agriculture, crop yield estimation is essential; producers, industrialists, and consumers all benefit from knowing the early yield. Manual mango counting typically involves the utilization of human labor. Experts visually examine each sample to completethe process, which is time-consuming, difficult, and lacks precision. For commercial mango production to produce high-quality fruits from the orchard to the consumer, a quick, non-destructive, and accurate variety classification is required. Because of its effectiveness in computer vision, a convolutional neural network—one of the deep learning techniques—was chosen for this investigation. For yield prediction, a total of eight popular mango cultivars were utilized. A comparison with previously trained models was used to assess the proposed model. The performance of the classifiers was evaluated using evaluation metrics such as accuracy, loss, area under the receiver operating characteristic curve score, precision, recall, F1-score, sensitivity, specificity, positive predictive value, negative predictive value, and Cohen’s Kappa performance measure. In terms of performance evaluation criteria, it was found that the proposed approach outperformed the pre-trained models. The suggested model achieved 98.85% accuracy in the test set, which had 800 images. This outcome demonstrates the tangible applicability of the proposed methodology for mango crop estimation.