Data Leakage in Agricultural Object Detection Studies: A Critical Analysis, Recipes, and Recommendations

Data leakage is a widespread yet frequently overlooked pitfall in agricultural object detection research. It silently inflates reported performance metrics and undermines the practical reliability of machine learning (ML) models deployed in real-world farming conditions. This paper examines the characteristics of agricultural data that make these studies vulnerable to leakage. These characteristics include temporal and spatial autocorrelation, batch effects from data collection, sequential video frames, limited annotated data, image-level near-duplication, and merging datasets without checking for duplication. Drawing on best practices from ecology, geostatistics, and ML reproducibility research, we have developed a systematic, 12-criterion (C1–C12) recipe for identifying, assessing, and mitigating leakage risk. The recipe assigns each paper a score based on the applicable criteria (with a variable maximum) and maps the score to one of five bands: Strong, Minor Gaps, Material Gaps, Serious, or Critical. This process separates genuine paper-specific leakage signals from field-level reporting norms that no paper in the corpus can currently satisfy. We applied the recipe to 33 peer-reviewed journal articles published between 2023 and 2026 that used the PlantDoc dataset for plant disease detection or classification. The distribution is centered on the middle band: 7 papers are rated ”Strong,” 7 are rated ”Minor Gaps,” 15 are rated ”Material Gaps,” 4 are rated ”Serious,” and none are rated ”Critical.” The scope of augmentation is the dominant discriminator: papers that explicitly restrict augmentation to the training split average approximately 89% of their applicable maximum. Those that leave the scope ambiguous average approximately 57%, and those that apply augmentation before splitting average approximately 31%. Reported performance is inversely correlated with documentation quality at the high end. The three highest PlantDoc mAP@0.5 values among papers with C8 (”Augmentation and resampling restricted to training”) ≤ 2 in the corpus are 97.4%, 96.57%, and 84.5%. Only 5 of the 33 papers (15.2%) provide both public code and a public dataset link, and only two perform a within-dataset near-duplicate audit (with one performing a cross-dataset check).We conclude that the field is advancing faster in reported performance than in leakage safeguards. We provide practical recommendations for authors, reviewers, and dataset maintainers. Adopting a single convention — an explicit augmentation scope statement — would substantially raise the corpus mean and elevate the 4 papers rated ”Serious” to the ”Material Gaps” band.