RiceMultiModalNet: a proof-of-concept biologically constrained multi-modal deep learning for rice disease detection integrating phenological stage and environmental risk

Rice disease detection systems based on visual deep learning achieve high classification accuracy but fail to enforce biological plausibility, provide pre symptom risk forecasting or deliver agronomically interpretable explanations. This paper presents RiceMultiModalNet, a proof-of-concept three-branch late-fusion framework. Evaluated on a computationally synthesised from independent public repository multimodal dataset in which leaf images, environmental variables, and phenological stage labels were not co-collected from the same field locations or time points. Results demonstrate the feasibility of three operational capabilities under controlled conditions and should not be interpreted as validated field-deployment or real-world early-warning capability. It integrates a ResNet-50 visual classifier, a phenological growth stage encoder, and an eight variable meteorological risk (multilayer perceptron) MLP in a single pipeline, evaluated on a computationally synthesised dataset of 4804 rice leaf images from publicly available repositories (Mendeley, Kaggle) together with NASAPOWER reanalysis environmental data assigned computationally. Three operational contributions are reported as (1) an 8 × 3 literature derived phenological susceptibility constraint module suppressing biologically implausible prediction at inference time, (2) an image free environmental risk sub model enabling pre symptom disease advisory from meteorological inputs alone, and (3) SHAP-based agronomic explainability linking predictions to phenological and environmental drivers, spatially validated by Grad-CAM (mean lesion energy concentration 80.2%, mean IoU 46.6, pointing game accuracy 60–65%, full per disease breakdown in Table 11). The framework’s primary contributions are three operational capabilities (1) an 8 × 3 phenological susceptibility constraint module suppressing biologically implausible predictions, (2) an image-free environmental risk sub-model (AUC = 0.99%, ECE = 0.0281) enabling pre-sympton disease advisory from meterological inputs alone, and (3) SHAP based agronomic explainability validated spatially by Grad-CAM (mean lesion energy concentration 80.2%, mean IoU 46.6%, pointing game accuracy (60–65%). The visual branch achieved 99.58% classification accuracy on the benchmark, confirming correct visual branch functionality, however all ablation configurations including the single-model visual baseline achieved identical accuracy, confirming that multi-modal value resides entirely in the three operational capabilities described above. All three modalities were computationally synthesised rather than co-collected in the field. The AUC and SHAP rankings may reflect algorithmic correlations introduced during synthesis and should be treated as proof-of- concept indicators only. Longitudinal field validation with co-collected multimodal data remains an essential next step.