Today, we are bringing you a paper recently published in Veterinary Research. This collaboration between the MycoLab at the University of Minnesota, Texas A&M, HIPRA Laboratories, and the University of Barcelona aimed at comparing scoring of cranioventral pulmonary consolidation by computer vision system and by a human evaluator.
Methods
- HIPRA Laboratories developed the Computer Vision System (CVS) to evaluate lung consolidation.
- 1,050 images of swine lungs from a Spanish slaughter house were collected.
- 1,000 images were scored by the CVS and five human evaluators looked at 200 each to evaluate the CVS.
- Inter-evaluator variability was evaluated with a set of 50 additional images.
- Three sets of 30 images ( randomly selected amongst the first 1,000) were used to evaluated intra-evaluator and intra-CVS variability. Each images was scored five times.
- The Madec and Kobisch lung lesion scoring method was used by both CVS and evaluators and the percentage of affected lung area was calculated.
Results
- 4% of lung lobes scorings were not completed by evaluators due to the image quality issues.
- The lung scorings performed by the CVS were significantly lower on average (mean difference = -0.6; 95%CI -0.76, -0.44).
- Inter-evaluator variability was moderate in this study. Highest agreement was found in the right diaphragmatic and cardiac lobes as well as the left apical lobe.
- The intra-evaluator variability was low whereas the CVS scored every image identically.
Abstract
Cranioventral pulmonary consolidation (CVPC) is a common lesion observed in the lungs of slaughtered pigs, often associated with Mycoplasma (M.) hyopneumoniae infection. There is a need to implement simple, fast, and valid CVPC scoring methods. Therefore, this study aimed to compare CVPC scores provided by a computer vision system (CVS; AI DIAGNOS) from lung images obtained at slaughter, with scores assigned by human evaluators. In addition, intra‑ and inter‑evaluator variability were assessed and compared to intra‑CVS variability. A total of 1050 dorsal view images of swine lungs were analyzed. Total lung lesion score, lesion score per lung lobe, and percentage of affected lung area were employed as outcomes for the evaluation. The CVS showed moderate accuracy (62–71%) in discriminating between non‑lesioned and lesioned lung lobes in all but the diaphragmatic lobes. A low multiclass classification accuracy at the lung lobe level (24–36%) was observed. A moderate to high inter‑evaluator variability was noticed depending on the lung lobe, as shown by the intraclass correlation coefficient (ICC: 0.29–0.6). The intra‑evaluator variability was low and similar among the different outcomes and lung lobes, although the observed ICC slightly differed among evaluators. In contrast, the CVS scoring was identical per lobe per image. The results of this study suggest that the CVS AI DIAGNOS could be used as an alternative to the manual scoring of CVPC during slaughter inspections due to its accuracy in binary classification and its perfect consistency in the scoring.