(522c) Investigating Performance Metrics and Machine Learning Models Towards Automating the Diagnosis of Membranous Nepthropathy

Membranous nephropathy (MN) is a glomerular disease that causes nephrotic syndrome among adult patients. Records indicate that approximately one third of the patients that possess MN further progress into end-stage renal diseases in 5 to 15 years; among patients that are above 60 years old, 20-30% of the cases are found to be associated with malignancy such as lung cancer and prostate cancer.[2][3]

Diagnosis of MN is exclusively achieved by renal biopsy with a focus on the glomerular basement membrane area, which involves examination to the images of histochemically stained tissue samples captured by either light microscopy (LM) or electron microscopy (EM) [1]. For precise identification of glomerular features, the pathoanatomical examination often involves using whole-slide images (WSIs) of tissues stained in multiple histochemical approaches, which is a labor-intensive process.

Motivated by this, we aim to develop an automated examination method based on artificial intelligence and investigate its efficacy towards reducing the workload of clinicians and improving the consistency of diagnostic results. The basis for this effort is artificial intelligence algorithms to design an efficacious data-driven classification model.

In this work, a Convolutional Neural Network (CNN)[4] based model was structured and trained to determine the existence of MN within the images of glomeruli. To further address the issue of model accuracy not satisfying the requirement of the application, a method called high confidence acceptance (HCA) was designed to boost the precision of the model to an application specified target of 95\%. With HCA, we are aiming to use only strongly confident predictions as automated diagnostic results, leaving the relatively lower confidence instances to manual inspections performed by renal pathologists. In such scenario, a partial replacement of the manual examination by renal pathologists at the demanded accuracy can be accomplished. A metric called Acceptance Rate (AR) was accordingly defined and used in our scheme to measure HCA performance; AR represents the percentage of workload being automated and replaced by the CNN model.

As a result, the model trained in this work which initially had an overall accuracy of approximately 86% demonstrated an AR of approximately 34%, with a precision that met target accuracy as high as 95%. This result implied that one third of the workload will be fully automated at target accuracy, and the rest of the less confident instances (declared inconclusive) will be sent to renal pathologists for manual inspection. Furthermore, the model developed in this work also presented an acceptance rate of 23% when applied to a dataset solely comprised of HE (Hematoxylin-Eosin) stained biopsy images, which indicated a potential reduction to the staining effort which conventionally requires up to 4 different staining methods and usage of excessive specialist labor. A further test on images of larger field of view was also carried out; a performance consistent with tests on glomeruli-level images was obtained and the practical value of the model and HCA method is thus illustrated.

The result illustrated the feasibility of automatically diagnosing membranous nephropathy with CNN, which has considerable value to the relevant medical profession. Meanwhile, the HCA method is an universal technique that can be applied to improve and measure the applicability of any classification model that is not satisfying the required accuracy of the target application, and may thus further broaden the horizon of artificial intelligence in practical implementation.

References:

[1] Agarwal, S. K., Sethi, S., & Dinda, A. K. (2013). Basics of kidney biopsy: A nephrologist’sperspective.Indian Journal of Nephrology,23(4), 243–252. doi: 10.4103/0971-4065.114462

[2] Couser, W. G. (2017). Primary membranous nephropathy.Clinical Journal of the AmericanSociety of Nephrology,12(6), 983–997. doi: 10.2215/CJN.11761116

[3] Lai, W. L., Yeh, T. H., Chen, P. M., Chan, C. K., Chiang, W. C., Chen, Y. M., . . . Tsai,T. J. (2015, 2).Membranous nephropathy: A review on the pathogenesis, diagnosis,and treatment(Vol. 114) (No. 2). Elsevier. doi: 10.1016/j.jfma.2014.11.002

[4] LeCun, Y., Haffner, P., Bottou, L., & Bengio, Y. (1999). Object Recognition with Gradient-Based Learning. In (pp. 319–345). Retrieved fromhttp://link.springer.com/10.1007/3-540-46805-619doi: 10.1007/3-540-46805-6{\}19