- Research Article
- Open Access
Progression Analysis and Stage Discovery in Continuous Physiological Processes Using Image Computing
EURASIP Journal on Bioinformatics and Systems Biology volume 2010, Article number: 107036 (2010)
We propose an image computing-based method for quantitative analysis of continuous physiological processes that can be sensed by medical imaging and demonstrate its application to the analysis of morphological alterations of the bone structure, which correlate with the progression of osteoarthritis (OA). The purpose of the analysis is to quantitatively estimate OA progression in a fashion that can assist in understanding the pathophysiology of the disease. Ultimately, the texture analysis will be able to provide an alternative OA scoring method, which can potentially reflect the progression of the disease in a more direct fashion compared to the existing clinically utilized classification schemes based on radiology. This method can be useful not just for studying the nature of OA, but also for developing and testing the effect of drugs and treatments. While in this paper we demonstrate the application of the method to osteoarthritis, its generality makes it suitable for the analysis of other progressive clinical conditions that can be diagnosed and prognosed by using medical imaging.
Osteoarthritis (OA) is a highly prevalent chronic clinical condition that limits mobility and causes substantial disability in late life . It is estimated that of the population over the age of 65 have radiographic evidence of osteoarthritis , and given the increasing longevity in the industrialized world the prevalence of osteoarthritis is expected to increase further in the developed countries. While OA is one of the most prevalent diseases in the industrialized world, the physiological mechanisms of OA are poorly understood . Yet, due to the increasing prevalence of knee osteoarthritis and its consequent effects on functional limitation and general life quality at older ages, there is a growing need for scientific tools that can be reliably used to study the mechanisms of OA.
The presence and progression of osteoarthritis is usually evaluated by trained radiologists, who read knee X-rays and score them by using the standard Kellgren-Lawrence (KL) system [4, 5]. The KL classification scheme is a validated method for classifying individual joints into one of five grades, with 0 representing healthy joints, 1 representingdoubtful OA, 2 representingmild OA, 3moderate OA, and 4 being the most severe radiographic disease. This classification is based on features ofosteophytes (bony growths adjacent to the joint space),narrowing of part or all of the tibial-femoral joint space, andsclerosis of the subchondral bone, which reflect the progression of the disease. Figure 1 shows four knee X-rays of KL grades 0 (normal), 1 (doubtful), 2 (mild), and 3 (moderate).
It should be noted that while the Kellgren-Lawrence classification is the most commonly used classification scheme, there is no scientific evidence that the KL system provides an accurate direct assessment of the progression of OA [6, 7]. This downside of the KL classification scheme limits its potential as an objective tool that can be used to directly study the mechanisms and nature of OA, as well as assess the efficacy of drugs and treatments.
Another downside of the KL scoring system is that the parameters used for the classification are not discrete, and therefore different readers may have different assessments of the progression of each parameter, leading to a different conclusion regarding the presence or severity of [8, 9]. This subjectiveness makes the KL scores rough approximations of the actual progress of the disease. Although the KL scale has just five degrees of severity, even well-trained and highly experienced readers assign the same score to the same knee X-ray in just 80% of the cases [6, 10].
An alternative way to diagnose and evaluate the progression of OA is by applying quantitative image analysis methods that can measure subtle changes in the bone structure, which have been correlated with biochemical, biomechanical, and structural alterations of the articular cartilage and the subchondral bone tissues [11, 12] associated with cartilage degeneration [13, 14]. Previous studies have shown that the alteration of the bone structure can be sensed by analyzing features of the radiographic bone texture, and can be used to detect osteoarthritis by using image computing methods [10, 15–20].
While these previous studies used the radiographic bone texture for the purpose of automatic detection of osteoarthritis, the high prevalence of OA introduced an emerging need for clinical and scientific tools that can provide an objective and quantitative measurement of OA progression. These tools can be used for understanding the mechanisms of the disease and ultimately for studying the effect of drugs and treatments that may be proposed in the future and will require testing and evaluation. In this study we propose a method that applies image computing to quantitative measurement of the progression of OA and show how this method can be used to study the nature of the disease and potentially be used to better understand the mechanisms that are associated with cartilage degeneration. The analysis revealed little difference between healthy anddoubtful osteoarthritic knees (KL grade 0 and KL grade 1); a finding that could be confirmed by using images obtained with a different imaging technique, MRI. This showed that the presented technique is sensitive to changes in the disease state of an individual knee, regardless of imaging technique used to analyze the joint in question. We argue that the multipurpose nature of the method can potentially make it a useful tool for studying other progressive clinical conditions that can be diagnosed or assessed by using imaging.
The image analysis method used in this study iswndchrm [21, 22], which has been found effective for the analysis and detection of osteoarthritis using knee X-rays .Wndchrm is a multipurpose image classification method that makes use of a large set of image features, extracted from several image transforms and compound transforms. The image features used bywndchrm can be classified into high-contrast features, textures (e.g., Haralick, Tamura), statistical distribution of the pixel values (e.g., multiscale histogram, first four moments), and polynomial decomposition of the image. The image transforms that are used are Fourier transform, Chebyshev transform, Wavelet (symlet 5, level 1) transform, and edge transform. A detailed description of the image features can be found in [21, 22]. In this study the larger set of image transforms was used, as described in .
After all image features are computed, Fisher scores are computed individually for each image feature, and 93% of the features with the lowest Fisher scores are rejected. The rejection of the 93% of the features was determined experimentally, but as thoroughly discussed in [22, 23], changing this value has marginal effect on the performance. The classification can then be made by using the remaining 7% of the features with a Weighted Nearest Neighbor rule, such that the Fisher scores are used as weights. This type of nonparametric classification can be used not just for finding the class that is the closest to a given test sample, but also to measure the distances between any given test image to any training image , by using
where is the Fisher score of feature f, and , are the values of the feature f in the images and , respectively.
Similarly, the similarity between an image and a certain class can be measured using
where is the computed similarity between the imagem and the classc, N is the total number of classes, and is the shortest weighted Euclidean distance between imagem and any image in the training set that belongs in the classc, as described by (1). This assigns each feature vector of an image with a vector ofN values within the interval , representing the similarities of the image to each class. Naturally, for classification purposes the class that has the shortest weighted Euclidean distance to a given test image can be determined as the predicted class of that test image. Averaging all similarity vectors of all images of a given class provides a vector that reflects the similarity of that class to each of the other classes. Repeating it for all classes results in a similarity matrix, which represents the similarities between all pairs of classes, and can be also visualized as a phylogeny by using the Phylip package . The analysis of the similarities between classes is described in further detail in .
Once each class is assigned with a numeric value, the similarities of any given test image can be used to compute and interpolate a value for that image using
where is the interpolated value, N is the number of classes, is the similarity of imagei to the classc as determined by (2), and is the value assigned to classc. This interpolation can be used to deduce the values of test images of "in-between'' cases, which do not fit perfectly into a single class.
The classes in this study are the KL grades assigned by readers who manually assessed each of the knee X-rays. A more detailed description of the KL scoring procedure and protocols is described in [10, 25]. As described in Section 1, the KL grades are subjective and dependent on the perception of the individual reader. However, the method described here is based on the assumption that a large set of knees that were all assigned the same KL score provides a reliable representation of that OA stage. This assumption can be used to provide an automated classifier, which can automatically determine the KL grade for a given knee X-ray, and also estimate the KL grade for each test X-ray image with resolution that is higher than the individual KL grades. For instance, an X-ray of an osteoarthritic knee that is between KL grade 2 and KL grade 3 can be assigned with the score of 2.6, rather than classified into KL grade 2 or KL grade 3.
Data for the experiment were X-ray images taken from the BLSA project  that were used in  and MRI images taken from the Osteoarthritis Initiative . The KL grades of each of the BLSA X-rays were determined by two highly experienced readers, such that a third reader adjudicated knees with discordant grades. Previous studies showed that the classification method used here could automatically classify between KL grade 3 and healthy knees with accuracy of 91%. with sensitivity and specificity of 95% and 86.5%, respectively. KL elements from the Osteoarthritis Initiative (OAI) knees were scored by different readers in each of the four independent centers that participate in the project, and the individual assessments of the KL elements were compiled into a final KL score of the knee.
The images were preprocessed such that the center of the joint is detected using the method described in , and then the area around the joint center was separated from the image. Figures 2 and 3 show sample X-ray and MRI images used for this study, respectively.
In the experiment with the X-ray images, 30 images of each class were used for training and nine images per class for testing. The experiment was repeated 100 times such that in each run different images were allocated randomly for training and testing. The predicted values of the test images were scored as described in Section 2, and the averaged scores for the different KL grades are shown in Figure 4.
As the figure shows, the mean predicted KL grades ofhealthy anddoubtful OA joints are nearly identical, showing that there are no substantial differences in the bone texture between healthy joints anddoubtful OA joints, while significant differences are clearly noticeable for the image analysis of the bone textures ofdoubtful andmild OA andmild andmoderate OA. It should be noted that KL grade 1 is determined by the presence of mild osteophytes which are not present in the joint itself and are often difficult to detect using X-ray. KL grade 1 patients usually do not suffer from pain symptoms, but more importantly, are not in the phase of fast loss of cartilage that starts around KL grade 2 and estimated in 0.25 mm per year . While the correlation between the radiographic bone texture and cartilage loss has been identified as described in Section 1, this finding shows that rapid loss of cartilage is accompanied by a faster alteration of the radiographic bone texture, suggesting that both can be related to the same mechanism and become more dominant when OA progresses beyond KL grade 1.
Another analysis of the predicted KL grades can be done by using the amount of samples that were predicted to have a KL grade lower than a certain value. Figure 5 shows, for each of the actual KL grades, the amount of sample images (-axis) that their predicted KL grade score is lower than a certain value (-axis). For instance, looking at the curve of the X-rays scored by radiologists as KL grade 3, 0.2 of these X-rays were assigned by the computer analysis with a predicted KL grade lower than 1.9.
As the figure shows, the curves for actual KL grade 0 and actual KL grade 1 are very similar to each other, showing that the distribution of the predicted KL grade values for these two classes is relatively close, while the predicted KL grade distribution is noticeably different for X-rays that were diagnosed by radiologists as KL grades 2 and 3.
Different visualization of the analysis can be provided by using an evolutionary tree that reflects the similarities between the KL grade classes, as described in Section 2. Figure 6 shows the phylogeny produced by using the Phylip package to visualize the similarity matrix of the KL grade classes.
The phylogeny shows that the system was able to deduce that the KL grade classes represent a continuous process and was also able to find the correct order of the classes. However, it also shows that the X-rays scored as KL grade 0 (healthy joints) are very similar to the X-ray images scored as KL grade 1 (doubtful OA), showing that the radiographic texture does not significantly change between these two grades of the disease. This type of visualization can also be useful for more complex experiments, with a larger number of stages and possible paths of progression.
A similar experiment with MRI knee images, which features different frequency and different resolution compared to the X-ray images, leads to a similar conclusion. The experiment used 100 knee images (similar to the sample Figure 3) for each KL grade for training and 50 images per class for testing. The experiment was repeated 20 times such that in each run different images were used for training and testing. The averaged predicted values of the test images are shown in Figure 7.
As before, this shows that the progression of the alteration of the bone texture shows no significant difference between healthy (KL grade 0) anddoubtful OA (KL grade 1) joints, and a faster process of bone alteration starts when the disease reaches KL grade 2. The fact that the image analysis method does not detect the differences between healthy and doubtful OA joints shows that the bone texture does not change significantly between these two grades, compared to the alteration of the bone texture associated with further stages of the disease.
While osteoarthritis is one of the most prevalent diseases in the industrialized world, and one of the most significant economical burdens on the healthcare system, little has yet been discovered about its physiological mechanisms, which their understanding is acute for the purpose of developing drugs or preventive treatments. For instance, it is yet unknown whether OA is a systemic clinical condition driven by controlled physiological pathways or simply stochastic "wear and tear'' that leads to cartilage degeneration .
In this study we used a quantitative comparison of X-ray images to analyze the progression of OA and the differences between the KL grades. The images are compared by using a large set of image content descriptors computed from each X-ray, and the weighted Euclidean distance reflects the similarity between each two X-rays in the dataset. This quantitative assessment of the differences between images is used to quantify the differences between the KL grades, which are determined quantitatively by averaging a large number of images from each grade and providing anaverage distance between each two neighboring grades based on the physiology of the knee.
The quantitative analysis of the differences between the different KL grades shows that the alteration of the bone texture, which has been associated with OA as discussed in Section 1, does not progress at a constant rate, which can be interpreted as an indication that OA is not driven solely by stochastic accumulation of damage and that controlled physiological processes can be involved in the processes of cartilage degeneration. It also shows that the progression of the biochemical and biomechanical processes associated with the disease progress faster between KL grade 1 and KL grade 2. These findings suggest that the KL scale might not be linear to the actual progression of OA and also indicate that understanding the mechanisms of the disease might require the analysis of changes between KL grade 1 to KL grade 2.
While it can be assumed that the image analysis method might not be sensitive enough for detecting the differences between healthy and doubtful OA joints, it should be noted that the same method is sensitive enough to detect osteoarthritis by examining the radiographic texture of the bone years before the disease becomes symptomatic , and when the joints are still scored by radiologists as healthy.
The "gold standard'' for this study is the KL grades, which can be subjective, and not necessarily an accurate quantitative reflection of the actual progression of OA. Since the classification betweendoubtful OA (KL grade 1) and nonosteoarthritic joints can be difficult, the findings presented in Section 3 might be due to the difficulties of the readers to distinguish between these two grades.
The KL classification scheme is exposed to the subjectiveness of the reader, and it can not be considered as an accurate quantitative method for OA assessment. Since computer analysis of the texture can provide an objective measurement, it can ultimately provide a systematic and accurate scoring system for the evaluation of the presence or progression of OA. This scoring technique can better reflect the actual progress of the disease, and its quantitative fashion can potentially make it suitable for studying the disease mechanisms or the effect of drugs and treatments on the clinical condition. While indicators such as the KL grade or pain symptoms are merely rough measurements of the disease progression, the computer analysis of the bone texture can provide a higher resolution estimation of the state of the disease, and its response to drugs can be more sensitive and objective than other OA indicators.
In this study we apply thewndchrm tool to the assessment of the progression of OA, but it should be noted thatwndchrm is a general-purpose biomedical image analysis tool and can provide an informative analysis for a variety of subjects, magnifications, and types of imaging . Therefore, we argue that it is possible that the image analysis method described in this paper can be used for studying and evaluating the progression of other continuous clinical conditions that can be sensed and diagnosed by using microscopy or other forms of biomedical imaging. Medical imaging is used for diagnosis and prognosis of very many diseases and clinical conditions, and the link between image morphology and disease progression has been well established. In this paper we propose to use image analysis to detect stages of rapid progression of the disease, which can be used to better study the disease mechanisms.
Finally, while this study focused on an unbiased analysis of the image content, a more comprehensive study can also include clinical data such as pain, history of injuries, relevant medication information, age, weight, BMI, and other clinical indicators that can be relevant to OA. These data can provide a more detailed analysis that can detect stages in the progression of the disease that correspond to other clinical data that affect osteoarthritis.
The software used for the experiments described in this paper, including the source code, is available for free download at http://www.phy.mtu.edu/~lshamir/downloads/ImageClassifier, and users are encouraged to download and use it for the analysis of continuous image data of disease progression. Technical description of the code is available at .
Stockwell RA: Cartilage failure in osteoarthritis: relevance of normal structure and function. A review. Clinical Anatomy 1991, 4(3):161-191. 10.1002/ca.980040303
St. Clair SF, Higuera C, Krebs V, Tadross NA, Dumpe J, Barsoum WK: Hip and knee arthroplasty in the geriatric population. Clinics in Geriatric Medicine 2006, 22(3):515-533. 10.1016/j.cger.2006.04.004
Loeser RF: Molecular mechanisms of cartilage destruction in osteoarthritis. Journal of Musculoskeletal Neuronal Interactions 2008, 8(4):303-306.
Kellgren JH, Lawrence JS: Radiologic assessment of osteoarthritis. Annals of the Rheumatic Diseases 1957, 16: 494-501. 10.1136/ard.16.4.494
Kellgren JH, Jeffrey M, Ball J: Atlas of Standard Radiographs. Blackwell Scientific, Oxford, UK; 1963.
Spector TD, Cooper C: Radiographic assessment of osteoarthritis in population studies: whither kellgren and lawrence? Osteoarthritis and Cartilage 1993, 1(4):203-206. 10.1016/S1063-4584(05)80325-5
Emrani PS, Katz JN, Kessler CL, Reichmann WM, Wright EA, McAlindon TE, Losina E: Joint space narrowing and Kellgren-Lawrence progression in knee osteoarthritis: an analytic literature synthesis. Osteoarthritis and Cartilage 2008, 16(8):873-882. 10.1016/j.joca.2007.12.004
Sun Y, Günther KP, Brenner H: Reliability of radiographic grading of osteoarthritis of the hip and knee. Scandinavian Journal of Rheumatology 1997, 26(3):155-165. 10.3109/03009749709065675
Croft P: An introduction to the atlas of standard radiographs of arthritis. Rheumatology 2005, 4, supplement: 42.
Shamir L, Ling SM, Scott WW Jr., Bos A, Orlov N, MacUra TJ, Eckley DM, Ferrucci L, Goldberg IG: Knee X-ray image analysis method for automated detection of osteoarthritis. IEEE Transactions on Biomedical Engineering 2009, 56(2):407-415.
Aigner T, McKenna L: Molecular pathology and pathobiology of osteoarthritic cartilage. Cellular and Molecular Life Sciences 2002, 59(1):5-18. 10.1007/s00018-002-8400-3
Martel-Pelletier J, Pelletier J-P: Osteoarthritis: recent developments. Current Opinion in Rheumatology 2003, 15(5):613-615. 10.1097/00002281-200309000-00015
Buckwalter A, Mankin HJ: Instructional course lectures, the American Academy of Orthopaedic Surgeons Articular Cartilage—part II: degeneration and osteoarthrosis, repair, regeneration, and transplantation. Journal of Bone and Joint Surgery 1997, 79: 612-632.
Radin EL, Rose RM: Role of subchondral bone in the initiation and progression of cartilage damage. Clinical Orthopaedics and Related Research 1986, 213: 34-40.
Boniatis I, Costaridou L, Cavouras D, Kalatzis I, Panagiotopoulos E, Panayiotakis G: Osteoarthritis severity of the hip by computer-aided grading of radiographic images. Medical and Biological Engineering and Computing 2006, 44(9):793-803. 10.1007/s11517-006-0096-3
Boniatis I, Cavouras D, Costaridou L, Kalatzis I, Panagiotopoulos E, Panayiotakis G: Computer-aided grading and quantification of hip osteoarthritis severity employing shape descriptors of radiographic hip joint space. Computers in Biology and Medicine 2007, 37(12):1786-1795. 10.1016/j.compbiomed.2007.05.005
Messent EA, Ward RJ, Tonkin CJ, Buckland-Wright C: Cancellous bone differences between knees with early, definite and advanced joint space loss; a comparative quantitative macroradiographic study. Osteoarthritis and Cartilage 2005, 13(1):39-47. 10.1016/j.joca.2004.10.009
Podsiadlo P, Wolski M, Stachowiak GW: Automated selection of trabecular bone regions in knee radiographs. Medical Physics 2008, 35(5):1870-1883. 10.1118/1.2905025
Podsiadlo P, Dahl L, Englund M, Lohmander LS, Stachowiak GW: Differences in trabecular bone texture between knees with and without radiographic osteoarthritis detected by fractal methods. Osteoarthritis and Cartilage 2008, 16(3):323-329. 10.1016/j.joca.2007.07.010
Kamibayashi L, Wyss UF, Cooke DV, Zee B: Trabecular microstructnre in the medial condyle of the proximal tibia of patients with knee osteoarthritis. Bone 1995, 17(1):27-35. 10.1016/8756-3282(95)00137-3
Shamir L, Orlov N, Eckley DM, Macura T, Johnston J, Goldberg IG: Wndchrm—an open source utility for biological image analysis. Source Code for Biology and Medicine 2008., 3:
Orlov N, Shamir L, Macura T, Johnston J, Eckley DM, Goldberg IG: WND-CHARM: multi-purpose image classification using compound image transforms. Pattern Recognition Letters 2008, 29(11):1684-1693. 10.1016/j.patrec.2008.04.013
Shamir L, Orlov N, Goldberg IG: Evaluation of the efficacy of image features extracted from multi-order image transforms. Proceedings of the International Conference on Image Processing Computer Vision and Pattern Recognition, 2009 37-42.
Felsenstein M: PHYLIP phylogeny inference package. 2004.
Shamir L, Ling SM, Scott W, Hochberg M, Ferrucci L, Goldberg IG: Early detection of radiographic knee osteoarthritis using computer-aided analysis. Osteoarthritis and Cartilage 2009, 17(10):1307-1312. 10.1016/j.joca.2009.04.010
Shock NW, Greulich RC, Andres R, et al.: Normal human aging: the Baltimore Longitudinal Study of Aging. Government Printing Office, Washington, DC, USA; 1984.
Schneider E, NessAiver M, White D, Purdy D, Martin L, Fanella L, Davis D, Vignone M, Wu G, Gullapalli R: The osteoarthritis initiative (OAI) magnetic resonance imaging quality assurance methods and results. Osteoarthritis and Cartilage 2008, 16(9):994-1004. 10.1016/j.joca.2008.02.010
Lequesne M, Brandt K, Bellamy N, Moskowitz R, Menkes CJ, Pelletier JP, Altman RD: Guidelines for testing slow acting drugs in osteoarthritis. Journal of Rheumatology 1994, 41(12, supplement):65-71.
Shamir L, Orlov N, Mark Eckley D, Macura TJ, Goldberg IG: IICBU 2008: a proposed benchmark suite for biological image analysis. Medical and Biological Engineering and Computing 2008, 46(9):943-947. 10.1007/s11517-008-0380-5
This paper was supported entirely by the Intramural Research Program of the NIH, National Institute on Aging. The OAI is a public-private partnership comprised of five contracts (N01-AR-2-2258; N01-AR-2-2259; N01-AR-2-2260; N01-AR-2-2261; N01-AR-2-2262) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by the OAI Study Investigators. Private funding partners include Merck Research Laboratories; Novartis Pharmaceuticals Corporation, GlaxoSmithKline; Pfizer, Inc. Private sector funding for the OAI is managed by the Foundation for the National Institutes of Health. This paper was prepared using an OAI public use data set and does not necessarily reflect the opinions or views of the OAI investigators, the NIH, or the private funding partners.
About this article
Cite this article
Shamir, L., Rahimi, S., Orlov, N. et al. Progression Analysis and Stage Discovery in Continuous Physiological Processes Using Image Computing. J Bioinform Sys Biology 2010, 107036 (2010). https://doi.org/10.1155/2010/107036
- Cartilage Degeneration
- Image Analysis Method
- Fisher Score
- Healthy Joint