2.1 Gaps in Literature

While many studies have demonstrated the effectiveness of deep learning techniques for knee osteoarthritis (OA) classification, several challenges persist in the field that limit the applicability and reliability of these models. These gaps include:

2.2 Proposed Solutions

In this study, we address these challenges by implementing the following strategies:

By addressing these limitations, our approach aims to develop a more robust and reliable model for knee OA classification, enabling effective deployment in clinical settings.

Figure 1 Illustration of knee X-ray grading: Grade 0 shows a normal knee, Grade 1 indicates early osteoarthritis, Grade 2 represents mid-level, Grade 3 is moderate, and Grade 4 signifies severe osteoarthritis.

3.1 Dataset

As shown in Figure 1, the data set of knee X-ray images labeled in accordance with the Kellgren Lawrence (K.L.) scale was used in this study. The dataset was formed by using the data from the Osteoarthritis Initiative (OAI) and also mentioned by Abedin et al. [3]. Data was used to train and validate the suggested model. There are 5947 people in this dataset, with ages ranging from 45 to 79. There are around 12,000 photos in the dataset. Following the loading of the images, the data was divided into 70% and 30% for the model's training and testing, respectively. Grade 0 illustrates the normal knee x-ray, whereas Grade 1 displays a knee that may have osteoarthritis. Knee osteoarthritis is classified as grade 2 for mid-level and grade 3 for moderate. The ultimate grade of 4, or severe osteoarthritis, is used.

3.2 Data preparation

Because the number of images varies with the K.L. scale, the dataset has a problem with class imbalance. The effect of class imbalance on categorization has a detrimental impact on the model's performance [29]. To create a balanced dataset and improve deep learning model performance for small datasets, we use data augmentation. New training data were made from old data by using data augmentation methods. Picture data augmentation is the process of converting identically categorized copies of the source picture. Three image augmentation techniques, rotation, distortion, and flipping, are used to produce the visuals. During the rotation process for picture enhancement, the images are rotated both clockwise and counterclockwise by a maximum of 10 degrees. After that, we randomly deform the image. Finally, we flip the photos vertically and horizontally with a probability of 0.5. We transform them to an array dataset and resize them to 224 × 224. Finally, we encode every label once. Due to class imbalance in the dataset, where certain categories had fewer images, data augmentation techniques were applied to balance the dataset and improve model performance. The data augmentation techniques used were rotation, flipping, and distortion. After augmentation, the total number of images increased from 12,000 to 36,000, with each image being modified to generate new training samples. This increased dataset size helped the model better generalize and reduced the risk of overfitting.

3.3 Model Selection

Convolutional Neural Networks (CNNs): CNNs are the cornerstone of modern image classification tasks due to their ability to automatically learn relevant features from raw image data without the need for manual feature extraction [30, 31, 32, 33]. For knee osteoarthritis (OA) classification, CNNs are particularly well-suited because they can effectively capture spatial hierarchies in X-ray images, enabling the learning of discriminative features at multiple scales, such as bone texture and joint spaces. Prior studies, including those by Jena et al. [23] and Rehman et al. [28], have demonstrated the effectiveness of CNNs in diagnosing knee OA from radiographs, making CNNs a natural choice for this task.

VGG16: VGG16 is a widely used deep learning architecture known for its simplicity and depth. It consists of 16 layers, including 13 convolutional layers, and has achieved impressive results in various image classification tasks. VGG16 has been successfully applied in several medical imaging domains, such as brain tumor detection and breast cancer classification, demonstrating its ability to generalize well to medical images. The architecture of VGG16, characterized by small 3×3 filters and deep convolutional layers, enables it to learn fine-grained patterns in complex images, such as knee X-rays, which are crucial for accurate OA severity classification. Furthermore, transfer learning with VGG16 allows the utilization of pre-trained weights from large datasets like ImageNet, enhancing model performance even with relatively smaller datasets of knee X-rays.

Figure 2 Layer architecture of the VGG16 model.

3.4 VGG16 Architecture

The Visual Geometry Group (VGG) at Oxford University created the deep convolutional neural network architecture known as VGG16. The architecture of VGG16 is illustrated in Figure 2. Its sixteen-weight layers contain three fully connected layers and thirteen convolutional layers. The convolutional layers enable the network to effectively learn hierarchical feature representations from pictures by using max pooling and tiny 3x3 filters for downsampling, as shown in 2. For knee osteoarthritis detection, VGG16 leverages transfer learning. The model is first pre-trained on a large dataset like ImageNet to learn general image features. We then fine-tune this pre-trained model by retraining the top classification layers on our dataset of knee X-ray images labeled with osteoarthritis severity grades. Fine-tuning allows VGG16 to adapt to recognizing features specific to knee OA from the X-rays. In our experiments, we trained two versions of VGG16: one for ten epochs and another for 20 epochs. In order for the model to be able to generate correct predictions on the training data, the model's parameters must be adjusted during the training phase, which is a crucial stage in the machine learning or deep learning process. In our instance, we use X-ray pictures of knee OA to train the suggested model. Once trained, the model may be used to predict the presence or absence of knee OA, if present, the disease's stage.

The additional epochs allow VGG16 to learn better representations for identifying different grades of knee osteoarthritis. The hierarchical feature learning capabilities and transfer learning approach enable VGG16 to classify knee osteoarthritis severity from X-ray images effectively. The model utilizes multiple modules, which consist of parallel convolutional layers of varying kernel sizes, to capture multi-scale features from the input images. The extracted features are then aggregated and fed into fully connected layers for classification. The activation function is used for the output layer. We utilized the proposed model weights to train it on the input data because it is a pre-trained model.

Backpropagation will only update the thick layer, not any other layers. Our data's mean and variance might not match the pre-trained weights, therefore we didn't freeze the batch normalization layers to improve performance. Consequently, we applied the auto-tuning approach to these layers. The suggested model uses the Stochastic Gradient Descent (SGD) as its optimization function. SGD was chosen because it performs better for multiclass classification issues. The performance of the model is assessed using the loss function Categorical Cross Entropy. Through this methodology, the Inception v4 model achieves high accuracy and robustness in detecting and classifying Knee OA, contributing to improved diagnosis and treatment of this debilitating condition.

The VGG16 model uses a series of convolutional layers to extract features from the input images. The output of each convolutional layer is passed through a rectified linear unit (ReLU) activation function, which is defined as:

The convolutional layers are followed by max-pooling layers, which perform downsampling of the feature maps. The max-pooling operation is defined as:

where $R_{i,j}$ is the pooling region centered at $(i,j)$, and $x_{m,n}^k$ and $y_{i,j}^k$ are the input and output feature maps, respectively. The VGG16 model also includes fully connected layers, which combine the features extracted by the convolutional layers to produce the final classification output. The output of the fully connected layers is passed through a softmax function to obtain the class probabilities:

where $z_j$ is the input to the softmax function for class $j$, and $K$ is the total number of classes.

4.1 Performance of Proposed CNNs

The proposed CNNs model achieved strong performance, attaining 99% training accuracy and 80% testing accuracy after 50 epochs of training as shown in Figures 3 and 4. The high training accuracy indicates that the CNNs model was able to effectively learn the features and patterns in X-ray images associated with different grades of knee osteoarthritis severity. The gap between training and testing performance implies some overfitting, but the competitive 80% testing accuracy is promising for real-world applications. As a tailored architecture designed specifically for this task, the CNNs leverages domain-specific features crafted through an optimized combination of convolutional, pooling, and fully connected layers. The use of dropout regularization and data augmentation also helped improve generalization and testing performance. The proposed CNNs demonstrates that specially engineered deep networks can deliver excellent results for knee OA classification from X-rays [34].

Figure 3 CNNs accuracy and loss curve.

Figure 4 CNNs confusion matrix.

The confusion matrix demonstrates that the proposed CNNs model performs very well in correctly classifying the different grades of knee osteoarthritis severity. The strong diagonal from the top left to the bottom right shows that the majority of samples are classified correctly, with 116, 109, 119, and 115 correct predictions for each of the four classes, as shown in Figure 4. The high values along this diagonal compared to the off-diagonal elements indicate that the model is able to effectively distinguish between the different disease grades based on the X-ray image features. There is some confusion between neighboring classes; for example, grade 1 is misclassified as grade 0, but very few severe misclassifications. The confusion matrix reflects the high accuracy and robust performance of the CNNs architecture for this classification task.

The accuracy of the model is then calculated as:

The precision, which measures the fraction of correct positive predictions, is given by:

The recall, which measures the fraction of actual positives that are correctly identified, is defined as:

Finally, the F1 score, which is the harmonic mean of precision and recall, is calculated as:

4.2 VGG16 Model with 10 epochs

The VGG16 model trained for ten epochs also achieved good performance, reaching 98% training accuracy and 89% testing accuracy, as shown in Figures 5 and 6. The high training accuracy shows that VGG16 was quickly able to adapt its pre-trained feature hierarchies to the knee OA classification task. The smaller gap between training and testing performance indicates less overfitting compared to the CNNs model. This can be attributed to the regularization effect of transfer learning, where the foundations of the model were pre-trained. The competitive testing accuracy demonstrates the power of fine-tuning VGG16's pre-trained representations on a modest-sized dataset of just a few thousand X-ray images. Even with just ten epochs, VGG16 is able to learn meaningful features for discriminating between different grades of knee osteoarthritis.

Figure 5 Shows accuracy and loss matrix of the (VGG16 with 10 epochs).

Figure 6 VGG16 (10 Epochs) confusion matrix.

The confusion matrix for the 10-epoch VGG16 model also shows strong performance in correctly classifying the knee osteoarthritis severity grades. The diagonal values of 143, 109, 133, and 105 indicate that the majority of samples are predicted correctly for each class, as shown in Figure 6. There is some confusion between the moderate and severe grades, with a few mild cases misclassified as severe and vice versa. This suggests the model may need more training to distinguish better the more damaged cartilage features between these advanced disease stages. The high diagonal values compared to the off diagonals demonstrate that fine-tuning VGG16 for just ten epochs can already achieve good classification accuracy for the four grades of knee osteoarthritis severity from the X-ray images.

The categorical cross-entropy loss, which is commonly used for multi-class classification problems, is defined as:

where $N$ is the number of samples, $K$ is the number of classes, $y_{i,j}$ is a binary indicator (0 or 1) indicating whether the $i$-th sample belongs to class $j$, and $p_{i,j}$ is the predicted probability that the $i$-th sample belongs to class $j$. During the training process, the model's parameters are optimized to minimize this loss function using an optimization algorithm, such as stochastic gradient descent (SGD). The update rule for the model's weights $w$ and biases $b$ using SGD is given by:

where $\alpha$ is the learning rate, and $\frac{\partial L}{\partial w}$ and $\frac{\partial L}{\partial b}$ are the gradients of the loss function with respect to the weights and biases, respectively.

4.3 VGG16 Model with 20 epochs

When taken to 20 epochs of training, the performance of VGG16 improved further, achieving 99.8% training accuracy and 91% testing accuracy see Figure 7. The additional ten epochs allow the model to optimize its weights better to precisely recognize features associated with each OA grade from the X-ray images. The training accuracy is now almost perfect, while the testing accuracy sees a worthwhile boost as well. The model is able to generalize even better, likely due to the depth of representations learned from the convolutional layers pre-trained on a diverse dataset. The results for the 20-epoch VGG16 model demonstrate the benefits of increased training time and highlight the potential of transfer learning for knee OA detection. Competitive performance can be reached rapidly by fine-tuning a robust pre-trained model like VGG16.

Figure 7 VGG16 (20 Epochs) accuracy and loss curve.

Table 2 summarizes the performance evaluation metrics of the proposed CNN model and the VGG16 model across different training epochs. The proposed CNN achieved high accuracy (99%) with a faster training time, while the VGG16 model showed slightly improved recall and F1-Score with increased training epochs. These results highlight the trade-off between performance and computational efficiency.

4.4 Comparison of CNNs and VGG16

The proposed CNNs model, tailored specifically for knee osteoarthritis classification, achieved strong performance but required the most training time. Trained for 50 epochs, it reached 99% training accuracy and 80% testing accuracy but took 161 seconds per step, totaling over 2 hours to converge. The extensive training enables robust learning of domain-specific features through its optimized convolutional and dense layer architecture. In contrast, transfer learning with VGG16 leverages broad pre-trained feature representations, allowing quicker training. The 10-epoch VGG16 attained 98% training accuracy and 89% on the test set in just 528 seconds total. It was extending VGG16 to 20 epochs further enhanced performance, achieving near-perfect 99.8% training accuracy and 91% testing accuracy in 240 seconds per step as shown in the Table 2 and Figure 8.

The additional ten epochs enable better adaptation of VGG16's hierarchical feature extractors to the target task. The proposed CNNs reaches marginally higher testing accuracy, but VGG16 offers a superior speed accuracy trade-off by leveraging pre-trained representations. After just 20 epochs, it matches the CNNs's performance at 3x faster training. For real world applications, the rapid training of transfer learning is preferred over extensive optimization from scratch. VGG16's learned general features transfer well for classifying osteoarthritis severity, reaching competitive results with the CCNNsNN in significantly less training time. Its established architecture also provides more predictable convergence compared to individually designing and tuning a custom CNNs.

Figure 8 Comparative analysis of the proposed models.