As reported in [5], the image registration based JSN progression quantification

framework delivers high accuracy and sensitivity in the monitoring of RA. Further, [5] reports an improved phase only correlation algorithm called “partial image phase only correlation (PIPOC)”. PIPOC can simultaneously monitor the movement of multiple objects with sub-pixel accuracy. When compare to the related works, PIPOC shows promising potential in RA monitoring and diagnosis. However, from the experiments we learnt that an inconsistent angle between the upper and lower bones of the joints can affect the accuracy of JSN progression quantification, and it may also lead to mismatches. Even though using guide lines one may standardize hand posture when taking radiographic images, thus effectively reducing the number of inconsistent joint angles. However, RA can cause ankylosis in peripheral synovial joints and impair the movement and flexibility of digits, and consistent joint angle may be difficult for RA patients with limited finger function. Therefore, we introduce rotation invariant phase only correlation (RIPOC) [7] based joint angle correction in the JSN progression quantification to control such mismatch cases.

The basic pipeline of the JSN progression quantification with joint angle correction can be described as shown in Figure 1. Consider two joint radiographic images, a baseline joint radiography f ( x , y ) and a follow-up joint radiography g ( x , y ) . The joint angle of f ( x , y ) and g ( x , y ) is different. These two joint radiographic images can be divided into upper and lower bones. The segmentation information is included in a set of binary matrixes s i ( x , y ) ( i = 0 indicates upper bone and i = 1 indicates lower bone). The θ 0 and θ 1 represent the angle difference of upper and lower bone respectively. The ( α 0 , β 0 ) and ( α 1 , β 1 ) represent sub-pixel displacement of upper and lower bones respectively. Then, the follow-up joint radiography g ( x , y ) can be represented by the baseline joint radiography f ( x , y ) as Equation (1).

g ( x , y ) = ∑ i = 0 1 f ( x cos θ i + y sin θ i + α i , − x sin θ i + y cos θ i + β i ) ∗ s i ( x , y ) (1)

As reported in [5], segmentation in frequency domain can affect control the phase dispersion in phase difference spectrum when compared to segmentation in spatial domain. Take an example of quantifying the movement of bones, the phase spectrum of upper bone f ^ 0 ( x , y ) , g ^ 0 ( x , y ) and lower bone f ^ 1 ( x , y ) , g ^ 1 ( x , y ) can be denoted as Equation (2) and (3).

f ^ i ( x , y ) = F − 1 ( F ( f ( x , y ) w ( x , y ) ) / | F ( f ( x , y ) w ( x , y ) ) | ) ∗ s i ( x , y ) (2)

g ^ i ( x , y ) = F − 1 ( F ( g ( x , y ) w ( x , y ) ) / | F ( g ( x , y ) w ( x , y ) ) | ) ∗ s i ( x , y ) (3)

Here, F denotes the 2D Fast Fourier transform (FFT), and F − 1 denotes the 2D Inverse Fast Fourier transform (IFFT). w ( x , y ) denotes the window function, we used a Hanning window in this work.

Next, the bone angle of follow-up radiography is corrected to baseline radiography by using RIPOC [7], as shown in Figure 1. Let M f ^ i ( ξ , η ) and M g ^ i ( ξ , η ) denote the magnitudes of f ^ i ( ξ , η ) and g ^ i ( ξ , η ) respectively. Combining the Fourier transform property and the Fourier transform rotation property, we can represent the magnitude as in Equation (4).

M g ^ i ( ξ , η ) = M f ^ i ( ξ cos θ i + η sin θ i , − ξ sin θ i + η cos θ i ) (4)

Rotation can be represented as a translation displacement in polar coordinates, as given in Equation (5).

M g ^ i ( ρ , θ ) = M f ^ i ( ρ , θ − θ i ) (5)

As shown in Figure 1, combining the Equation (5) and the full image phase only correlation, the angle difference θ i can be easily calculated. Next, the corrected phase spectrum of upper bone of follow-up radiography g ^ ′ i ( x , y ) can be represented as follow.

g ^ ′ i ( x , y ) = f ^ ′ i ( x + α i , y + β i ) (6)

The normalized cross phase spectrum R ^ i ( u , v ) of g ^ ′ i ( x , y ) and f ^ i ( x , y ) with Fourier transform can be defined as Equation (7).

R ^ i ( u , v ) = F ( f ^ i ( x , y ) ) F ( g ^ ′ i ( x , y ) ) ¯ / | F ( f ^ i ( x , y ) ) F ( g ^ ′ i ( x , y ) ) ¯ | (7)

Therefore, the location of the peak of Dirac delta function δ in the phase difference spectrum can be determined as

( α ′ i , β ′ i ) = arg max ( x , y ) F − 1 ( R ^ i ( u , v ) ) (8)

The least-square fitting method is employed to estimate the displacement ( α i , β i ) around the maximum peak ( α ′ i , β ′ i ) . Therefore, the J S N f g between image f ( x , y ) and image g ( x , y ) can be quantified as Equation (9).

J S N f g = β 0 − β 1 (8)

In this section, we propose a metric to quantify the reliability of the JSN progression quantification with joint angle correction. This metric can be used to evaluate the probability of mismatch. This reliability metric is determined based on phase dispersion.

According to the experiments in [5], changes in bone contours can affects the accuracy of JSN progression quantification, and result in phase dispersion in the phase difference spectrum. This phase dispersion appears as multiple low peaks in the phase difference spectrum instead of a single Dirac delta function. This phase dispersion can affect the Dirac delta function position quantization and cause measurement errors or even mismatches. Figure 2 shows two 3D graph of the phase difference spectrum for the clinical images, and the maximum peak value for each spectrum is normalized as 255.

By comparing the phase difference spectrums in Figure 2, we can easily observe that the phase dispersion of a mismatch case is clearly higher than that of the success case. In this study, the phase dispersion is used to represent a confidence index using standard deviation. For the normalized image, to suppress the influence of the main Dirac delta function, the standard deviation of the entire image is obtained by setting the 3 × 3 pixels around the peak as 0, and the sum of the upper and lower bones are used as the confidence index.

We used the dataset from Sagawa Akira Rheumatology Clinic (Sapporo, Japan), Sapporo City General Hospital (Sapporo, Japan) and Hokkaido Medical Center for Rheumatic Diseases (Sapporo, Japan). The dataset contains 1120 hand posteroanterior projection radiographic images from patients in the early stages of RA [5].

The radiographic imaging device used in preparing this dataset is DR-155HS2-5 from Hitachi Corporation, with 1.5 mm X-ray aluminum filter thickness. The centering point of the X-ray beam is the MCP joint of the middle finger. Digital imaging and communications in medicine (DICOM) standard is used in preparing and managing this dataset. The size of each image is 2010 × 1490 pixels, the spatial resolution is 0.175 × 0.175 mm, and the bit depth is 12 bits. The tube voltage of X-ray is 42 kV, the tube current is 100 mA. The exposure time of image is 20 msec, and the distance between the X-ray source and the image is 100 cm.

For the confidence index, the mean value for the measured clinical image dataset is 40.83, whereas the mean for the 10 pairs of images with confirmed mismatches are 79.42. Figure 3 shows an example of the joint images that are measured and their confidence levels respectively. The standard deviation with no rotation is 22.12, while the standard deviation with rotation is 50.82, indicating that the standard deviation increases as the mismatch probability increases.

We invite one radiologist and one radiological technologist in our experiment. According to the confidence index, we divide the JSN progression quantitative results into high index group (mismatch group) and low index group (success group). Both the radiologist and the radiological technologist find that the JSN progression quantitative results in the low index group are more consistent with the trends they assessed.

In the literature, there are no means to discriminate the mismatch of a successful case in phase only correlation. But the proposed confidence index can help joint space narrowing progression quantification algorithm to be applied in clinical environment, and improve the accuracy of the measurement by giving a reference for the reliability. In addition, the best quantization result can be selected from multiple quantization results according to the confidence index, which will result in higher accuracy.