Design
This is a methodological reliability study of a novel system to evaluate CCF movement control using the CCF ROM to define the targets. The study followed a reproducibility protocol with three phases: training, overall agreement and study [35]. The training and overall agreement phases have already been finished and were the basis of the design used in the present study. Therefore, this study addresses the study phase of the reproducibility protocol, as well as the analysis of its safety and usability. In the training and overall agreement periods, the examiners designed and developed the new tool and discussed and agreed on how the test should be performed, the use of the inertial wearable sensors and the associated software. The procedure described in this section to assess the reliability was based on the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) methodology [31].
The study was approved by the Research Ethics Committee of CEU San Pablo University (236/17/08) and performed in accordance with the Declaration of Helsinki and World Health Organization (WHO) standards [32]. The participants received a complete verbal description of the procedures and the purpose of the study and gave their written informed consent before enrolling in the study. They were able to withdraw at any time during the study.
Sample and selection
Both asymptomatic participants and patients with chronic neck pain between 18 and 65 years old were invited to participate. They were recruited through convenience sampling via e-mails and informational posters at the CEU San Pablo University and the CEU San Pablo University clinic. The inclusion criteria for patients with chronic neck pain included the following: > 3 months of pain duration, a score of > 4 on the Neck Disability Index (NDI) [38] and a score of > 3 on the visual analogue scale [33, 34]. Healthy participants had to have a score of 0 on the NDI and the visual analogue scale. The exclusion criteria were previous surgery in the neck or head area, previous diagnosis of headaches and temporomandibular disorders and any neurological deficits. Immediately after the participant signed the informed consent and was screened based on the selection criteria, they provided information about their demographic characteristics (age, weight, height and occupation).
CFMCT measured by inertial sensor technology
The construct of this novel test is the flexion movement control capacity through different levels of the whole CCF ROM of the participant. The tests require the subject to accurately achieve and maintain different CCF ROM targets guided by software displayed on a computer/tablet screen as biofeedback. This section describes the inertial sensor-based system used for the novel CFMCT based on the list of components of outcome measurement instruments of the COSMIN methodology.
Preparatory actions preceding data collection
Preparation of the participant by the examiner
The participant was placed in the supine position with their arms resting on their abdomen, their knees bent and their feet resting on the table. The procedure for standardising the initial neutral cervical position was to ask the subject to perform three maximum CCF repetitions, relaxing the flexors to return from each maximum CCF to neutral while avoiding contraction of the extensor muscles. The initial position of the test was the one achieved after relaxing the flexors in the previous repetition. This procedure was designed to avoid a lack of standardisation of the initial position based on anatomical landmarks, which are highly influenced by anatomical variability between subjects.
Preparation of equipment by the examiner
Once the initial position was achieved, a single 4 × 4 × 8 cm wireless wearable inertial sensor (Werium Solutions©, Madrid, Spain) was placed on the participant’s forehead with a double-sided tape. Before starting each measurement, the inertial sensor was calibrated to zero. This inertial sensor allows real-time tracking of the progressive ROM increase while performing CCF. This instrument has previously demonstrated good to excellent inter- and intra-rater reliability in the assessment of global neck movement [35] and CCF movement [36].
The inertial sensor contains a Micro-Electro-Mechanical Systems-based IMU (Inertial Measurement Unit) with 9 degrees of freedom. This IMU integrates three distinct sensors: a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis magnometer. The sensor also includes a microcontroller unit which is responsible for acquiring data from the sensors via I2C and computing the angular orientation (yaw, pitch, and roll) before forwarding it to the communications module. The IMU operates at a sample rate of 50 Hz. [35] This sample rate ensures accurate and real-time monitoring of head movements, allowing precise motor control during the CCMCT.
Computer
A computer screen, which displays the ROM during CCF captured by the inertial sensor, was positioned in front of the subject’s eyes at a comfortable distance. The computer screen was placed in a horizontal position parallel to the bed to standardise the initial position. This could be performed with the help of an articulated arm (Fig. 1). Real-time feedback was provided by innovative software that was created for the purpose of this study (described below).
Displayed computer biofeedback screen and experimental setup. Image on the left shows the screen displayed for the patient used as a biofeedback. The red line represents the craniocervical flexion range of motion achieved during the test over time. The blue coloured bars represent the range of motion stages the patient should reach and maintain. Image on the right shows the experimental setup that includes a tablet attached to an articulated arm and the inertial sensor sticked to the forehead
Characteristics and settings of the beta version of the software
Once the participant was in the initial position, they saw two references to look at on the computer screen: first, a red line indicating the trajectory of the movement in terms of the achieved CCF ROM; second, blue stripes that they must reach and stay inside (see the supplementary video). To set the degrees of CCF ROM to be achieved in each blue stripe (levels), it was necessary to determine the maximum CCF of each participant, because the blue stripes represent the percentage of the maximum CCF ROM of each subject (see calculation below). Therefore, each participant performed three maximum CCF movements, and the software provided the examiner with the maximum CCF ROM reached by the subject.
The settings of the software to perform the CFMCT are described below (see Fig. 1).
-
1.
Number of stages: this test was designed to reach a total of nine stages of a percentage of the maximum CCF ROM. The first five stages of ROM percentage targets were based on the CCFT pressure levels as reported in a previous study [19]. The next four stages were added based the mean average increment between the previous stages, so the last stage was the closest possible to the maximal CCF ROM.
-
2.
Stages based on ROM values: the percentages of the maximum CCF ROM where the blue stripes are displayed on the screen were: stage 1, 20.7%; stage 2, 30.7%; stage 3, 45.2%; stage 4, 52.2%; stage 5, 61.6%; stage 6, 70%; stage 7, 79%; stage 8, 88%; stage 9: 97%.
-
3.
Stage level limits: the limits for each stage were 0.3° above and below the target CCF ROM level.
-
4.
Duration of the stage (seconds): the duration of each stage (blue stripes) was 3–4 s (depending on the protocol [see below]), while the time between the stages was 4 s. This duration is based on the analysis of performance of the CCF action within the clinical protocol for the CCFT [10].
-
5.
Appearance of the stage (seconds): this setting dictated at what time (in seconds) on the timeline from the start each blue stripe would appear. On the screen, the subject saw how each blue stripe appeared in the previously configured stage.
Test performance
After performing the three CCF repetitions, the instructions for the participant were:
The red line you see on the screen corresponds to your movement. Your goal from now on is to reach and stay within the blue lines (stripes) that will sequentially appear on the screen performing the nodding action.
We designed two different and independent protocols to assess CCF movement control. Both protocols were based on the stages of the CCFT explained above, but one of these protocols showed the nine stages in a progressive order (progressive consecutive stages protocol), while the other showed them in a predetermined random order (random stages protocol). The objective of both protocols was to measure the accuracy of the subjects to reach the targeted CCF ROM and be able to control the movement during the ROM.
For the progressive consecutive stages protocol (see the supplementary video), the nine stages increased progressively from 20.7 to 97% of maximum CCF ROM. After reaching each CCF stage, the subject must return to a neutral position, which is represented as a blue stripe at zero (same as the initial position where the sensor was calibrated). The stripes appeared for 3 s in the CCF position and for 4 s in the neutral position (0° of CCF; resting stages). The final score was based on the points obtained for all stages, including the neutral stages. The calculation of the final score is explained in more detail below.
For the random stages protocol, the previously mentioned stages were randomised, generating a more dynamic and less predictable movement pattern for the participant. The blue stripes appeared for 4 s and there were no resting stages; thus, the muscular demand was greater that for the progressive consecutive stages protocol. Once randomised, the template created with the same randomised order of the percentages was used for all participants.
Reliability design
The reliability of the CFMCT was assessed through both intra- and inter-rater reliability. During the first measurement day (first measurement), each subject performed both protocols with each examiner; therefore, each subject performed each protocol twice. This measurement protocol was repeated 1 week later (second measurement). Therefore, inter-rater reliability was calculated by comparing the measurements obtained on each measurement day between examiner 1 and examiner 2, and intra-rater reliability was calculated by comparing the measurements of each examiner obtained for the first and second measurement. The examiners were blinded regarding the protocol to minimise any potential influence or bias in the assessment. When one examiner was conducting both protocols with the subject, the other examiner was not present in the room.
Data processing and storage
Each measurement was automatically saved in the software and exported as an Excel file with the following information:
-
1.
degrees of ROM flexion at each time point – indicates the CCF ROM every 20 milliseconds, when the inertial sensor records movement information;
-
2.
stage reached – indicates whether the CCF ROM achieved at each time point was inside the blue stripe of the corresponding stage displayed at that time on the computer biofeedback.
Assignment of the score
The final score was calculated as the percentage of time points the participant was within the blue stripe area and ranges from 0% (lowest movement control) to 100% (highest movement control). Therefore, if a participant is within all blue stripes during the test, the total score is 100%.
Summary of the study protocol for reliability analysis
After providing information about their pain and demographic characteristics, the entire measurement procedure was explained to the participant. The study was conducted by two examiners, who each performed both protocols for each subject. The order of the examiners and the protocols was randomised by using an online randomiser (https://www.randomizer.org).
The first examiner guided the participant through a seated warm-up of three movements in each of the three planes of neck movement: flexion-extension, right and left tilts and right and left rotations. The subject also practised the CCF movement, which the examiner described as a gentle and slow head-nodding action of ROM flexion [10]. The examiner gave the following instructions:
‘Please lower your chin towards your neck (nodding movement) in a controlled manner without moving your neck, such that the head rotates slightly, reaching as far as possible. The posterior side of the head will slide smoothly on the bed during the movement and the head should not separate from the bed or push into it during the movement’.
If necessary, the examiner corrected the movement verbally and manually [28].
Once the participant was lying on the bed, they performed the CCF movement as a training exercise. Once the subject had properly executed the nodding movement without any compensations after warming up, the measurement started. As mentioned above, the neutral cervical starting position was obtained after three maximum repetitions of CCF. At this point the inertial sensor was placed and calibrated. Each participant performed three maximum CCF repetitions to obtain the maximum CCF for each participant and to calculate the ROM of each stage of the test. At this point, the examiner explained to the participant that their objective was to reach the blue stripes that would appear on the screen and to remain steady for as long as possible. In addition, the examiner explained to the subject that they should reach the targets by correctly performing the previously learned CCF movement, without making any compensatory movements.
The participant performed each protocol once for each examiner, with a 5-minute resting period between the examiners. Therefore, by the end of the test each participant had performed each protocol twice.
After completing the test, each participant filled out a data sheet that included the following: (a) the intensity of pain during the test based on the visual analogue Scale [37], (b) muscular fatigue during the test based on the fatigue VAS, (c) any other perceived symptoms or adverse event (e.g. nausea, dizziness) via an open question and (d) the System Usability Scale (SUS) [38]. The SUS includes 10 statements about the perceived usability of the software. Each statement received a score of 1–5, where 5 corresponds to strongly agree and 1 corresponds to strongly disagree. The odd-numbered questions constitute positive statements and are scored by subtracting one point from the user’s rating. The even-numbered questions represent negative statements and are scored by subtracting 5 points from the user’s rating. To obtain the final SUS score, the sum of all answers is multiplied by 2.5. The resulting final score ranges from 0 (indicating low usability) to 100 (reflecting high usability) [39]. A final score between 68 and 84 points indicates good usability of the system and a score > 84 points indicates excellent usability [38].
Data analysis plan
All data were analysed using SPSS Statistics version 27.0 (IBM Corp., Armonk, NY USA) and R version 4.1.3 (R Foundation for Statistical Computing, Vienna, Austria). The Kolmogorov–Smirnov test was used to determine whether the quantitative data followed a normal distribution; it showed that the data did not follow a normal distribution. Quantitative variables are presented as mean ± standard deviation (SD) and categorical variables are presented as absolute values and frequencies (%). The final score of each test was calculated as the percentage of time points the participant was within the blue stripes.
The reliability of the CFMCT was determined with intraclass correlation coefficients (ICCs) based on a two-way random effects model, an average measure and an agreement definition [40] The reliability analysis included intra-rater reliability (first measure versus second measure) and inter-rater reliability (examiner 1 versus examiner 2) of the measures. Based on the 95% confidence interval, the ICC agreement was interpreted as follows: < 0.5 indicates poor reliability, between 0.50 and 0.75 indicates moderate reliability, between 0.75 and 0.90 indicates good reliability and > 0.90 indicates excellent reliability [41].
The SEM was calculated using the formula SEM = SD × [√(1 – ICCagreement)] [40].
The sample size was based on international guidelines for reliability studies that have reported 40 subjects are sufficient for reproducibility [42].
Add Comment