- Split View
-
Views
-
Cite
Cite
Jane Smith, Anne Forster, John Young, Use of the ‘STRATIFY’ falls risk assessment in patients recovering from acute stroke, Age and Ageing, Volume 35, Issue 2, March 2006, Pages 138–143, https://doi.org/10.1093/ageing/afj027
- Share Icon Share
Abstract
Objectives: to investigate the predictive validity and reliability of the STRATIFY falls risk assessment tool as applied to patients recovering from acute stroke.
Design: prospective cohort study.
Setting: six stroke rehabilitation units in the North of England.
Subjects: all patients with a diagnosis of acute stroke admitted to the participating stroke units during a 6-month study period.
Assessment: on admission, falls risk (STRATIFY), disability (Barthel index), mobility (Rivermead mobility index), cognitive impairment (abbreviated mental test score) and visual neglect (Albert’s test) were assessed. Then, STRATIFY was completed weekly and within 48 h of anticipated discharge. Consenting patients were contacted at 3 months after discharge to determine falls.
Outcome measures: occurrence of a fall within 28 days of the baseline STRATIFY (in-patient study), falls in the first 3 months after discharge (post-discharge study) and falls during stroke unit stay (reliability study).
Results: from 387 patients admitted to the participating units during the study period, 225 contributed to the 28 day in-patient study, and 234 were followed up at 3 months after discharge. STRATIFY performed poorly in predicting falls in the first 28 days (sensitivity 11.3% and specificity 89.5%) and after discharge (sensitivity 16.3% and specificity 86.4%). Agreement was ‘fair’ between baseline and discharge scores (kappa = 0.263) and ‘good’ between the pre-hospital discharge score and that obtained in the week preceding discharge (kappa = 0.639).
Conclusion: STRATIFY performed poorly as a predictor of falls in a heterogeneous population of stroke patients. There is a need for a disease-specific rather than a generic falls risk assessment tool.
Introduction
It is not widely appreciated that falls are one of the most common medical complications after stroke [1, 2]. Three prospective hospital-based studies [1, 3, 4] have reported a wide fall frequency range of 10.5–46%, suggesting patient characteristics and/or environmental factors are highly influential. To an extent, falls represent an inherent consequence of an active rehabilitation process and therefore cannot be entirely avoided. Nevertheless, it is clearly desirable to minimise their occurrence and to implement targeted fall prevention strategies.
A risk assessment tool, STRATIFY [5], has been developed and is able to predict patients at high risk of falling with clinically useful sensitivity and specificity. It is increasingly used routinely in elderly care departments for this purpose. It comprises five questions about the absence (score 0) or presence (score 1) of falls risk factors: previous falls, agitation, visual impairment, frequent toileting and a mobility score of three or four. The mobility score is obtained by combining the transfer and mobility subsections of the Barthel index [6]. Thus, the score range is 0 (low falls risk) to 5 (high falls risk). STRATIFY is scored on the basis of existing knowledge about the patient and takes about 1 minute to complete. For a mixed elderly population, a score of 2 or more has a sensitivity of 92% and specificity of 68% [5]. In a review of 14 falls risk assessment measures, STRATIFY was identified as one of only two tools to have a clinically relevant sensitivity and specificity [7].
A reliable falls risk indicator for routine use on stroke units would be a clinically useful component of a falls prevention programme, but it cannot be assumed that STRATIFY will perform equally well for stroke patients as for an elderly population with mixed conditions. However, it would seem sensible to test an existing risk indicator before developing a stroke-specific tool. It is possible that STRATIFY could perform adequately for stroke patients but with an adjustment to the cut-off threshold. We therefore undertook a study to investigate the utility of STRATIFY for the specific condition of stroke and to determine the score threshold which optimises sensitivity and specificity.
Methods
The utility of STRATIFY in stroke was investigated in three ways: falls risk assessment in hospital, falls risk assessment in the early post-hospital discharge period and its test–retest reliability.
Setting
The study was conducted in six stroke rehabilitation units in the North of England. These units provide multidisciplinary team care, and the staff are members of a clinical research network, Stroke United Network Yorkshire (SUNY), established to undertake multicentre stroke evaluation research. Staff in these units were trained in the use of STRATIFY and in the collection of baseline study measures. None of these units had used STRATIFY prior to the study.
Study participants
All patients with a diagnosis of acute stroke admitted to the participating stroke units during a 6-month study period were eligible for inclusion in the study. Patients were approached for written consent for follow-up contact. If the patient was unable to give consent, for example, due to aphasia or cognitive impairment, written consent for contact, or assent to contact a third party, was sought from their carer. Ethical approval for the study was obtained from the Multi-centre Research Ethics Committee and from the Local Research Ethics Committees of the participating units.
Assessment
Patient demographic and stroke details were recorded, and a baseline assessment comprising the STRATIFY falls risk assessment, the abbreviated mental test score [8] (cognitive impairment), the Barthel index [6] (disability), Albert’s test [9] (visual neglect) and the Rivermead mobility index [10] (mobility) was completed for all patients by the stroke unit multidisciplinary team (doctor, nurse, physiotherapist and occupational therapist). Any baseline assessment measure not completed within 2 weeks of admission was recorded as missing.
A STRATIFY falls risk assessment was then completed weekly and within 48 h of anticipated discharge for each patient by their primary nurse or at the multidisciplinary team meeting. Question 1 asks about presentation with a fall or falls since admission. As stroke onset can be associated with a fall, this question was modified to ask only about falls occurring since hospital admission. To minimise the falls risk assessment result influencing subsequent management, the STRATIFY assessment form was presented in a yes/no format rather than the usual numerical score, and the completed forms were collected in separate folders rather than filed in the patients notes.
Outcome measure
Patients were classified into two groups ‘fallers’ or ‘not fallers’ using the standard and common sense definition of a fall as ‘an incident in which a patient suddenly and involuntarily comes to rest upon the ground or a surface lower than their original situation’ [11]. Patient falls are routinely recorded in the participating stroke units as part of the hospital critical incident reporting system and risk management. These records were inspected weekly by a researcher, and a standardised proforma was used to record the patient, time, site and circumstances for each fall.
Follow-up
Patients or carers were telephoned at 3 months after discharge to determine fall events. If it was not possible to make contact via telephone, for example, due to aphasia or hearing difficulties, the patient or their carer was contacted by post. Falls information for patients who had been discharged to residential care was obtained from care home staff. Prior to contact, confirmation was sought from the general practitioner that the patient was still registered.
Analysis
Statistical analyses and plots were performed using the software SPSS 11.0.0 and Microsoft Excel. The unit of analysis was the patient rather than each fall as this avoided duplication bias for patients with multiple falls and was consistent with the main purpose for STRATIFY to identify individuals who are at high risk of falls.
In-patient study
The predictive validity of STRATIFY for stroke inpatients was determined during a pre-specified time period of 4 weeks following the baseline assessment to avoid bias through variation in exposure to risk of falls. The incidence of falls (one or more) during the 4 weeks was categorised by the baseline STRATIFY scores of 0–1 and 2–5, and sensitivity, specificity, positive predictive value, negative predictive value and likelihood ratio were calculated with 95% confidence intervals (CI). The relationship between STRATIFY score and falling was explored by plotting the receiver operating characteristic (ROC) curve, and this was used to inform identification of the optimal threshold.
The analysis was repeated, restricting the population to patients who had achieved mobility and therefore with a higher risk of falling. Mobility was defined as a combined transfer and mobility subsection score on the Barthel index of greater than 0, and the first STRATIFY score following the attainment of mobility was used as the baseline.
Post-discharge study
The above analyses and ROC analysis were repeated for falls occurring post-discharge with patients classified by the STRATIFY scores obtained within 48 h of anticipated discharge.
Reliability
The reliability of the STRATIFY score was assessed by comparing the STRATIFY scores of patients who did not fall whilst in the study, since for these patients their falls risk should remain fairly constant. Scores obtained at discharge were compared to the penultimate scores using Cohen’s kappa coefficient.
Results
Three hundred and seventy-eight patients with a diagnosis of stroke were admitted during the study period and 284 consented to follow-up contact (Figure 1). Summaries of demographic and baseline characteristics are shown in Table 1. The main reasons for non-consent were refusal (n = 36), discharge, transfer or death (n = 24) and patients with aphasia or cognitive impairment or non-English speaking who had no carer available (n = 33).
. | n = 359 . |
---|---|
Age [median (range)] | 78 (34–100) |
Sex [male (%)] | 176 (49) |
Side of hemiplegia | |
Left (%) | 185 (52) |
Missing (%) | 2 (0.5) |
Fall in previous 6 months | |
Yes (%) | 57 (16) |
Missing (%) | 44 (12) |
Stroke unit stay (days) (excluding deaths) | |
Median (range) | 44 (3–200) |
Missing (%) | 1 (0.3) |
Discharge destination | |
Own home (%) | 229 (64) |
Nursing/residential (%) | 89 (25) |
Other (%) | 15 (4) |
Missing (%) | 26 (7) |
Barthel index | |
Median (range) | 7 (0–20) |
Missing (%) | 43 (12) |
Rivermead mobility index | |
Median (range) | 3 (0–15) |
Missing (%) | 48 (13) |
Albert’s test (visual neglect) | |
Abnormal (%) | 33 (9) |
Missing (%) | 92 (26) |
Abbreviated mental test | |
≤6 (%) | 59 (16) |
≥7 (%) | 200 (56) |
Missing (%) | 100 (28) |
. | n = 359 . |
---|---|
Age [median (range)] | 78 (34–100) |
Sex [male (%)] | 176 (49) |
Side of hemiplegia | |
Left (%) | 185 (52) |
Missing (%) | 2 (0.5) |
Fall in previous 6 months | |
Yes (%) | 57 (16) |
Missing (%) | 44 (12) |
Stroke unit stay (days) (excluding deaths) | |
Median (range) | 44 (3–200) |
Missing (%) | 1 (0.3) |
Discharge destination | |
Own home (%) | 229 (64) |
Nursing/residential (%) | 89 (25) |
Other (%) | 15 (4) |
Missing (%) | 26 (7) |
Barthel index | |
Median (range) | 7 (0–20) |
Missing (%) | 43 (12) |
Rivermead mobility index | |
Median (range) | 3 (0–15) |
Missing (%) | 48 (13) |
Albert’s test (visual neglect) | |
Abnormal (%) | 33 (9) |
Missing (%) | 92 (26) |
Abbreviated mental test | |
≤6 (%) | 59 (16) |
≥7 (%) | 200 (56) |
Missing (%) | 100 (28) |
. | n = 359 . |
---|---|
Age [median (range)] | 78 (34–100) |
Sex [male (%)] | 176 (49) |
Side of hemiplegia | |
Left (%) | 185 (52) |
Missing (%) | 2 (0.5) |
Fall in previous 6 months | |
Yes (%) | 57 (16) |
Missing (%) | 44 (12) |
Stroke unit stay (days) (excluding deaths) | |
Median (range) | 44 (3–200) |
Missing (%) | 1 (0.3) |
Discharge destination | |
Own home (%) | 229 (64) |
Nursing/residential (%) | 89 (25) |
Other (%) | 15 (4) |
Missing (%) | 26 (7) |
Barthel index | |
Median (range) | 7 (0–20) |
Missing (%) | 43 (12) |
Rivermead mobility index | |
Median (range) | 3 (0–15) |
Missing (%) | 48 (13) |
Albert’s test (visual neglect) | |
Abnormal (%) | 33 (9) |
Missing (%) | 92 (26) |
Abbreviated mental test | |
≤6 (%) | 59 (16) |
≥7 (%) | 200 (56) |
Missing (%) | 100 (28) |
. | n = 359 . |
---|---|
Age [median (range)] | 78 (34–100) |
Sex [male (%)] | 176 (49) |
Side of hemiplegia | |
Left (%) | 185 (52) |
Missing (%) | 2 (0.5) |
Fall in previous 6 months | |
Yes (%) | 57 (16) |
Missing (%) | 44 (12) |
Stroke unit stay (days) (excluding deaths) | |
Median (range) | 44 (3–200) |
Missing (%) | 1 (0.3) |
Discharge destination | |
Own home (%) | 229 (64) |
Nursing/residential (%) | 89 (25) |
Other (%) | 15 (4) |
Missing (%) | 26 (7) |
Barthel index | |
Median (range) | 7 (0–20) |
Missing (%) | 43 (12) |
Rivermead mobility index | |
Median (range) | 3 (0–15) |
Missing (%) | 48 (13) |
Albert’s test (visual neglect) | |
Abnormal (%) | 33 (9) |
Missing (%) | 92 (26) |
Abbreviated mental test | |
≤6 (%) | 59 (16) |
≥7 (%) | 200 (56) |
Missing (%) | 100 (28) |
One hundred and eight patients (30%) fell at least once between the baseline STRATIFY assessment and discharge. Of the 234 patients who were followed up, 80 (34%) fell at least once during the 3 months after discharge.
In-patient study
Two hundred and twenty-five patients (59%) contributed to the 28 day in-patient STRATIFY study. The other 153 were excluded from the analysis because of incomplete data sets, because discharge was less than 28 days after baseline assessment (n = 113) or because of missing data (n = 40).
There was little difference in the percentage of fallers for either of the STRATIFY thresholds (0–1 = 25% fallers; 2–5 = 23% fallers) in the first 28 days. For the higher STRATIFY cut-off of 2 or more, the sensitivity of the baseline STRATIFY was found to be low (11.3%, 95% CI 2.8–19.9) as few fallers scored 2 or more. The positive predictive value of the baseline STRATIFY assessment was also low (25.0%, 95% CI 7.7–42.3), and the likelihood ratio was imprecise (1.08, 95% CI 0.45–2.58) (Table 2). Restricting the primary analysis to patients who had achieved mobility (n = 199) still only resulted in a low sensitivity (15.6%, 95% CI 6.7–24.5).
. | Baseline STRATIFY assessment . | . | Discharge STRATIFY assessment . | . | ||
---|---|---|---|---|---|---|
. | Estimate (%) . | 95% CI . | Estimate (%) . | 95% CI . | ||
Sensitivity | 11.3 | 2.8–19.9 | 16.3 | 8.2–24.3 | ||
Specificity | 89.5 | 85.0–94.1 | 86.4 | 80.9–91.8 | ||
Positive predictive value | 25.0 | 7.7–42.3 | 38.2 | 21.9–54.6 | ||
Negative predictive value | 76.6 | 70.8–82.5 | 66.5 | 60.0–73.0 | ||
Likelihood ratio | 1.08 | 0.45–2.58 | 1.19 | 0.63–2.25 |
. | Baseline STRATIFY assessment . | . | Discharge STRATIFY assessment . | . | ||
---|---|---|---|---|---|---|
. | Estimate (%) . | 95% CI . | Estimate (%) . | 95% CI . | ||
Sensitivity | 11.3 | 2.8–19.9 | 16.3 | 8.2–24.3 | ||
Specificity | 89.5 | 85.0–94.1 | 86.4 | 80.9–91.8 | ||
Positive predictive value | 25.0 | 7.7–42.3 | 38.2 | 21.9–54.6 | ||
Negative predictive value | 76.6 | 70.8–82.5 | 66.5 | 60.0–73.0 | ||
Likelihood ratio | 1.08 | 0.45–2.58 | 1.19 | 0.63–2.25 |
. | Baseline STRATIFY assessment . | . | Discharge STRATIFY assessment . | . | ||
---|---|---|---|---|---|---|
. | Estimate (%) . | 95% CI . | Estimate (%) . | 95% CI . | ||
Sensitivity | 11.3 | 2.8–19.9 | 16.3 | 8.2–24.3 | ||
Specificity | 89.5 | 85.0–94.1 | 86.4 | 80.9–91.8 | ||
Positive predictive value | 25.0 | 7.7–42.3 | 38.2 | 21.9–54.6 | ||
Negative predictive value | 76.6 | 70.8–82.5 | 66.5 | 60.0–73.0 | ||
Likelihood ratio | 1.08 | 0.45–2.58 | 1.19 | 0.63–2.25 |
. | Baseline STRATIFY assessment . | . | Discharge STRATIFY assessment . | . | ||
---|---|---|---|---|---|---|
. | Estimate (%) . | 95% CI . | Estimate (%) . | 95% CI . | ||
Sensitivity | 11.3 | 2.8–19.9 | 16.3 | 8.2–24.3 | ||
Specificity | 89.5 | 85.0–94.1 | 86.4 | 80.9–91.8 | ||
Positive predictive value | 25.0 | 7.7–42.3 | 38.2 | 21.9–54.6 | ||
Negative predictive value | 76.6 | 70.8–82.5 | 66.5 | 60.0–73.0 | ||
Likelihood ratio | 1.08 | 0.45–2.58 | 1.19 | 0.63–2.25 |
When the lower STRATIFY threshold of 0/1 was used, sensitivity was higher, but the corresponding specificity was lower. No truly acceptable threshold could be found, but the best was 0/1 with a sensitivity of 60.4% and specificity of 58.1%. The ROC ‘curve’ was almost a straight line indicating a test with poor discrimination.
Post-discharge study
Two hundred and thirty-four patients (82%) were followed up at 3 months after discharge. Of those who were not followed up, 29 had died, 18 could not be contacted, 1 was too ill and 2 had been transferred to other hospitals. STRATIFY performed poorly in the identification of falls in the 3 months after hospital discharge, either using the recommended score threshold of 2 or more (Table 2) or using a score threshold of 0/1. The ROC ‘curve’ was, as before, almost a straight line.
Reliability
There were 251 (69.9%) patients who had no falls on the stroke unit after the baseline STRATIFY assessment. When the discharge scores were compared to the scores obtained during the week preceding discharge, agreement was ‘good’ (kappa = 0.639).
Discussion
The falls risk assessment instrument STRATIFY has not previously been investigated specifically for stroke patients although previous study cohorts have included some patients with a diagnosis of stroke [5, 12]. Our study involved a heterogeneous population of stroke patients with generalisability achieved by recruiting consecutive admissions to six stroke unit sites. For this patient group, STRATIFY performed poorly as a predictor of falls occurring in the 28 days following a baseline assessment. Using the currently recommended score threshold (2 or more as high risk), STRATIFY was found to be imprecise with a sensitivity of only 11.3%. We found the optimum STRATIFY threshold for our stroke patients was a score of one or more, but even optimising the risk indicator resulted in a sensitivity of only 60% and this was achieved at the cost of reduced specificity. Similar imprecise results were obtained using a pre-discharge assessment to predict falls in the 3 months after discharge.
There are some potential limitations to this study that should be considered. Patients were not generally admitted directly to the participating stroke units but were transferred following a stay on other wards. It has been reported that falls on elderly and rehabilitation wards are most frequent shortly after admission [13]. The median time between admission to hospital and transfer to the participating stroke unit was 11 days. It may be that staff on the stroke units were not aware of falls occurring prior to transfer and therefore failed to include them in the STRATIFY baseline score. It has also been observed likely that hospital incident reports under-report fall events [4, 14]. However, for three of the six units in our study, falls records were completed by a research nurse who also worked on the unit and therefore more likely to be aware of patients who had fallen. In the other units, inspection of the ward incident reports was supplemented by inspection of the patient nursing notes and questioning of staff.
It has previously been suggested that the performance of STRATIFY may be affected by inconsistencies in its completion [5]. Two studies have investigated reliability and reported good inter-observer agreement following minimal training with either an explanation [12] or a 10-minute training session [15]. In our study, sessions lasting ~45 minutes were held in all units to train staff in the use of STRATIFY, and instructions were included on the STRATIFY form to guide the completion of question 5 (transfer and mobility). Although we did not undertake inter- or intra-rater reliability testing, our approach was more intensive than these earlier studies, and it seems unlikely that the poor performance of STRATIFY might be ascribed to instrument unreliability. Indeed, the test–retest reliability study (discharge and pre‐discharge) demonstrated good agreement between the two observations.
An underestimate of the predictive validity of STRATIFY might have occurred either because the introduction of a falls risk assessment raised awareness of falls as a general issue or because knowledge of a particular STRATIFY score led to a change in care for individual patients [5, 15]. To minimise these effects, the STRATIFY assessment was modified from its usual numerical format to yes/no questions, and the form was filed separately from the health records.
Another possible reason for the poor performance of STRATIFY in this study is because the constituent items of STRATIFY include some that are not relevant for all stroke patients. For example, ‘falls since hospitalisation’ and ‘mobility’ may not be relevant for immobile patients, so their score range is effectively restricted to 0–3 (only two patients had a score of 4, and none had a score of 5). However, restricting the primary analysis to the patients who had achieved mobility (combined transfer and mobility subsection scores on the Barthel index greater than 0) still only resulted in a sensitivity of 16%.
Several factors have been associated with falls in stroke patients, including male sex [16], increased motor response time to visual stimuli [17], postural sway [18], rate of rise in force during sit-to-stand movements [19] and depressive symptomatology [20, 21]. The inclusion of these and other risk factors may be more successful in predicting falls in stroke patients. A recent validation study [22] of a previously developed falls index [16], based on variables associated with falls risk in stroke patients, reported some correlation with falls risk, although accuracy was not acceptable. Following remodelling, however, the index showed higher association with falls risk, and the authors suggest that this could be used as a foundation for future research in the area. Our study would support the need for a disease-specific rather than a generic falls risk assessment tool for patients with stroke.
Key points
Falls are one of the most common medical complications after stroke.
A reliable falls risk indicator for routine use on stroke units would be a clinically useful component of a falls prevention programme.
STRATIFY performed poorly as a predictor of falls in patients recovering from stroke.
There is a need for a stroke-specific rather than a generic falls risk assessment tool.
Conflicts of interest
None to declare.
Declaration of sources of funding
Funded by The Stroke Association.
Acknowledgements
We thank consultant physicians, research assistants and staff at all centres who participated in the project: Ward 7B, Calderdale Royal Hospital (Dr I. Shakir, Carol Whitworth); Ward 10, Chapel Allerton Hospital (Dr P. Wanklyn, Heather Priestley); N ward, Seacroft Hospital (Dr N. Dudley, Heather Priestley); Rehabilitation Unit, Pinderfields Hospital (Dr A. Stanners, Caroline Marsden and Alison Sharp); Ward 39, York District Hospital (Dr J. Harbison, Carol Croser and Ann Lawman); and Stroke unit (F5), St Luke’s Hospital, Bradford (Dr C. Patterson.). We thank Dr Sue Bogle for statistical services. We are grateful to all the patients and carers who participated and to The Stroke Association who funded the project.
Comments