# Evidence-Based Practice (Technical Analyses) Examiners need to know that the interpretations they make are supported by data-based evidence. In this chapter, we review the types of questions examiners should ask about the nature of any test, which will inform their conclusions about test performance. Therefore, this section covers validity and reliability information as well as information concerning analysis of test bias. We continue to use the concept of constrained and unconstrained skills (cf. Paris, 2005) to interpret results. To review, constrained subtests tap skills that develop relatively early in development (e.g., knowing one’s name, naming common items). Unconstrained subtests tap skills that tend to show continued refinement over a wide range of ages. The PTBI contains both types of subtests, and the statistical results are presented separately for each type. A question and answer framework is used to describe the evidence base related to various clinical questions about the nature of the PTBI. ## Technical Notes ### Question 1: How do I interpret the criterion categories? The categories of performance for each age are based on the performance of the group of children without brain injuries selected to represent the general population. The percentages of these children whose scores fell into each of the four criterion categories are provided in Table 6.1. As this table shows, most scores fell into the High performance category. The differences in the percentages of children classified into the different performance categories are... ## EVIDENCE FOR CONSTRUCT-IDENTIFICATION VALIDITY | | Performance criterion categories | | | | | --- | --- | --- | --- | --- | | Subscale | Very Low | Low | Moderate | High | | 1. Orientation | 0% | 6% | 19% | 75% | | 2. Following Commands | 7% | 8% | 2% | 83% | | 3. Word Fluency | 3% | 20% | 25% | 53% | | 4. What Goes Together | 2% | 1% | 19% | 78% | | 5. Digit Span | 2% | 6% | 30% | 62% | | 6. Naming | 0% | 4% | 3% | 93% | | 7. Story Retelling-Immediate | 2% | 10% | 28% | 60% | | 8. Yes/No/Maybe | 0% | 5% | 18% | 77% | | 9. Picture Recall | 0% | 5% | 35% | 60% | | 10. Story Retelling-Delayed | 0% | 3% | 20% | 77% | Children were not screened for other conditions that occur in the general population that could affect performance (e.g., learning disability). Therefore, this group is intended to reflect the non–brain-injured general population. Sixty-nine of the children were white, twenty-three were African American, four were Hispanic, two were reported as “other,” and one was Asian. ## Technical Notes Ninety-nine children’s scores were compared for this source of evidence. Thirty-three children were included in each of the three groups (TBI, ABI, typical). These children were matched by age (mean age of 10 years, 7 months; range 6–16 years) and gender (14 males, 19 females per group). A variety of racial/ethnic groups were included (TBI: 26 white, 7 minority; ABI: 17 white, 16 minority; Typical: 22 white, 11 minority). The majority of the children in each group were right-handed (TBI: 27 right-handed; ABI: 27 right-handed, typical: 25 right-handed). Ability scores were tested using t tests with statistical significance at p<.05. --- ## Tables ### Table 6.2. Raw score means (standard deviations) for matched groups of children with Traumatic Brain Injury (TBI; n = 33), other acquired brain injury (ABI; n = 33), and typically developing children (Typical, n = 33). | Category | Constrained skills | | | Unconstrained skills | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | | 1. Orientation | 2. Following Commands | 6. Naming | 3. Word Fluency | 4. What Goes Together | 5. Digit Span | 7. Story Retelling—Immediate | 8. Yes/No/Maybe | 9. Picture Recall | 10. Story Retelling—Delayed | | TBI | $30.8^{a,b}$ (8.25) | $24.94^{a,c}$ (6.75) | $12.18^{a,b}$ (0.79) | $20.37^{a,b}$ (9.11) | $49.51^{b}$ (23.59) | $28.50^{b}$ (21.7) | $45.63^{b}$ (33.65) | $20.57^{b}$ (7.26) | $29.39^{a,b}$ (12.66) | $25.97^{b}$ (16.28) | | ABI | $36.02^{b,c}$ (5.29) | $14.90^{c}$ (0.54) | $12.40^{b,c}$ (0.38) | $27.27^{b,c}$ (7.74) | $69.02^{b,c}$ (19.83) | $40.75^{b,c}$ (20.40) | $61.33^{b,c}$ (34.53) | $24.03^{b,c}$ (7.37) | $34.42^{b,c}$ (8.31) | $43.90^{b,c}$ (19.91) | | Total possible | 38.0 | 15.0 | 12.5 | No limit | 100.5 | 82.5 | (1) 51.5 (2) 142.5 (3) 138.5 | 39.0 | 45.5 | (1) 51.5 (2) 83.0 (3) 111.0 | ### Table 6.4. Correlations between calculated ability scores and statistically derived ability levels | Constrained skills | | | Unconstrained skills | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1. Orientation | 2. Following Commands | 6. Naming | 3. Word Fluency | 4. What Goes Together | 5. Digit Span | 7. Story Retelling-Immediate | 8. Yes/No/Maybe | 9. Picture Recall | 10. Story Retelling-Delayed | | .94 | .96 | .96 | $1.00^{a}$ | .75 | .93 | 1) .942) .983).90 | .97 | .94 | 1) .952) .923).85 | ### Table 6.8. Test-retest reliability | | Constrained skills | | | | Unconstrained skills | | | | | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | | 1. Orientation | 2. Following Commands | 6. Naming | 3. Word Fluency | 4. What Goes Together | 5. Digit Span | 7. Story Retelling—Immediate | 8. Yes/No/Maybe | 9. Picture Recall | 10. Story Retelling—Delayed | | | | | | | Mean | 35.6 | 36.6 | 12.0 | 12.5 | 27.2 | 29.6 | 33.4 | 31.7 | 58.8 | 56.3 | 25.2 | 26.1 | 31.9 | 35.6 | | | SD | 4.4 | 3.1 | 3.38 | 3.45 | 7.9 | 7.5 | 14.0 | 13.6 | 38.1 | 31.8 | 8.0 | 7.5 | 9.5 | 10.6 | | | Adjusted r | .88 | .99 | 1.00a | .80 | .81 | .84 | 1).832).793).84 | 1).832).793).84 | .80 | .75 | | | 1).802).883).77 | | | ## Conclusion This chapter guided examiners through the questions to ask to determine test performance.