03Hotz_CH06(P2)

Evidence-Based Practice (Technical Analyses)

Examiners need to know that the interpretations they make are supported by data-based evidence. In this chapter, we review the types of questions examiners should ask about the nature of any test, which will inform their conclusions about test performance. Therefore, this section covers validity and reliability information as well as information concerning analysis of test bias. We continue to use the concept of constrained and unconstrained skills (cf. Paris, 2005) to interpret results. To review, constrained subtests tap skills that develop relatively early in development (e.g., knowing one’s name, naming common items). Unconstrained subtests tap skills that tend to show continued refinement over a wide range of ages. The PTBI contains both types of subtests, and the statistical results are presented separately for each type. A question and answer framework is used to describe the evidence base related to various clinical questions about the nature of the PTBI.

EVIDENCE FOR CONSTRUCT-IDENTIFICATION VALIDITY

Question 1: How do I interpret the criterion categories?

The categories of performance for each age are based on the performance of the group of children without brain injuries selected to represent the general population. The percentages of these children whose scores fell into each of the four criterion categories is provided in Table 6.1. As this table shows, most scores fell into the High performance category. The differences in the percentages of children classified into the different performance categories reflects naturally occurring differences in the range of performance that children in the general population show in these different skill areas.

Technical Notes

Scores from 103 children (57 girls, 46 boys) were classified against the criterion categories for this source of evidence. They ranged in age from 6 to 16 years, with an average age of 9 years, 11 months. No child had been diagnosed with a brain injury (TBI or ABI). Note that children were not screened for other conditions that occur in the general population that could affect performance (e.g., learning disability). Therefore, this group is intended to reflect the non–brain-injured general population. Sixty-nine of the children were white, twenty-three were African American, four were Hispanic, two were reported as "other," and one was Asian. Race was not reported for four children.

Table 6.1

As Table 6.2 shows, the typically developing children received higher ability scores on all subtests compared with the ABI group. Children with TBI scored significantly lower than their peers without brain injuries on all seven subtests that assess unconstrained skills and two of the constrained skills subtests. The ability scores on the constrained skills subtests were close to ceiling for all three groups; therefore, these subtests are most useful clinically for children with severe impairments. It is also the case that the children with ABI differed from their peers with TBI on four of the subtests, indicating that the pattern of deficits is.

Technical Notes

Ninety-nine children’s scores were compared for this source of evidence. Thirty-three children were included in each of the three groups (TBI, ABI, typical). These children were matched by age (mean age of 10 years, 7 months; range 6–16 years) and gender (14 males, 19 females per group). A variety of racial/ethnic groups were included (TBI: 26 white, 7 minority; ABI: 17 white, 16 minority; Typical: 22 white, 11 minority). The majority of the children in each group were right-handed (TBI: 27 right-handed; ABI: 27 right-handed, typical: 25 right-handed). Ability scores were tested using t tests with statistical significance at p.05.

Table 6.2. Raw score means (standard deviations)

... (more tables and content summarized)...

Conclusion

This chapter guided examiners through the questions to ask to determine test performance. The evidence base for the PTBI was presented. Chapter 7 provides a case study demonstrating a test and retest of the PTBI.