Comparing Scores
Comparing Group Test Results
One of the reasons that the same test is used year to year, and is used with all schools, is to allow
for comparisons over time and among schools. The most defensible comparison score is the "PAC 50"--
the percent of students in the group scoring at or above the 50th percentile. There are a variety of
comparisons possible, each with their own set of cautions.
Same Year, Within School Comparisons
A school may wish to compare the performance of students at different
grade levels. By looking at the percent of students scoring above the 50th
percentile at each grade level, in the same subject--for example,
reading--similarities and differences in the performance of students at
different grade levels can be noted. When making such comparisons it is
important to remember that the number of students in each group affects the
confidence in the inferences that one can make about such comparisons.
Same School, Different Years Comparisons
There are two types of comparisons that can be made in the same school
with two or more years of data. A cohort comparison looks at the
performance of a group of students over time. For example, hypothetically
in 1998 54% of the third grade students scored at or above the 50th
percentile in reading. In 1999, 58% of the fourth graders scored at or
above the 50th percentile in reading. That cohort's reading scores improved
four percentage points. In a cross sectional comparison, last year's third
grade results are compared to this year's third grade results. In the cohort
comparison it is important to remember that even if the same number of
students are represented in the '99 fourth grade as were in the '98 third
grade, the groups' composition may be quite different if student mobility
(transiency) is high.
Scaled Score Comparisons (Cohort)
While scaled scores cannot be compared between different tests or subject
areas, scaled scores are useful to compare performance over time on the
same test. For example, if the second grade in a school had 52% scoring
above the 50th percentile, and the same group of students as third graders
also had 52% scoring above the 50th percentile, the only way to determine
if the students actually demonstrated growth over the year is a comparison
of the average scaled score from one year to the next. In fact, if a group
maintains the same position relative to the norm group, the scaled score
would increase, because the norm group also increases its scaled score
from year to year. Even if the percent of students scoring above the 50th
percentile decreased slightly--say from 52% to 51%, it would be incorrect
to infer that the students had not improved. It is necessary to look at the
mean scaled scores in order to determine whether students have
progressed.
