Date of Award

1-29-2015

Document Type

Dissertation

Degree Name

Doctor of Education (Ded)

Department

Educational and School Psychology

First Advisor

Joseph F. Kovaleski, D.Ed.

Second Advisor

Mark J. Staszkiewicz, Ed.D.

Third Advisor

Timothy J. Runge, Ph.D.

Fourth Advisor

Joseph E. Betts, Ph.D.

Fifth Advisor

Tammy J. Croce, Ed.D.

Abstract

This study examined the predictive validity and diagnostic accuracy of STAR Reading, a computer-adaptive benchmark assessment tool, with the Pennsylvania System of School Assessment (PSSA), the state summative test. Student data for 320 students in grades 7 and 8 over two years of assessment were examined. Indices of correlation and predictions were analyzed. Multiple and hierarchical regression analyses examined the relationships of fall and winter administrations of STAR Reading for patterns of relationships of proximity of administration to the summative test. Logistic regression and ROC Curve analyses provided indices of classification accuracy for prediction of proficient performance and local cut scores for fall and winter administrations of STAR Reading. These local cut scores were compared to the published national benchmark cut points for STAR Reading. An attempt was made to cross-validate the prediction of proficient performance from the first year of the study to the second year. Results showed moderate to strong correlations among STAR Reading and PSSA Reading scores. Despite significant prediction indices for combined fall and winter STAR Reading scores in both grades, predictive values were weaker in grade 8 data. No clear evidence of a stronger relationship of prediction as a function of proximity to the completion of the state test was found, but hierarchical regression showed that the winter score added predictive value in three of four analyses conducted. Logistic regression indicated good classification accuracy and prediction of proficient performance on PSSA Reading for grade 7 students, but produced inconsistent statistical results for grade 8. ROC curve analyses showed adequate area under the curve (AUC) indices for three of four analyses. Inconsistencies were demonstrated frequently in grade 8 data; the attempt at cross-validation of probability of prediction was not able to be completed for grade 8 due to a slope intercept of .000 for one of the groups of eighth grade data. While evidence was provided that the overall models of prediction of STAR Reading to PSSA Reading was statistically significant, the lower effect size for grade 8 provided limitations to the mathematical utility of prediction to PSSA Reading scores.

Share

COinS