Evidence that CATS is valid and reliable comes from:
- Extensive, ongoing and independent validation studies
- Advice from a panel of the nation's top educational measurement experts
- Other national test results posted on this site, which also reflect achivement trends.
We can all have confidence in CATS test results. Please examine the extensive validity studies conducted by the independent research firm Human Resources Research Organization (HumRRO).
HumRRO is a research organization founded in 1951. It has to its advanatage experience with more than 5,000 contracts comprising a wide range of research issues.
The Kentucky Department of Education (KDE) maintains an ongoing program of validity research, and it oversees the studies conducted by HumRRO. The goal of these studies is to gather evidence about the degree to which it is appropriate, meaningful and useful to draw conclusions from CATS test scores. Put simply, are the tests achieving their goals? Is CATS performing as planned? Is the test vendor – CTB/McGraw Hill – performing as expected? And, are the test results used in meaningful ways?
The validity of any statewide testing program, including CATS, is derived from the sum of its parts. That is, the steps in creating the program and putting it into operation are all subject to scrutiny. In a comprehensive testing system like CATS, that includes:
- Developing the content of the tests
- Consulting with stakeholders
- Communicating about the test to users
- Scoring
- Analyzing the data
- Reporting results and providing interpretive assistance
- Evaluating the various ways test results are used
In addition to maintaining an ongoing program of validity research, the Department of Education consults with a group of national testing experts who advise KDE on technical issues related to CATS (including issues of validity itself). This group, the National Technical Advisory Panel for Assessment and Accountability, or NTAPAA, meets quarterly. The panel members are national education testing experts, including some who participated in evaluations of Kentucky's previous testing system. NTAPAA advises the legislature and KBE concerning technical aspects of CATS. In a recent statement, NTAPAA expressed support for the validity of CATS, reflecting the general finding of many of the validity studies done on CATS. That is, CATS is being used appropriately to make the types of school accountability decisions for which it was designed.
Please visit the CATS Validity and Reliablility Studies site to examine the reports with your own eyes.