Independent study: Florida Standards Assessment is Valid

by Allison Nielsen / Sunshine State News - September 1, 2015

The Florida Department of Education has come back with the results of the summer-long independent validity test of the Florida Standards Assessment, and the final verdict: the test is verified valid. “I am pleased that this third-party study confirmed our confidence in Florida’s statewide assessments,” Commissioner of Education Pam Stewart announced Tuesday morning.

Earlier this year, the department contracted with Utah-based Alpine Testing, and Washington D.C-based edCount to complete the validity test. The two companies spent the summer analyzing the test, studying everything from its content to scoring methods. The study itself was a result of newly-passed legislation in which state lawmakers called for a full review of the FSA. Many legislators expressed concerns after Florida’s students saw many difficulties on the computerized aspect of the test.

The department said some of these glitches were the result of a server attack, but it has not completed the investigation of these claims.

According to Alpine and edCount’s final report, the key components of the FSA all follow “best practice” standards. These included test items, field testing and psychometric validity questions, among other aspects of the test which made the grade compared to other standardized tests in the industry.

The companies spent time across Florida talking to teachers and department officials about the FSA, questioning them about the testing process and especially about the technical errors, which left many students unable to complete the test or login at all.

Now that the validity study has been completed, the department can move forward with teacher evaluations and school grades, both of which had been held up until the companies reached their final verdict.

Though Alpine and edCount essentially gave a thumbs-up to the test, they did make several recommendations to the department moving forward. One of the biggest was on the topic of test administration.

“The spring 2015 FSA administration was problematic. Problems were encountered on just about every aspect of the administration, from the initial training and preparation to the delivery of the tests themselves,” read the report.

The companies said the department should work on bettering responses to any potential problems -- like tech glitches -- that might arise when students are taking the test. They also said the department should be more proactive and review better ways to communicate with administrators just in case any errors pop up. Teachers, students and parents also expressed skepticism over the test's value after the tech errors. To remedy that, the companies recommended the department create a communication and training program to help alleviate concerns about this year's FSA administration.

As of the report's release, the department hadn't yet determined what it would be doing to remedy public concerns.

"We are still in the stages of digesting whats in that report," said Stewart. "We will carefully consider those recommendations & where we believe it is important, and can, we will implement those recommendations."

Other recommendations included getting rid of test items from the FSA’s spring administration because some content was aimed at Utah standards. The companies recommended they be replaced with newer items more closely aligned with the Florida standards.

“While alignment to Florida standards was confirmed for the majority of items reviewed via the item review study, many were not confirmed, usually because these items focused on slightly different content within the same anchor standards,” they wrote. “It would be more appropriate to phase out the items originally developed for use in Utah and replace them with items written to specifically target the Florida standards.”

The companies also recommended the department conduct an alignment study for testing items to make sure they match up to the Florida standards.

More and more students are taking statewide standardized tests each year. During the spring administration of the FSA, 3.2 million tests were completed successfully. Computer-based assessments saw an uptick in 2015, with around 2.4 million tests completed this year, an increase from 1.7 million completed in 2014.

Stewart did not have a specific timeline on when school grades would be released, but indicated it would happen soon.

Read the report here.