Pressure Mounts for Efficacy Research in K-12 School Sales

by Denis Newman

There is a growing need for evidence of the effectiveness of products and services being sold to schools. In response to increased scrutiny by school administrators, publishers and developers are asking for advice about how they can best measure the effectiveness of the products they offer to schools and how they can make research they sponsor more useful as part of the process of selling to schools.  A new SIIA guidelines report provides just that. Here’s the backdrop.

Proof of Product Performance is an Education Marketing Imperative

It’s been almost a decade since NCLB (No Child Left Behind) legislation made its call for “scientifically-based research" (SBR) but the calls for research haven’t faded away.  Because funding available to schools has decreased, there is heightened importance of cost/ benefit trade-offs in spending. 
 
NCLB focused our attention on test scores. This metric is becoming more pervasive, e.g., getting tied to teacher evaluations and through linkages to dropout risk.  While NCLB fostered a compliance mentality—product specifications had to have a check mark next to SBR—the need to assure that funds are not wasted is now leading to a greater interest in research results.  Decision makers need to know how well products they are evaluating will work (or have been working) in their districts.

Improved K-12 Data Systems Drive Efficiency for Product Evaluations

Fortunately, assessment data is more reliable and easier to access than it has ever been.  The US Department of Education has set standards and poured hundreds of millions of dollars into state data systems to ensure that student achievement data is available to decision-makers. Districts continue to invest in data warehouses and data management systems. These improvements in achievement data access needed for product evaluations lower the cost to publishers and to their district customers, making it feasible to conduct multiple studies and even to answer questions from specific customers.

Updated Federal Policy Language Defines School Market Research

The reauthorization of the Elementary and Secondary Education Act (ESEA)—often referred to as NCLB—is also expected to continue the pressure to evaluate educational products. We’re still a long way from the draft reauthorization introduced in Congress becoming a law, but the initial indications are for continued requirement of product-effectiveness evidence. The language has changed somewhat. Look for the phrase “evidence-based.” Along with the term “scientifically-valid,” this new language is actually more sophisticated and potentially more effective than the old SBR neologism. 
 
Bob Slavin, one of the reviewers of the SIIA guidelines, says in his Education Week blog that “This is not the squishy ‘based on scientifically-based evidence’ of NCLB. This is the real McCoy.”  It is notable that the definition of “evidence-based” goes beyond setting rules for the design of research such as the SBR focus on a single dimension of randomization of “internal validity.” It now asks how generalizable the research is and its “external validity,”  i.e., does it have any relevance for decision makers.

New Policy Guidelines Promise Smoother Sales Process

One of the important goals of the SIIA guidelines for product effectiveness research is to improve the credibility of publisher-sponsored research.  It is important that educators see it as more than just “education market research” producing biased results.  In this era of reduced budgets, schools need to have tangible evidence of the value of the products they buy.  By following the new SIIA guidelines, publishers and developers will find it easier to achieve that credibility and remove barriers to their sales process.