i3 Funding Process Unfair to Small Businesses

keller80By Harry Keller
Editor, Science Education

The comment period for the $650 million Department of Education’s “Investing in Innovation Fund,” referred to as i3, has ended. An article in Education Week discusses the main thrusts of these comments. For the entire text of the proposed priorities, click here.

Some large urban school districts object to small rural districts being favored. Small rural districts have problems with devoting resources to writing such complex grant applications and with conducting the studies requested in the guidelines. A requirement for 20% matching funds from the private sector, including foundations, has also received criticism because of the very short time frame. Some districts complain of the “adequate yearly progress” (AYP) requirement.

My perspective is that of a small business president. For this purposes of comment, it only matters that I have been working on innovation in education for over ten years and have encountered just about every road block to having schools use my innovative services as you can imagine.

The i3 guidelines allow three different types of proposals: scale-up grants of up to $50 million, validation grants of up to $30 million, and development grants of up to $5 million. The last of these requires a two-stage application process and does not require the high level of studies with proven results that the other two do.

Here’s the description of the scale-up grants. (Emphasis added.)

Scale-up grants would provide funding to scale up practices, strategies, or programs for which there is strong evidence (as defined in this notice) that the proposed practice, strategy, or program will have a statistically significant effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, or increasing high school graduation rates, and that the effect of implementing the proposed practice, strategy, or program will be substantial and important.

Validation grants are described in the following way. (Emphasis added.)

Validation grants would provide funding to support practices, strategies, or programs that show promise, but for which there is currently only moderate evidence (as defined in this notice) that the proposed practice, strategy, or program will have a statistically significant effect on improving student achievement or student growth, closing achievement gaps, decreasing dropout rates, or increasing high school graduation rates, and that with further study, the effect of implementing the proposed practice, strategy, or program may prove to be substantial and important.

types of awards available under i3, explained in a table. Columns: Development, Validation, Scale-up; Rows: Estimated funding available, Evidence required, Scaling requiredClick the image for the PowerPoint presentation.

This is how development grants are explained.

Development grants would provide funding to support new, high-potential, and relatively untested practices, strategies, or programs whose efficacy should be systematically studied. An applicant would have to provide evidence that the proposed practice, strategy, or program, or one similar to it, has been attempted previously, albeit on a limited scale or in a limited setting, and yielded promising results that suggest that more formal and systematic study is warranted. An applicant must provide a rationale for the proposed practice, strategy, or program that is based on research findings or reasonable hypotheses, including related research or theories in education and other sectors.

Only school districts and nonprofit education businesses may apply. Entrepreneurs who provide tools are not eligible.

Note that the largest awards require “strong evidence.” Those districts that choose to submit “scale up” proposals must include innovations with this evidence. “Strong evidence means evidence from previous studies whose designs can support causal conclusions . . . and studies that in total include enough of the range of participants and settings to support scaling up to the State, regional, or national level . . . .”

It’s a very reasonable assumption that most of the new, innovative tools for education will come from small businesses. In the difficult education marketplace, having a new and better way to provide some aspect of education provides an edge over large existing businesses. The large education companies have an established way of doing business and usually will not seek change unless forced to do so by the market.

The i3 guidelines, however, are skewed toward large entities by their requirements for studies, which are quite expensive to carry out.

The i3 guidelines, however, are skewed toward large entities by their requirements for studies, which are quite expensive to carry out. My business has not been able to do a study and most of those I’ve looked at have the same problem. This basic problem seems to pervade many federal and state operations. The large businesses that can afford lobbyists, studies, extensive marketing, and other activities not accessible to their smaller kin get the bulk of federal largesse.

Besides, education studies often have flaws. I’ve seen two studies produce opposite conclusions on the part of the investigators. Generally, education studies compare a new method or device in classrooms with the status quo. Of course, the teachers and students know that they’re doing something differently and react to that fact as well as to the actual new method or device.

The “new math” was studied and found to be the great savior of our student mathematical literacy. What happened? When rolled out at scale, it just didn’t work, and a generation of students was hobbled in its mathematics learning by this idea. Suddenly, it was “back to basics” again.

The i3 study requirement is therefore doubly flawed. Studies do not produce reliable black-and-white results. Understanding their data requires very knowledgeable people and often they will conclude only that the new idea may help students. It’s much too easy to bias the study results in the direction that the investigator wishes.

The second flaw in the requirement is the institutional bias that such requirements have against our greatest innovators, small organizations and individuals. The greatest new idea in education could be out there right now seeking acceptance, crying in the wilderness and unheard by the districts, agencies, and foundations. You can be sure that a number of good ideas are struggling to be recognized.

The i3 program also appears to assume that innovation will come from within schools. But schools tend toward inertia. An entire system of school districts, state departments of education, and colleges of education has been built to keep things stable, to avoid change. Good ideas have originated within schools to be sure. However, this approach of the i3 program ignores our greatest resource, entrepreneurship. The program should reward schools that reach out to the entrepreneurial community to find new, exciting, and innovative ways to improve education.

We do not know yet what we’ll see in the final guidelines. However, none of the comment summaries in the Education Week article suggest a movement toward encouraging entrepreneurship. If we’re to make a real difference in education, we must engage all of our resources including the most powerful agent for change we have. While, as an entrepreneur myself, I am biased, I believe that the facts support my conclusions.

Let’s engage all of our national resources in this important effort.

Investing in Innovation Fund: Criteria May Be a Barrier to Some Innovators

Harry KellerBy Harry Keller
Editor, Science Education

Arne Duncan, the new Secretary of Education and much-praised previous superintendent of Chicago Public Schools, is applying $650 million of his ARRA (federal stimulus) money to a new initiative: Investing in Innovation Fund (i3). At first glance, this program is bold and should bring some much-needed innovation to education, at least in part through education technology and change.

As with any such program, the details will make all the difference. The rules for the i3 program are going out for comment and have already been released in preliminary form. Their thrust is commendable. The rules require that all proposals come from LEAs (essentially school districts), non-profits affiliated with LEAs, or consortia of schools.

Arne DuncanThe program lists four “absolute” priorities. Your proposal only has to meet one of these. In short, they are teacher quality, data use, standards and assessments, and low-performing schools. The program goes on to list four more “competitive” preferences: early childhood programs, college access, disabilities and limited English proficiency, and rural schools. Addressing competitive priorities will gain evaluation points.

The program will provide grants in three categories: scale-up grants, validation grants, and development grants. The size of the grants runs from high to low across these three as does the requirement for evidence for effectiveness of the proposal, which includes research on, significance of, and magnitude of the effect.

All those interested in innovation in education in the United States should be prepared to comment on this major initiative. I see a major weakness of the program as the dependence on experimental studies for deciding which proposals to fund. Perhaps, few alternatives present themselves. Still, I have read a number of studies that purported to prove opposite conclusions. I cannot imagine the equivalent of a double-blind study in education because the instructor and students know that they’re doing something different, and change affects performance.

Then, there’s the expectations. Expecting better results generally creates better results in these studies.

Here’s one definition of “strong evidence” from the draft of the program: “one large, well-designed and well-implemented randomized controlled, multisite trial that supports the effectiveness of the practice, strategy, or program.” This definition leads to the question of who can afford to conduct such a trial?  So, are only wealthy purveyors of education innovation eligible for this sort of grant?  Will the “usual suspects” garner all or nearly all of this federal largesse?

Looking at the criteria for the three grant types yields some interesting information.

Table of criteria for i3 grants

Even if you’re ready to scale to national level, you must have “strong evidence” before doing so. What is “national level” anyway? The draft program requests estimates of costs for reaching 100,000, 250,000, and 500,000 students for validation and development grants. It requests estimates for 100,000, 500,000, and 1,000,000 students for the scale-up grants. Does that mean that national scale is just double regional scale?

As a small business operator, I find myself in a difficult position in the education marketplace, and this program simply underlines my situation. I cannot afford to conduct large studies of my effectiveness. Yet, without such expensive studies, I have trouble attracting enough business or investment to conduct such studies. So, at this time, I have to rely on anecdotal evidence including quite a few enthusiastic testimonials.

For the sake of argument, let’s say that my business has a truly transformative innovation. I can certainly say that it’s new, unique, and exciting without running studies, but let’s assume that such studies would prove the case for my service. The publicity benefits of receiving validation from the Department of Education in the form of one of these grants to a school district for the use of my service would be huge. If my assumption for the sake of argument were true, then schools across the country would reap large benefits too. Yet, I see no clear path to such a result.

I believe that my situation is not unique. Others must have excellent innovations ready to be deployed more widely and are facing numerous obstacles in doing so. Can this government program be amended to provide the opportunity for such innovations to be recognized?  If so, how?