Ranking Online Education Programs

Judith McDanielBy Judith McDaniel
Editor, Web-based Course Design

U.S. News & World Report announced last week that it was expanding its program of college ranking with a new category for online universities and programs. The announcement from editor Brian Kelly sounded upbeat, nonpartisan, and helpful. “Later this year, U.S.News & World Report will be publishing an expanded directory of online education programs with more detailed information including rankings and other searchable data.”  The goal, says Kelly, is to help students make more “informed choices” about an online program. The survey questionnaires will be sent out next week (mid July) and will ask schools to provide information. The questions in the survey, Kelly promised, “are based on academic and industry literature reviews, as well as consultation with numerous heads of online degree programs in multiple disciplines.”

My own experience with the U.S.News rankings is limited and specific. As my first year of law school drew to a close, I realized that I wanted to transfer to another school — for a variety of reasons, both personal and academic. The school I was attending was ranked in the top 25 law schools in the country. The school that attracted me, because of its location and its focus on international law, ranked more than 50 places lower. And yet when I told the dean at the school I was currently attending that I was thinking about transferring and asked her about a reference, she was impressed that I had gotten into that school.

When I told her that my new school ranked in the lower 50, not the top 25, she was astonished. “They have such an outstanding reputation,” she said. “Those ratings…” and she shrugged. Once I was established at the new school, I asked that dean about the law school ratings. “Well, there are a lot of criteria that we don’t do well in,” he said. “One is alumni donations, since nearly half of our students are international and go home when they graduate. Donating to an alma mater is not really on their agendas.” Left unspoken was the question about what alumni donations could possibly have to do with the quality of a law school.

As with so many things, we have to take the good with the bad. When it comes to ranking educational programs, transparency is good. We should be able to ask questions of the collective educational endeavor and be able to measure programs and universities according to a variety of criteria, whether they are brick and mortar or online.

And the bad? The U.S. News headline proclaims: “U.S. News Seeks to Fill 3 Gaps in Online Education Data.” In the text of the article, these gaps become “deficiencies” in online education. They are identified as:

  • there is no national way to assess the quality of online education offerings,
  • there is no standard definition of what constitutes an “online” degree or program, and
  • there is no comprehensive listing of online programs.

This is all true, so where is the “bad”?

No mention is made of the several educational organizations that have been working toward collective definitions of what defines quality in online programs for several years. Nor is there any reference to the hundreds of journals and blogs that have also been holding community conversations to create these standards. If the U.S.News effort was primarily to bring together the work that is being done in the field by educators and educational administrators, then I believe we might have a higher level of confidence in the results.

For example, in 2009, the Western Interstate Commission for Higher Education (WICHE) and The Campus Computing Project partnered to develop and conduct the first “Managing Online Education” survey. This initial report covered topics important to understanding the role of online education in higher education —

  • Factors that impede institutional efforts to expand online education
  • Reorganizing the management of online education
  • Faculty training as a major investment for online programs

In addition, WICHE has collected data on ADA compliance in online programs, the level of technical support for students in online programs and much more.

The Sloan Consortium — “a consortium of individuals, institutions, and organizations committed to quality online education” (Sloan-C) — has also conducted and published a number of surveys and reports about online education. The Sloan-C annual reports have become the standard for credible academic data and information about online education in the United States.

A number of conversations are taking place nationally about the U.S.News ranking effort. On the Linked-In group for Sloan-C, opinions are mixed. As far as I can tell, few participating in the conversation have ever looked at the rankings except as a parent trying to figure out where a child should apply. One recent comment hoped that U.S. News “will talk about things like time-to-degree, student retention, resources available to support online students.” The conversation on Twitter is, of necessity, even more limited.

But members of the WICHE group are asking questions among themselves and on other websites. “I’m curious what delivery methods U.S. News will consider to be ‘online,” posted one educator. “Does this include both synchronous (such as live video conferencing, Adobe Connect) and asynchronous (such as archived video courses, self-paced learning modules)?” These and other criteria need to be established, but if they have been, U.S.News has not published that information.” Others mentioned organizations that are collecting the information that U.S.News calls “deficient.” Methods of data collection, metric standardization — all of this will need to be considered before any “rankings” on online courses will be useful to consumers.

So will these new rankings be part of the good or part of the bad? There are some rather big “ifs” to hurdle.

U.S. News’s efforts at analyzing and ranking new online programs will be good if:

  • the data collected enhances knowledge about online approaches to higher education;
  • gathering information into one searchable site provides comparative data on the effectiveness of online academic programs;
  • the publicity assists the public to better understand online learning and the opportunities it offers;
  • and then (perhaps) this collective effort will expand the market for adult students.

If the new rankings are simply perceived as another way for some institutions to enhance their revenue, then this effort will not benefit a public that needs and deserves this information.

8 Responses

  1. Judith,

    Thanks for sharing your thoughts and specific experience about USNWR’s proposed rankings of online education programs.

    My own opinion of this venture is apparently rather more pessimistic, mainly because I have a low opinion of rankings systems in general. Essentially, I think this proposed venture boils down to the following:

    – USNWR is doing this mainly for profit, not for service. They got out of the print news business last year and have arguably stopped being a news organization altogether. They’re now defined primarily by their rankings empire and the revenue it generates.
    – Their track record for rankings is dubious at best.
    – Rankings systems themselves are of highly questionable value.
    – This venture will happen anyway, and people will pay attention to it because it’s USNWR and because we’re a ranking-crazed society.
    – Participants will game the new rankings system, and it will stabilize and become self-legitimzing as has happened with the current ones.
    – It would be better if USNWR made ranking systems with transparent criteria instead of proprietary ones, and which consumers could customize for their own purposes, but I’m not holding my breath that either of these will happen.
    – So I see little good and much potential for mischief in this venture.

    For more details on my thoughts on how USNWR’s planned expansion of its ratings empire into online education programs, see http://senerknowledge.com/blogs/thoughts-usnwrs-plans-rank-online-education-programs-0

  2. One clarification about my previous post: USNWR makes their criteria explicit, but the “scientific” formula for applying those criteria (e.g., weights, derivation of values) remains proprietary and thus not transparent. I don’t expect this to change… — John

    • Thanks, John, I think I am more of a pessimist than an optimist about this. If you look at the “good” list–I do say that is a very big set of “ifs”. I do rather wish USNews would/could collect the data in one place and publish it for consumers to use rather than insisting on “ranking” performance. Rank depends on what you as a consumer wants or needs, not on what USNews decides is important.

  3. Very good article Judith! Overall I support the efforts of U.S. News & World Report; simply because it offers another resource. However just like John said, rankings are dubious, and what do they really tell you at the end of the day? Reading both this and John’s posts have added some valuable perspective and has helped to highlight the cons of this situation. But I am still looking forward to this effort; let’s see what the results are, and then revisit this discussion then!

  4. Great Judith
    I thought you did not like US News and World.
    It seems they are not bad.

    We need these info always.
    thanks any way

    • Well, let’s just say they have the potential to be good. As I said, there are a few conditions attached to that!

  5. Thank you for this Judith.

    I have to agree with what you say.

    I can add a few things that may help people understand what it means that US News is no longer an independent news agency and why this is important to those that look to them for “trustworthy” information.

    On June 26 USNews “online education” published an article recommending online masters from Almeda University — Almeda is the largest diploma mill in the USA — accredited by a completely bogus accreditation agency.

    This happened because the articles they “publish” in the “online education” section aren’t research based. They are free press releases put out by schools (like Almeda). US News data scrapes and re-publishes these releases with minor alterations because this is the fastest way to create cheap online content on keyword bait words like “online masters.”

    Publishing other people’s press releases without independent verification as to source or newsworthiness is bad practice (if one is indeed a reliable “news” agency as opposed to a direct marketing agency that needs fast cheap link bait.)

    This is not your father’s US News; not even my high school US News.

    US News stopped publishing news in December last year and switched to an online lead marketing business model with a focus on expanding their “rankings” and using these rankings to gather and sell student leads. They are paid for every lead they get and send to a school. (Red flag again if they are indeed a trustworthy source serving consumers instead of the colleges)

    The new US News now ranks America’s Best Diets …and then sells these leads or the names of people who view these rankings and want to lose weight to the diet industry (another very lucrative direct marketing niche on the Net)

    Yes, consumers need more information about quality, but that information needs to be high quality, not more of the same old thing that is coming out of online lead generation mills in toxic quantity.

    Quality is difficult to assess. The leaders in this area are, of course, SLOAN and WICHE. WICHE is collecting data and research on “outcomes” as indications of quality — employment, student engagement — whereas US News is looking still at old school “input” like what’s your SAT score.

    We at Get Educated look at 3 things: 1) affordability; 2) student satisfaction; 3) public and employer perception or reputation (which highly affects the real world value of a degree even if INPUTS are identical).

    Yes, yes, to making information more transparent about online degrees, but for heaven’s sake let’s make sure that info is from sources that understand the demographic and know the quality metrics for online learning.

    Vicky Phillips

    • Thanks for some more specifics, Vicky. I had read about the Almeda University snafu in the Chrnonicle and Inside Higher Ed, but had not realized that USNews was no longer a news magazine. Imagine, it doesn’t matter whether you sell education or diets.

      I had not seen your website before I wrote the article, but it is one I’ll add to my list to recommend.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: