Review of ‘At-Risk’: A Simulation Training Program for College Staff

heeter80By Carrie Heeter
Editor, Games Development

I vividly remember the day I received email from a graduate student who had gone missing from my online class, announcing that he had “just gotten back from the loony bin.” He wrote that he had checked himself in to a mental hospital and was now back and ready to start making up late assignments (with one week left in the semester). Over the years as professors each of us comes to realize our students are enrolled in classes other than just the ones we are teaching, and beyond that they have real lives, jobs, and families. Our official job is to teach well, to inspire, and to grade fairly while juggling our own impossible to meet demands of work and life. Unofficially, the unfolding joys and concerns experienced by everyone’s whole self may enrich or undermine teaching and learning.

At-Risk is a simulation training program designed to addresses one specific, potentially lifesaving dimension of this complex milieu.

At-Risk was created by Kognito, in partnership with the Mental Health Association of New York City (MHA-NYC). MHA-NYC programs help raise awareness about mental health problems and encourage people to seek treatment. The At-Risk training simulation teaches college faculty to identify mental health problems among their students and to refer mentally distressed students to the college counseling office for assistance.

poster with 3 small people in front of 1 taller person and the words: at-risk - identify students in mental distress - refer them to the campus counselling center

In the simulated 20 person class, 6 students have been flagged as potentially experiencing mental distress. As the instructor, your goal is to talk with each of those students and, if appropriate, refer them to the counseling center. You can review each student’s grades, behavior in the class, and appearance. You are told at the beginning that three of the six are at-risk, but you are not told which three. The training simulation lasts approximately 45 minutes. It is 2D web based and includes many lengthy narrated explanations before and after the interactivity.

At-Risk uses “conversation menus” organized by category to offer choices of what to say next. The animated student responds, choices of what the instructor says next are presented, and the simulation offers encouragement or criticism about the conversation choices.

I played through the free online demo of one of the six students. Wendy’s problems were exaggerated and extreme. She is a 4.0 student who is so nervous she comes in to talk about every assignment. Heart palpitations caused her to go to the health clinic, causing her to skip the class presentation. As I played through the simulation, I argued with myself about whether it is reasonable for professors to call a meeting with 4.0 students who are nervous about speaking in class, even if the student is very nervous. I made a note to myself to check whether my university counseling center still exists, after the latest round of budget cuts, and what services they offer.

I also found that experiencing the simulated conversation was helpful and informative, even though I was trying to figure out what the simulation expected me to choose. It was useful to choose and hear spoken exactly how to bring up the counseling center. If sending students there has a chance of helping them cope better with life and with school, that’s something I would be willing to do. And now I have a better sense of how it’s done. The simulation was more useful in convincing me of the importance of identifying mental health problems and in showing me how to refer people than reading a brochure would have been.

clip-art-like image of a class where students at risk are marked by a white triangle above their heads

I also naively expect socially useful serious games to be free. At-Risk is definitely not free. Licensing fees are way beyond what any individual faculty member would consider paying. I am not familiar with how universities prioritize nontrivial expenses like this for 45 minutes of online simulation, especially in times of deep budget cuts. The online free demo for one of the six students was informative and useful. Playing the other five conversations would not add five times more value — just playing one was enough to get the most important message: referring students is not hard to do and could help them a lot.

Serious game design needs to be accompanied by research to determine whether the serious goals have been met. Kognito has taken this important step. They are studying their own product and using the findings in marketing. And yet, product specific efficacy studies are not an expected domain for academic scientific research. The research findings offer a window onto desired and achieved impacts of the At-Risk simulation. I contacted the company for details about the sample size that I didn’t see online. They responded that 42 colleges and universities (who were not paying customers) were invited to use a trial subscription. The first 35 individuals who completed the training at each institution were automatically invited to complete an anonymous online survey. Respondents who were full time practicing psychologists were excluded from study results which, instead, focus on faculty and staff reactions. A total of 375 respondents are represented in the results. No response rate percentage is known.

Key findings from the Kognito.com online research report:

  • Over 80% reported that At-Risk increased their awareness that identifying and referring students is part of their job role and that At-Risk made them more likely to engage in identifying and referring at-risk students.
  • 87% of respondents indicated they were better prepared to identify, refer, and approach at-risk students, and 82% felt better prepared to help a suicidal student.
  • 99% of respondents said the simulated conversations were realistic representations of conversations they were likely to have with at-risk students.

If I had been a respondent, I would have answered the way the majority of respondents did, based only upon playing the demo.

For more information about the simulation see http://www.kognito.com/atrisk/

If We Don’t, Someone Else Will

Harry KellerBy Harry Keller
Editor, Science Education

The United States is falling behind. For many, that’s not a surprising statement, but others will find it hard to believe.

We see statistics summarizing our declining science education, our lack of world-class Internet infrastructure, and many more. What we haven’t seen much of are examples of us falling behind in innovation. Yet, that’s exactly what’s happening. It’s been predicted for quite a long time now by some more pessimistic prognosticators but not demonstrated.

My field is science, and my current work centers on technology to support science education. It’s no surprise that my example comes from that area. For years, I’ve watched as company after company (and even individuals) make science simulations and attempt to sell them as science “labs.” Of course, they’re not truly labs, but that’s beside the point.

sebit2These companies all have produced essentially the same product. It’s a Flash-based animation system wherein students make some choices of parameters and see the result. These animations are two-dimensional and have little support added online for learning and essentially none for tracking. I don’t have to list them here because a quick Internet search for “virtual lab” will give you lots of examples.

So, from where does the first visually appealing, three-dimensional simulation system sold in the United States come from? Turkey! You may have thought of Turkey as some backwater country with lots of small, dusty villages. Not so. It’s a vibrant, secular society that’s put a premium on education in general and science education in particular. Furthermore, they’ve committed to using online education to reach their goal of an educated society. Sebit Technologies has been created by Turkey’s telecommunications leader, Türk Telekom. With all of the money at their disposal, they have made some real waves in online education.

turk_telekomYou can bet that Turkey will not be the last place we see new competition for United States education dollars. Unless our country gets moving with true innovation, we’ll watch as more and more foreign-created innovations take over our schools (and other business markets).

As I’ve suggested before, teaching itself could eventually be handled offshore. Your children or grandchildren may be learning from teachers in India or China. That might sound quite cosmopolitan but will have a huge impact on one of our most stable professions — teaching.

We shouldn’t give up without a fight. It’s time for our government to foster real education innovation. I don’t mean with tax breaks or allowing free market forces to work. We must have serious investment by government in technology infrastructure for education. We may even have to put tariffs on these sort of imports for a while in order to get our companies back into the game. The alternative is just to sit back and let the rest of the world take over education in the United States.

Old School Thinking Blocks Quality Online Science Classes

adsit80By John Adsit
Staff Writer
6 November 2008

Online education is bringing quality education to many thousands of K-12 students who would otherwise not be able to access it, but in doing so it is forcing us to rethink some of our traditional ways. Unfortunately, we are too often clinging to old rules and old ideas that stand in the way of this progress.

One example is in science education. In 2005, the National Research Council published America’s Lab Report, a scathing indictment of how science classes in regular schools include labs in their instruction. [The entire report is available online at no cost. Click here for the table of contents.] It identified seven goals for a lab program:

  1. Enhancing mastery of subject matter
  2. Developing scientific reasoning
  3. Understanding the complexity and ambiguity of empirical work
  4. Developing practical skills
  5. Understanding the nature of science
  6. Cultivating interest in science and interest in learning science
  7. Developing teamwork skills

After examining hundreds of studies, the NRC concluded that what it called “typical” lab programs did a “poor” job of attaining any of those goals.

adsit012The problem is not in the labs themselves but rather in how they are included in the instructional plan—or rather how they are not included in the instructional plan. The NRC identified a different approach, which it called an “integrated” lab program, a design that makes science labs a critical part of the instructional process in ways that are fully in keeping with modern concepts of best practice in education. The report is pessimistic about the chances for this happening, though, for it notes that these methods are not a part of typical science teacher training.

After reading the report, several educators active in the North American Council for Online Learning (NACOL) theorized that by using this report as a guide, online education schools could create science courses that far surpass the quality of what students in “typical” schools are experiencing now, and they convened a committee under the direction of Dr. Kemi Jona of Northwestern University. Eventually, Kemi and I coauthored the results of that committee’s work in the form of a white paper describing how online science courses could be designed, using a variety of inquiry experiences that can include high quality virtual labs, to follow those guidelines and produce an excellent total lab experience.

Unfortunately, the old rules are getting in the way.

For example, the University of California Office of the President (UCOP) has established the “a-g standards” that determine if a high school course is adsit021acceptable for admission to any of the UC schools. They require online students to make some sort of arrangements with a school to use their lab facilities under supervision. Students must leave their online setting, travel to a supervised lab, and follow precisely the procedures the NRC describes as “poor.” If an online program contains even a single virtual lab or simulation of any kind, it is not acceptable for admission. If an online class meets all their requirements and then decides to add a high quality simulation to the program, then it is no longer acceptable. If a student takes and passes a College Board approved Advanced Placement class that includes a single virtual lab, that course cannot be counted for college admission.

UCOP is only an example; it is not the only institution or state to have such a rule or law. If online education is to bring quality science education to students in remote areas, we must do what we can to help the die hard traditionalists who make the rules understand the new realities:

  • The traditional lab experiences students have in regular schools are not as valuable as is assumed, and in fact research says they are “poor” and ineffective.
  • Well-designed online classes have lab programs that are far superior to what students encounter in “typical” lab programs.
  • Archaic restrictions based on false assumptions are depriving thousands of students of the high quality online and computer-based educational resources that are not otherwise available to them.

Responding Article

Simulated Labs Are Anathema to Most Scientists by Harry Keller