John Katzman, Hannes Geldenhuys, and Melora Sundt
Introduction
The move to blended and online learning spotlights learning design, but it’s hard to measure what is good. Schools committed to the quality of their campus-based education are making decisions that undermine that commitment. Further, the lack of collaboration drives costs up significantly and unnecessarily. But there are ways to address these issues.
Quality Matters Is a Baseline
We define quality as an engaging experience through which the learner meets/exceeds expected learning outcomes and finishes wanting to do more; it’s an experience that leaves the learner satisfied and, at its best, inspires future learning. Quality Matters1 (QM) offers a way to assess whether a course is likely to do this.
QM was created by Maryland Online in 2003 to enhance online course quality within its consortium of colleges and universities. The project quickly evolved beyond Maryland, gaining national and international recognition for its robust rubric and standards focused on course design, learner interaction, and accessibility. As online education has expanded, the influence of QM has grown, becoming integral to institutions seeking accreditation or meeting regulatory standards. Offering professional development and certification, QM continues to adapt its criteria to reflect emerging research and technological advances and is seen as a research-based framework to promote the quality of online and blended education.
Can We Do Better?
While robust, QM standards emphasize technical requirements, which should be achievable with minimal expense—perhaps a $5,000 per course investment in learning design. But it’s clear that schools can do better in delivering blended and online learning.
At Noodle, we have extended Quality Matters, using available research, into two metrics:
- Our EPIIIC2 scorecard measures the quality of a course.
- Our RADAR3 framework, evolved from Merrill’s Principles4, shapes the arc of learning cohesion and quality across a program holistically.
Active Learning Adds Cost
Powerpoint, text, and webcam video are cost-effective, and they served their purpose in rapidly moving to technology-enabled learning during the COVID pandemic. But they don’t present a sustainable approach for high-quality learning, and most do not meet the baseline measures of Quality Matters.
Using studio-quality video adds polish but gets expensive quickly without fundamentally adding to the effectiveness of teaching. The same applies to other high-production approaches such as virtual reality and animation.
Interactive exercises add significant design and building costs, but they can make the learning experience richer in a number of ways while significantly reducing the load on faculty to provide more sophisticated feedback cycles.
The intent is to design according to the type of knowledge being taught (Anderson & Krathwohl5) and with an understanding of how people learn—how different types of information and skills get anchored in long-term memory. Interactivity isn’t just a “nice to have”—when well-placed, it has been demonstrated to increase knowledge retention and mastery of skills, more so than simply listening to a lecture.
How Much Better?
By applying our EPIIIC scorecard, schools can evaluate the efficacy of how learning design impacts student outcomes, ultimately driving continuous improvement in the delivery of high-quality education.
Engaging An engaging course is relevant and motivating to the student; it reaches out and grabs them. Our learners interact with the course on behavioral, emotional, and cognitive levels.
Personalized A personalized course treats learners as individuals with different backgrounds, abilities, and prior knowledge. Options and supports are provided so that all learners are optimally challenged. Learning experiences are built on the idea that “learning occurs when someone wants to learn,” not “when someone wants to teach” (Roger Schank).7
Interactive An interactive course is one in which students are asked to contribute knowledge and manipulate information throughout. Students are provided ample modeling, given the opportunity to demonstrate learning through meaningful tasks, and provided timely, concrete feedback on their work.
Intuitive Managing cognitive load throughout a course enhances learning. An intuitive course design allows all students to focus on content and learning rather than on navigating pages and looking for information. Content is efficiently and effectively tied together throughout the course.
Inclusive An inclusive course presents a diversity of perspectives, authors, and learning experiences and/or assignments, giving students an opportunity to connect their own experiences to the content.
Collaborative Creating a knowledge exchange is imperative for learning. A collaborative course is one in which students are provided ample opportunities to learn from their peers.
Courseware
Some universities take a shortcut to course design and simply buy a full curriculum for Gen Ed and other courses. But while courseware is an inexpensive path to materials of reasonable quality, replacing a professor’s narrative arc with that of a textbook company runs the risk of commoditizing the learning experience.
Synchronous and Asynchronous Learning
Cost is predominantly linked to the asynchronous experience. If we use the Carnegie standard as a rough guide (# “contact hours” needed are based on credit value of the course and the length of the term), the less time students spend in real time with the instructor, the more asynchronous time will be needed, driving up costs.
In any course, online or on-ground, optimal learning design involves opportunities for students to be introduced to foundational concepts and skills, practice applying these with frequent feedback, and then apply these to novel or more complex situations, cases, and problems. The ability to have these experiences in collaboration with other learners and under the guidance of a “more knowledgeable other”—an instructor—increases the “uptake” of the learning (Silber & Foshay, 2010).6
The asynchronous modality provides an environment where faculty, with the help of instructional designers, can create learning opportunities that engage and reinforce necessary knowledge and skills—and, to some extent, provide feedback—in ways that allow students to repeat experiences as often as necessary to anchor those skills. Live, synchronous sessions allow for application of knowledge and skills, as well as community building that creates a learning network to support students when they return to asynchronous experiences. The two modalities are best seen as complementary rather than oppositional.
It’s not a question of which one, but rather what ratio of each. The research evidence is growing clearer that a combination of asynchronous and synchronous time produces better learning outcomes than either modality alone. The most cited evidence is the 2010 meta-analysis by the US Department of Justice. In our experience, the actual ratio between the two modalities varies by academic discipline, perhaps reflecting traditions from on-ground programs and preferences of faculty. For example, many of the education programs we have assisted use a 2.5:1.5 ratio of asynchronous versus synchronous time for a typical 3-unit course. They design two-and-a-half hours’ worth of asynchronous contact and one-and-a-half hours of class time a week in a 15-week semester for a 3-unit course. In contrast, most of the business schools we’ve worked with lean more heavily on asynchronous time and typically do about one-and-a-half hours of class time every other week and about five hours of asynchronous time over two weeks for a 3-unit course.
The data on asynchronous-only courses suggest that few students learn well that way. Good learning design should incorporate synchronous engagement strategically.
AI’s Impact
The predominant view on AI is that it will make things cheaper. The more significant opportunity is getting rid of the concept of a single set of asynchronous materials; in effect, we can create a textbook for each student.
Collaboration
As noted earlier, collaboration among students is not just a good idea, it is a key facilitator of learning. Collaboration across departments or even among institutions is also a good idea that can significantly reduce the cost of creating more effective online learning.
Creating effective, engaging media is expensive. Yet when that media focuses on the most challenging concepts for learners—the areas learners frequently get wrong—and allows them to revisit those concepts and practice asynchronously, it can transform failure into success. The authoring schools can offset that cost by “leasing” the media to other institutions. The authoring institution can also control access; the media doesn’t have to be published for everyone—if they share the cost with three other non-competitive schools, they cut 75% of their cost.
So What Do High-Quality Online Education Materials Cost?
Learning Designers cost $50-100/hour, and a 3-credit course might take 700-1400 hours of work, excluding media or faculty time.
To note, the bulk of the cost for asynchronous experiences is media creation and faculty efforts. Faculty time, often overlooked, can easily come to 200+ hours for a 3-credit course.
Therefore, for stand-alone work, a good, solid job would come in at $50-100k per course, plus another $10-20k per year to keep the course up to date. This figure can be reduced to $25-40k by sharing assets among companies and schools. An initial investment which, on condition the course is maintained and updated regularly, should benefit both students and schools at scale for years to come.
Summary
Quality does matter; even if it’s hard to measure, it impacts learning and graduation rates. From hitting the basics of QM to a course heavy in interactive learning objects, outcomes will tie more to good design than to polished video or VR. Through thoughtful collaboration, we can achieve learning goals at reasonable cost.
Let’s TalkFootnotes
1 Quality Matters. Retrieved from https://www.qualitymatters.org/
2 EPIIIC Scorecard. [PowerPoint slides].
3 RADAR Framework. [PDF document].
4 Merrill’s Principles of Instruction. Retrieved from https://elearningindustry.com/merrills-principles-instruction-definitive-guide
5 Blooms Taxonomy Revised. [PDF document]. Retrieved from https://quincycollege.edu/wp-content/uploads/Anderson-and-Krathwohl_Revised-Blooms-Taxonomy.pdf
6 Silber, K. H., & Foshay, W. R. (2010). Handbook of Improving Performance in the Workplace, Volume Two: Selecting and Implementing Performance Interventions. John Wiley & Sons.
7 Schank, R. C. (1995). Tell Me a Story: Narrative and Intelligence. Northwestern University Press.
