CASE STUDY

Forum Classroom

Minerva was established in 2013 to provide an extraordinary liberal arts and sciences education to the world’s top students using techniques informed from the science of learning. A fully accredited university, we graduated our first class in the summer of 2019. Our mission has attracted a lot of attention, sometimes being called "The Future of College" and "the most interesting and important highter education program in the world."

As the fifth person brought on to the founding team, I was tasked with imagining how we could create innovative software to accelerate the lives of the world’s brightest young students.

The Challenge

The Minerva education experience was built around the concept of live, online seminars based on the science of learning. Maintaining a synchronous classroom experience allowed us to base final grades on in-class performance instead of exams—something core to our pedagogical approach.

The classroom needed to support up to 20 user AV feeds at the same time under a variety of network conditions, as well as allow for a flexible interface that could support a combination of users, resources, and activities.

For students, it had to demand the full attention of each classroom participant, provide streamlined mechanisms for student input, and make it feel like every student was sitting in the front row, ready to be called on in a moment’s notice.

For faculty, the classroom had to provide robust teaching tools that would enable them to efficiently change the format of the class and bring as much life as possible into the lesson plan. 

For both, the technology would need to recede into the background and allow the focus of the users to remain on the in-class discussions. 

The Users

The users of Forum’s seminar classroom were twofold: Ivy-league, tech-savvy 18-year-olds, and faculty members leading fast-paced, ninety-minute class sessions on topics of their expertise. We could assume that both had a base level of comfort and familiarity with tech, but could not assume that all were power users.

My Role

I led the product research, ideation, and design of the classroom, supported by our insanely talented engineering team, Chief Product Officer, and our Chief Learning Scientist. I also worked closely with our Founding Dean to better understand the needs of faculty teaching on the platform. 

The Limitations

The project’s ambitions were grand, and we had a firm deadline to build a classroom ready for Minerva’s inaugural class of students in the fall of 2014. 

This needed to be done with limited financial resources, no formal QA team, and while personally working remote. A daunting challenge, but one I was excited to dig into.

The Process

Initial Research

Research began by discussing the engineering limitations with the team and working to determine a core list of “must-haves’ from a feature standpoint that would allow our curriculum to be effectively taught by the first day of classes. 

These included representing every student in class via a video stream, a dynamic, flexible classroom that could quicky adapt to a wide range of layouts to support active learning, systems to measure every event that happened in class, and the need to make every student feel fully engaged in a tight-knit, seminar environment. 

With this as a starting guide, I began by recruiting college students who had taken a portion of their classes online to better understand their experience. After interviewing a half a dozen students I found there were themes around pain points that repeatedly appeared in our conversations. These included:

• feeling disconnected from their classmates
• not feeling enabled to speak up in class, either to make a point or ask a question
• having these feelings lead to a sense of boredom and “tuning out”

I also interviewed our Founding Dean to better understand the needs of faculty that were to be hired, which boiled down to the ability to work closely from a lesson plan to ensure that different students taking the same class from different professors would receive the same level of education, while also allowing the teacher to dive deeper into a subtopic of class interest freely, supported by tools that led for rich expression.

Personas

Armed with this information, I created several personas to use as the design process progressed. The student personas included “Miranda”, who was eager, prepared and excited to contribute to class, and “Carlos”, a student who, while containing the same amount of raw aptitude, fell a little more on the introverted side of things and might be more reluctant to participate in class. 

Personas helped keep the team focused on a wide range of students with varying needs.

Being able to refer back to these personas helped the team better calibrate each feature so that it was beneficial to both types of personalities. 

Designing and Testing Assumptions

With the beginnings of research starting to aim the team in the right direction, I then led a series of multi-day design sprints to explore areas rich for ideation, including:

• How might we make the classroom feel as good as, if not better than a real-world seminar classroom?
• How might we allow professors to quickly modify the layout of class?
• How might we allow students to quickly break out into smaller discussions, generate artifacts, and then quickly bring these back into the classroom?

After seeing which storyboarded ideas resonated with the team the most, I'd then synthesize these into an actionable solution using the personas as a guide, and then create a low fidelity prototype. Time permitting, I’d then either personally recruit more college students to synchronously test with, or set up remote, asynchronous testing sessions via usertesting.com. 

Design sprints led to rapid feature ideation that could then be quicky prototyped and tested. 

Lessons Learned and Moving into Production

Testing the prototypes against users was invaluable and informed us as to which core pieces of classroom functionality resonated the most with users. Feeling more confident, the engineers began on coding our initial MVP version of the classroom. 

With this functional prototype in place, the product team took turns alternating between teaching a class on an area of their expertise using the faculty tools and acting as an attentive student participating in the class discussion. 

This led to more feature maturity, as interactions were smoothed and the layout tightened for greater efficiency.

Our progress only accelerated once our Founding Dean began to teach on the platform. He was adept at drawing students into incredibly active discussions and relied on decades of teaching experience to fully push our ideas of what the classroom software could be capable of.

With the learnings from the research and robust prototype testing, I made the following design decisions at this stage:

Video layout

In initial classroom layouts, the student and professor videos were situated around the edges of the screen, surrounding the main display area in the center of the stage. The intent was to make the classroom visually resemble a small seminar classroom that had chairs arranged in a circle. 

The biggest drawback of this approach was that it led students to look at varying regions on their screens, which led to eyes that jumped around in the resulting feed. 

Early classroom layouts experimented with placing students on all sides of the classroom.

I realized that the closer the student’s gaze was to their webcam, the more natural-looking the result would be in the resulting classroom. We experimented with placing the videos of students in a single row at the top of the screen, which led to students focusing their gaze closer to the top-mounted webcam. The effect was a closer approximation of the eye contact students would expect from a face-to-face conversation and a much more natural feeling experience overall.

Resource Layout

The team all agreed that the largest region of the screen (which we referred to as the “stage”) would be reserved for either resources or enlarged video feeds, and initial prototypes had the main area of the stage alternating between a full-bleed display and a half-and-half layout. But testing paper prototypes of this layout with our Founding Dean led me to realize that more flexibility would be needed. 

With this in mind, I came up with a solution in which the stage could be quickly divided into a number of “panes” that could contain a video feed or any resource necessary for the lesson. The number of panes was tied to keyboard shortcuts, so for example when the faculty pressed “4” the stage would divide into four equal regions that could then have resources or student videos dragged and dropped into place. 

Careful consideration was given to allow the professor to rapidly change the layout of the classroom to facilitate rich student discussion. 

By quickly being able to go from a single document to nine users in a 3x3 (lovingly referred to as “Brady Bunch mode”) in a single keystroke was powerful and facilitated to richer classroom discussions.

With a single keypress, the professor could bring a quick nine by nine grid of the least talkative students to the "stage".

Polls

The initial poll implementation was functional but focused only on multiple-choice questions and the display of these in aggregate once the poll was over. While it worked as an MVP, something more was clearly desired. We added support for long-form written polls and, based on early feedback from test classes, also added two important features that have seen a lot of use: The ability to pull up results of two earlier polls side by side for comparison’s sake, as well as the ability for the professor to select one student’s long-form written poll answer and present this to the rest of the class.  

Being able to quickly compare two different results of teh same poll allowed the class to easily see whose minds were swayed by classroom discussions.

Breakouts

The ability to divide students into smaller groups to work on problems before reconvening back to the classroom was a known need from the beginning. 

The initial implementation worked and was far more efficient than the real-world equivalent of having students physically rearrange themselves into new groups, but we learned that additional enhancements would be necessary to make it a truly useful feature. 

One problem was that once the breakout was ended, the document that the breakout group collaborated on was often brought back into class. However, just presenting the document to the class seemed like a wasted opportunity. The real need that we heard from the teacher was the ability to not only bring the resource back into the classroom but the students who created it within the group as well. 

With this knowledge, I designed a control for the professor that allowed them to bring every member of a breakout group and their resource back into the classroom with a single click. This led to much richer discussions and learning opportunities as students were forced to explain and defend their decisions to the rest of the class.  

Initial Launch and Observation

With the feature set becoming more stabilized week by week and the number of bugs steadily declining, we finally had a product that was ready for actual classes. And in the fall of 2014, Minerva Schools at KGI welcomed its inaugural class.

This was an exciting moment for the team. What we had painstakingly created was now being put to the real-world test with 24 classes being taught every week in the classroom. 

Invisibly observing as many classes as I could, I began to notice some issues that seemed to occur on a regular basis. I began booking 1:1 time with our teaching faculty and students to begin digging into these a bit more.

Issues Discovered and Solved 

Talk Fairness

One thing that became apparent in the first weeks of class was that despite the best intentions of faculty, certain students would tend to dominate the discussion. This was a critical problem for classes that were designed to get every student to contribute. 

As we had data on who exactly spoke when in class, I sketched up a variety of solutions that could bring more fairness to speaking time, from a dynamically changing list aggregate talk time per student, to a region of the screen that would display the avatar of the student to call on next. 

One solution I prototyped seemed to resonate with faculty in which by pressing and holding the “T” key would make one of three colors appear as a transparent overlay on the top row of student videos: Red if the student had talked more than their peers, yellow if they had contributed a moderate amount, and green if they hadn’t contributed enough.

By keeping the trigger for this feature tied to an active keypress, it could be summoned and dismissed quickly, directly at the moment of decision. And by using the color overlay, the professor could maintain eye contact with students and easily see who would be the fairest person to call on next. 

This feature was released with some skepticism by the faculty, but it quickly won over converts. Said one faculty member:

“I was initially skeptical of this feature but by the end of the semester my ‘T’ key was darker from overuse of this feature.”

With a press of the "T" key, professors can easily see the average talk time for all students and can quickly identify which should be brought into the discussion next.

Student Expression

In an effort to reduce “hot” microphones picking up background noise that could bleed into the class and disrupt classroom focus, the decision was made to disable student microphones by default. While every student can actively and easily turn their mic on to contribute to class, the side effect of this mute-by-default is that a lot of passive audio such as laughter is lost. 

While every effort was made to make the virtual classroom seem as similar to a live seminar classroom as possible, the use of small webcams that capture only a portion of student’s bodies meant that a lot of emotive body language didn’t translate into the Forum’s classroom. While students did and do have access to an in-class chat room (with emojis!), they didn’t feel like they could fully express themselves. 

I surveyed the students who felt strongly about this issue and scheduled a working session in the San Francisco residence hall. Turnout was spectacular, due in no large part I’m sure to my having ordered six extra-large pizzas.

Over the next ninety minutes, I learned that much of what students wanted to convey during class was outside the context of explicit commentary. Working with the group and writing on a whiteboard, we identified a set of expressions that students wanted the ability to easily communicate. These included:

• Smile
• Frown
• Laugh
• Agree (snap)
• Disagree
• Confusion

With this knowledge, I began working on a system to support this. I initially explored leveraging the chat portion of the interface to enable these emotions, specifically richer controls that conveyed more than a simple emoji. While promising, this approach didn’t take into account the simple fact that not every student has the chat interface open during class; additionally, professors tended to open the chat only when they were notified about a direct message from one of the students. 

Heading back to the drawing board, I began looking at how the signals could appear on student videos instead. We had already introduced a pattern in which simple “yes”, “no”, and “hand raise”  symbols would appear on top of a small portion of the videos, and I began to explore how this pattern could work with emotions. 

I designed a system in which hitting a shortcut would make an overlay appear, explaining the available emotions. I also created a keyboard shortcut for direct access.

Once selected or the shortcut pressed, a small, colored emotion overlay appears on top of the video feed, pauses for a beat before shrinking and disappearing. 

The result is a really fun, expressive effect that makes the classroom feel warmer and more interactive. And when triggered en mass, the result is a classroom that feels much more alive and, well, human. 

Students can express themselves with a series of lightweight, animated reactions without disrupting the flow of class.

Timeline

From the beginning, I had designed the teaching experience within the classroom to be flexible to accommodate the widest range of curricula possible. This gave the professor a multitude of virtual levers to reconfigure the classroom in any number of ways with the ultimate aim of advancing the discussion and deepening learning. 

This, however, came at a cost. 

The cognitive load required to lead a classroom discussion while following a lesson plan was enough to push the limits of even the most seasoned faculty members. When then asked to manually configure the layout of the classroom on top of this for every step within the activities, things began to break down. What we needed was a way to remove the necessity of manually reconfiguring the classroom at every step in an activity. 

I explored a set of different solutions including everything from a series of advanced keyboard shortcuts that could rearrange the classroom more quickly to shifting stage management responsibilities from the professors to the TAs.

What started to resonate with faculty, however, was a feature that represented lesson plans as clickable timelines within the classroom. I designed a professor-specific sidebar that would show every activity in the lesson plan, broken down into clickable steps that would automatically change the layout of the classroom. The initial version of this proved wildly successful and after watching it in use I realized that it could be leveraged to help professors stay on time. 

Within the Course Builder tool, we added time limits on a per activity basis and surfaced this information within the classroom. At a glance, the faculty member could see how much time was allotted for each activity, as well as where they were in relation to this. If they fell 1-5 minutes behind, the time in the upper right of the timeline would turn red. If they fell  more than 5 minutes behind, the time indicator would turn red. 

These features not only led to more consistent completion of the entire lesson plan in every class, but also alleviated pressure on faculty to stage manage and allowed them to focus on teaching their class instead.

The timeline (pictured at right) allows the professor to access an interactive outline of the lesson plan that automatically adjusts the classroom layout to enhance active learning.

Outcomes and Lessons Learned

The initial Forum classroom launch was a solid release, but one that still required multiple rounds of observation, research and design thinking post-launch. Throughout this process, I realized that we had both underestimated the cognitive load that effectively teaching a class would require, as well as the desire for student self-expression. 

Iteration and innovation in the seminar classroom continues to this day, with the Forum recently launching a new 2.0 version that allows it to scale from 20 students to hundreds. 

Minerva matriculated their first class of graduates in the summer of 2019, all of whom received their education through the Forum classroom. Currently, the classroom software is used to teach hundreds of classes a week at Minerva as well as dozens of partner institutions globally.

Contact

Email: mattregan@gmail.com
Twitter: @mattraygun
LinkedIn: /reganmatt
Instagram: @mattraygun

© Matt Regan 2021