Confab Higher Ed:"How Do You Make the Good Great? A Case Study on Red…

No notes for slide

Presentation to Confab Higher Ed 2017.

Hello! I’m Kelly and this is my colleague Jackie.

Michigan Medicine is the academic medical center at the University of Michigan. It’s a big place! 25,000-plus employees, including 2,700 faculty. We have a top-ranked medical school with 800 students and a clinical operation that sees more than 2 million clinic visits a year.

Today, we want to tell you the story of one of our flagship websites, the main U-M Medical School site, and how used user research to drive improvements to the site. I’m going to be talking about how we worked with stakeholders on this project, and later how we translated our research into design ideas, and Jackie will be sharing on the research process itself.

But first, we need to start with a little history.

This is the Medical School site in 2012, a few months before the launch of a major redesign.

This site suffered from what I think is a familiar problem to those of us in higher ed or nonprofit tech -- places where decision-making can be very distributed and consensus-driven, and big, bold changes can take a long time to effect -- and that’s the tragedy of the commons.

While our school continued to attract excellent students and to train excellent doctors, its website was a bit of a mess -- thousands of static HTML pages, multiple menus and calls to action, and a muddied focus.

In 2013, we launched a big change -- moving from static HTML to a modern CMS (Drupal!), introducing a responsive design, and cleaning up the site architecture and its content to be more strongly tailored to our core user group of prospective students, for instance, by adding homepage calls to action based on educational program.

What you can’t see from this screenshot is the work we’d been doing behind the scenes to shift the organization to a model of stronger central stakeholder involvement in web projects. In this case, that meant assembling a formal committee of folks from across the institution.

That fall, we got back responses from a survey of prospective students on the website changes. Here are a few comments from students.

That probably sounds like a great place to end this story, right, a happy ending?!

The users loved it! Mic drop!

But, there’s no quitting in web design and we knew, about 18 months into the lifecycle of the new site, that we needed to start thinking about what was next … eep!

So, this is going to be the story of what comes after Happily Ever After.

Our story today is in three parts:

Part 1: Aligning stakeholders. This part of the story is about re-engaging with business owners to identify a direction for the project.
Part 2: Designing the experience. This chapter is about translating a business need into a user research question on an interesting institutional problem.
Part 3: A modular content system. This last section is about how we used the lens of time to think about our content management system in a new way, ultimately building a whole new approach.

So let’s get started with stakeholder wrangling. A word for our colleagues:

What do I mean when I say “business owners”? In our organization, with more than 25,000 employees and dozens of departments, centers, and programs, run by thousands of independently minded faculty, it’s not always easy to identify who this may be.

I’m talking about the folks who:
Have power to make decisions and drive alignment in the org. (Because, let’s face it, consensus is a fact of life for most of us in higher-ed cultures.)
Have insight into the strategic vision of the org. Where are we going and how are we trying to get there?
Understand how the pieces of the org fit together, and where there are opportunities to collaborate, or where politics may necessitate treading lightly.

A little background on our team’s role within our org because I think it’s useful context: We’re an embedded creative services team, and we function similarly to external consultants, even though we are part of the organization — clients hire us to provide service on projects large and small. This gives us an interesting insider-outsider perspective on the org, as we get to be both consultants and boots-on-the-ground workers.

We’re an embedded creative services team, and that means we do projects for clients across our organization. We’ve learned that ongoing stakeholder relationships are vital to understanding the business needs that are driving requests for web services.

Because a carousel is never just a carousel — as Freud might have put it — but may represent our institutional hopes, dreams, and fears. So we’re always digging beneath the surface of a web request to better understand what led to it. By keeping a bit of the mindset of an external consultant, we can stay curious about our own organization and continue to ask good questions that get at underlying needs — this mindset can help us build better products.

Now, when you go to your stakeholders, it’s easy to forget that maybe not everyone is excited about the prospect of redesigning an enterprise website with dozens of stakeholders and half a million annual users (go fig).

In our case, we got a few excited responses, and a few “Um, didn’t we just do this … ?” (Well, technically it was 2 years ago, but who’s counting …)

There’s some interesting debate in the field about making incremental improvements (e.g. evolutionary redesign) vs. pursuing big cutting-edge changes, and so I think it’s worth considering what your organizational culture and your team’s resourcing levels can tolerate.

One of our tactics for engaging stakeholders ...

In our case, it helped to call our project a “revamp,” because we really weren’t sure going into it, what the scope would or should be, and we also found that by calling our project a “revamp” instead of a “redesign,” it put stakeholders into a more playful and creative mode, instead of girding their loins for another epic cut-and-paste into a CMS.

So to kick off the project, we threw a “revamp” brainstorming party, put on music, fed everyone some good food, and seeded the session by turning our conference room into a gallery of screenshots of beautiful new higher ed sites, screenshots of our competitors’ sites, and images and quotes from our users so we could “hear” from them directly about what they liked and didn’t like about their site experience.

Then we got out a tool that we’ve found to be excellent at democratizing feedback and revealing misalignment in a group of stakeholders …

This method is called the Performance Continuum, which I learned in a workshop with Dan Klyn from The Understanding Group.

The way I explained it to my group of stakeholders was like this:

The idea behind Performance Continuums is to set a strategic direction — what before how. (Remember, it’s never just about a carousel, so let’s talk about what are our real business drivers.) In practice, this is an activity that asks you to create a handful of factors that describe what we are trying to accomplish with this project and where we stand today. Key detail: Stakeholders develop these factors together, so they are integrally involved in defining the metrics of success, which means -- cha-ching! -- instant buy-in.

Then you place a marker on a scale that represents a desired future state reflecting a successful outcome for the website.

Among the performance metrics that our stakeholders defined was how we approached content on the site, and a desire to move strongly toward more storytelling and image and video-driven content vs. traditional informational “facts & data” content.

This slide shows how we captured stakeholders’ current state assessment (red) and future state desire (green), e.g. where we are / where we want to be.

The wonderful thing about this tool is that because opinions become notches on a scale that then are aggregated, the scale naturally reveals where there is alignment -- and where there isn’t. In our organizational culture, this can have a democratizing effect on feedback that is transformative.

As a result of this exercise, we identified only 1 of the 5 performance metrics where the group was not strongly aligned, and that led to a fruitful conversation about how to approach the difference of opinion in the remaining area.

These exercises also reaffirmed the original principles that animated our first big redesign -- including a strong focus on prospective students, and flexibility for a large team of distributed content managers to be empowered by the CMS structure to publish great content.

So, with the stakeholders on board, and some idea of what “better” might look like to our committee -- more storytelling and more engaging content presented to our core audience of prospective students -- we got focused on identifying a core use case to explore through research with the goal of learning something new about our users that would inform our next steps as designers.

Jackie’s going to tell you more about how we approached the research process.

We are fortunate that we have great partners at the Office of Admissions who were invested in us doing research to identify the needs of prospective students and how that research could inform design decisions. Kelly and I sat down with the marketing lead from the Office of Admissions, who walked us through the Medical School recruiting process and recruiting goals of the school including the types of students they wanted to attract.
It was during this meeting that we learned about a couple of interesting recruitment challenges: The first challenge was that ((early application is better for students because they have a better shot at getting one of the limited number of interview slots))

While a student may receive an offer of admission as early as October, they aren’t required to accept or reject that offer until April. This 7-month window is crucial for the school because this is where the student is making a decision and potentially weighing different offers of admission.

So we were curious: What’s happening during this time time gap and does the Medical School website factor into this decision-making process?
So, we did what any good prospective medical student would do: We did a ton of research on applying to medical school and we started here, with this guide for medical students provided by the Association of American Medical Colleges or AAMC, which is the governing body of medical schools in the U.S.

From there we branched out. We learned that applying to medical school is both a lengthy and complex process that has a central application process through the AAMC, with other deadlines and demands set by the schools themselves.

So we took all of these deadlines and mapped out a timeline of the major milestones of the application process, and added U-M’s deadlines and touch points such as secondary applications and interviews.

Doing this mapping allowed us to test our understanding of the process of applying for med school and helped us identify areas and times where we didn’t necessarily understand what was happening.

What you see here is a cleaned-up version of our timeline. This is what happens when you have a graphic designer make something nice out of my mess.

Remember that 7-month gap between October and April? The map let us see how interesting that initial question was, because we were able to see how many other milestones were occurring during this window, including financial aid offers, scholarships, and Second Look Weekend (this is when admitted students come back to campus to get more of a feel for the community).

This timeline also helped us ask better questions of our students during our interviews because we could see both the quantitative (what were you doing when) and qualitative (how did you feel about) parts of the process.

Ultimately, we were able to use this timeline to create our final research question: Do prospective med students’ information needs change over the course of the application process, and if so how?

So, with a better idea of what question we were trying to answer, Kelly and I scheduled interviews with 8 first-year med students.

We sought out a group of students that could speak to the recruiting goals of the school: in-state, out-of-state, nontraditional, and students who came from traditionally under-represented groups in medicine.

Additionally, we wanted to interview students who received multiple offers of admission so we could understand what students were weighing between October and April before choosing to come to Michigan.

Preparing for the interviews, we were inspired by the school’s Instagram, which is mostly content generated by the students themselves.

Looking at these photos was another way to challenge our preconceptions of who our users were. These students did not fit into what I thought of as the Type A cookie-cutter medical student. These students had diverse background and interests and through their Instagram account gave us a flavor of the type of culture students are creating as a community of learners.

Remember that timeline? It was also a great interview prompt. While we initially created it for ourselves, we used it in our interviews as a way to talk about what students did and when they did it. It especially worked well for helping students remember what they were doing during specific parts of the process. We also asked that students bring any of their own application artifacts such as pro/con lists, spreadsheets, and any other decision-making artifacts they created during the application cycle.

Using the timeline to address our research question of whether students’ information needs changed during the application life cycle, this approach helped us understand:

What information were they searching for?
How much time did they spend with the information?
What questions did they have during different parts of the cycle?
How did they react to or respond to the content they found?

One of the interesting artifacts exposed by the interviews was something called the “LizzyM Score.” Early on in the process, students are narrowing down the list of schools they choose to apply to and the LizzyM helps them do this.

During our interviews this was actually a spreadsheet shared via a premed Subreddit, but is now a snazzy web app.

Students use this to compare their academic test scores and GPA to historical data from medical schools so they can see where they have a chance of being admitted.

This artifact was interesting:

1. Because it exposed that early thought process during applications where students are narrowing down their choices; and
2. It shows some very early information filters that students are using before they even apply to med school and even before they look at our website.

Originally we assumed that our medical school website was the center of their application universe, but came to understand that we were just a pale blue dot in the universe of information that our users were relying on.

Expanding our reference to account for this larger constellation of information sources was a big perspective shift for us.

Another surprising detail from our interviews was that some of the students had impressions of the university from their time school shopping as undergrads, and in some instances that impression clashed with what they were looking for in a med school.

Was U-M a big state school known for its football team and vibrant undergraduate social scene?

Or was it the home for rigorous and top-rated professional schools?

Of course, it’s both, but it’s worth bringing attention to the many facets of an organization’s public persona and how that affects the way your users view your message.

So we were surprised by a few things, but what did we learn?

We found ourselves increasingly returning to the lens of time as a new place of meaning that offered insights for understanding how users experienced the process of applying to school — and for understanding how the application timeline led to specific patterns of information consumption on our website.

As UX researchers in higher ed, we’re always looking for good tactics to borrow from our peers in the private industry. One of the frameworks for analyzing our research that appealed to us was the framework of the customer journey or user journey.

This is a map of a customer’s behavior from the beginning to end of a process, and usually includes details like their motivations, their actions, and the touchpoints where they engage with an online product.

But medical school is not just any old product. So here’s how we adapted the idea of a customer journey to work for us:

First, we reviewed all our user interview transcripts and highlighted important details — motivations, actions, thoughts, questions that users expressed.
Then we wrote all those notes on individual Post-It notes and grouped them thematically.
Then we grouped comments by timeline, mapping users’ recollection of what they did when to our existing admissions timeline with all its milestones.
From there, we developed high-level descriptors of each set of comments to tell the story of how students’ experience changed over time, and what was most important to them, when.

The giant map you see here is the final result.

Returning to our original research question, it turns out that yes, students’ information needs changed over time and there were commonalities in the genres of information that they found compelling, especially at the beginning of the process.

We found that early in the admissions process, quantitative data such as school rankings and data that allowed students to do snapshot comparisons of schools was sought after during the awareness and research phases — remember, they are applying to 25-30 schools.

However, that information was less important during their decision-making and commitment phases. Instead, they wanted qualitative data such as a school’s learning philosophy and how their interests and voices were incorporated into the school’s culture. They wanted to know if they could see themselves here and they wanted information that would allow them to feel like a part of the community before they arrived here.

Taking our timeline and user interview findings, we then looked deeper into these phases of the customer journey to see where students’ information needs intersected with our website.

Again, we used our user comments and our knowledge of what they were doing when to break up the interview data into 4 components:

Time: in blue, which is when they were visiting the site
Topic: in pink, what information they were seeking
Modes of seeking: in yellow – were they skimming, comparing information, doing a deeper dive — in other words, what was their motivation at the time, and how were they using information?
Genre of information: in green – what format of information might best match their mode of seeking (profiles, news, informational content, graphics, data, etc.) Genre of information is also sometimes called a content pattern. These were our ideas for thinking about what format of content might best meet user needs, based on everything else we’d learned from the mapping exercise.

As a lens, time gave us a new dimension to consider for our research:

1. It made us think about the opportunities to improve site engagement during certain times when students were reading more deeply and wanted more qualitative information; and
2. It allowed us to see constants that time places on our students, including how much time they have to interact with our site — are we putting information into the best form for it be consumed?
The next step here was to turn these opportunities and constraints into a strategy and a design for the website. I’m going to turn it back over to Kelly to talk more about that.

And this seems completely obvious in retrospect, but we also discovered by using the lens of time is that students were retaining information along the way — each time they came back to the site, they might have a different information need and be looking for different information. In other words, while it might be helpful to see prominent school rankings on a top landing page early in the process, that information was old news — even taking up valuable real estate — later in the process.

This was a lightbulb moment for our design process, as we started thinking about how to design our CMS to offer a flexible information hierarchy that could serve up chunks of information in appealing genres based on what we knew about changing patterns of use during admissions.

If we could make the CMS more flexible, we could empower content managers to make more choices about what to emphasize when, and provide more actionable and appealing information throughout the cycle.

In the wireframes here, you can see a “first look” admissions landing page and a “later cycle” version of the same page, where we envisioned how modular chunks of content could be swapped in or out to emphasize different kinds of information.

Early in the admissions process, like in the design on the left, we might want to highlight key facts and easy points of comparison. Later in the process, we want to highlight the active application process using our popular Tracker widget, that provides transparency into the number of applications and interviews — and gives students a reason to keep coming back to check on the website.

Another detail from our user research inspired us to reframe the way we presented the student profile content on the site, a series called Points of Blue. With the understanding from our user journey model that students were returning to the site later in the application process in part to assess school culture, we redesigned the presention of the content to better showcase the faces and individuals being profiled. You can see the old site on the left and the new one on the right.

In all, we designed 12 modular content components for the new site, which launched in January of this year.

For our fellow Drupal nerds, I’m happy to talk your ear off about how we implemented this system in the CMS. So find me later :)

From a design perspective, we approached the creation of these modular components by combining what we had learned about students’ information-seeking needs (what they wanted to know when) and we also audited our existing site content by looking for recurring content patterns, printing out those pages, and then using a pen to outline the unique components that made up a page: pull quotes, callouts, social media buttons, images, video, etc.

We were looking for the building blocks of our site.

We also encountered a few challenges designing content in a new, modular way:

How do you come up with descriptive names for a component that also make sense irrespective of the component’s position on the page (hello, mobile design)? Tip: I looked at the source code for lots of websites using similar components to get ideas for what to name them!
What was the right number of components? This grew a little organically as we learned some things in the build stage, so we have a couple of components that are pretty similar to each other, and could probably be combined.

Here’s one more before-and-after showing our old Admissions Timeline page and our new one.

The MD Admissions Timeline page used to be a brief month-by-month overview of the process and then a “subway map”-style graphic that explained all the steps in the process.

We worked with marketing to combine the content into a more robust month-by-month timeline page that re-integrated the timeframe with the milestones so it was more clear what an applicant could expect, or needed to do, and when.

For the new page, we used the Metrics Blocks component to highlight key facts and deadlines, shown in the blue boxes on the page.

So, what was the outcome of this redesign?

We’ve gotten excellent feedback from our content managers, who vastly prefer the modular system to dealing with a WYSIWYG.

We also have been really pleased with the flexibility of the modular system. We think the content patterns we identified have longevity that could allow us to do our next redesign as a simpler look-and-feel update instead of a full migration.

Some other measures of impact:

The Admissions office saw a big bump in both early and overall applications this year. While we don’t know for sure all the factors that led to this increase, we know the website is the most important source of information to students, and so we expect that the site played a role here.

We also surveyed applicants to the school in July of this year, and received high marks from prospective students overall and in delivering information that provided the details they needed at any given point in the application cycle.

(Data from July 2017 survey of prospective students who had applied to Michigan; 515 survey responses total.)

And one last quote from an applicant in this year’s survey …

We’re feeling like this all over again. And we think we’ll take a few months off before the next redesign.

If you’d like to learn more about making your content modular, here are some articles we found helpful! Our slides will be posted with the conference materials later.

Thanks for listening and we’re happy to take a few questions!

Confab Higher Ed:"How Do You Make the Good Great? A Case Study on Redesigning the University of Michigan Medical School Website"

Jackie Wolf

Confab Higher Ed:"How Do You Make the Good Great? A Case Study on Redesigning the University of Michigan Medical School Website"