Using online tools to
help us assess our PLE work
Presented by Fiona MacCool,
Your Legal Rights Project Manager
CLEO (Comm...
What we’ll cover:
•
•
•
•
•
•

Introduction to Key Performance Indicators
Google Analytics (The Basics!)
Online User Testi...
Let’s think about evaluation not as a single
measurement in time nor as a vast collection of
statistics you collect for fu...
What are Key Performance
Indicators?
Performance measures (or KPIs the terms are
more often used interchangeably these day...
Why use KPIs?
• They allow us to set targets and improve our
site/projects over time
• They allow us to take action to imp...
Key Learning(s)…
• This is not something you should be left to do on

your own as the “web person” or project manager.
You...
The Process

1. Why - Look at your Vision Statement (for the org) – how can
your website help you deliver on this objectiv...
What makes a good KPI?
“We know we will be successful when this RESULT is achieved”
Types of Success Evidence
1.
2.

3.
4.
5.

Financial – online purchases, tickets sold, fundraising $ - Success
Evidence – ...
Examples of KPI Metrics
• % of content greater than 1 year old – (insert
appropriate time frame based on sector/topic)
• %...
Key learning(s)…
• It is worth the time making a few customizations or
configurations to your website or online project ea...
How to get started…
•
•
•
•
•

Think about a project in your organization…
Think of what the goal of the project is?
Think...
Website Sample
Project

Goal

Result

Measurement

CLEONet.ca –a PLEI
site for service
providers was rebranded and
launche...
Project Sample
Project

Goal

Result

Measurement

PLEI Webinars for
service providers in
Ontario

Increase the reach
of P...
Project Sample
Project

Goal

Result

Measurement

At Your Legal Rights
we want to make
sure we are
responding to
outreach...
Benchmarking and Baselines
Benchmarking vs. Baseline
•
•

Benchmarking – comparing one’s business process against competit...
Featured Tools
What are users doing on your site and how?
• Google Analytics – Tracks use and can follow “goals” or
“event...
Featured Tools
Online User Testing
• Usability testing – You can pay third party sites/services to set
up task-based tests...
Featured Tools
Voice of the User
• 4Q/IPerceptions – Survey that is used widely and
asks 4 basic questions to random visit...
Google Analytics – Basics

Bounce rate is the percentage of visits that go only one page before
exiting a site: If the poi...
Custom Dashboards

Set widgets for the stats you care about
most – In this case, visits, bounce rates,
sources and goals
Use Google Analytics to Calculate
Percentage Shifts Over Time
Set “Goals” in Google Analytics
• If there are things you really want your users to “do” on your
site, set them up as “Goa...
Examples of Goals Types and Reports
Heat Maps and Screen Recordings
Google Adwords - Keywords

Adwords is a great tool to suggest keywords and help you know which
words are commonly used in ...
User testing Best Practices
• Should be task-based rather than “do you like
our site?”
• Should not be “guided” or assiste...
In-Person User Testing
• Should be facilitated or led by a neutral 3rd
party that can provide tasks and instructions
and s...
Virtual or Online User Testing
• Because we are trying to reach a vast
geographic area with limited funds, we
elected to c...
Online User Testing: A Case Study
• In April and then in July 2011 we conducted sets
of online usability tests for the You...
Wireframe Test: Where would you click
to find information on eviction?
Mock-up Test: Where would you click to find
information on eviction?
Sample Test Result Showing Heat Map
Where would you click to find information on Eviction?
Examples of
Comments Reports
You can use these tools to test
accessibility, navigation,
emotional responses, user’s
memory, and A/B testing
Samples her...
A/B Testing
• Create two “competing” versions of a page and
then have Google create a test where 50% of
users see one and ...
User Experience Surveys
• Create a post-visit survey and set an invitation
rate (20 to 50% of visitors) for key time
perio...
IPerceptions (formerly 4Q)
Rate of return based on Invitation Rate
We set the Invitation
rate to 20% for desktop,
10% for mobile and 20%
for tablet a...
User Experience Survey
Comments by Purpose of Visit
Challenge:
People keep
“anonymously”
telling me what
they wish they
had found and I
can’t hel...
Social Media – Followers/Users
Twitter
As of July 31, 2012 we have sent out 3900 tweets and have 1,560
followers.
As of Ma...
Social Media– Measuring Engagement
Twitter - “Re-tweets” and “Mentions” – Hard to measure this over time, better to
log it...
Social Media – Facebook Insights
Social Media – Klout Score
Tweetreach

https://tweetreach.com/reports/7661919?v=2
Discussion Questions?
How do we figure out who the audience is and why they
are visiting our sites or using our PLEI? It i...
Let’s do a sample together…
•
•
•
•
•

Think about a project in your organization…
Think of what the goal of the project i...
Keep in Touch!
Fiona MacCool
Your Legal Rights Project Manager
Tel: 416-408-4420 ext. 829
Email: fionamaccool@yourlegalrig...
Upcoming SlideShare
Loading in …5
×

Using online tools to help us assess our public legal education work

718 views

Published on

October 2013 - Public legal education (PLE) is increasingly delivered online. This webinar will look at how to leverage a number of free or low-cost online tools (including Google Analytics and iPerceptions surveys) to acquire data to measure your impact and align with your key performance indicators or KPIs.

Other tools that will be discussed include online user testing tools and what metrics matter when it comes social media evaluation.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
718
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • {}
  • Using online tools to help us assess our public legal education work

    1. 1. Using online tools to help us assess our PLE work Presented by Fiona MacCool, Your Legal Rights Project Manager CLEO (Community Legal Education Ontario / Éducation juridique communautaire Ontario) October, 2013
    2. 2. What we’ll cover: • • • • • • Introduction to Key Performance Indicators Google Analytics (The Basics!) Online User Testing User Experience Surveys Social Media Evaluation Questions and Discussion
    3. 3. Let’s think about evaluation not as a single measurement in time nor as a vast collection of statistics you collect for funders. Evaluation, especially when it comes to online tools or websites, should be an ongoing process that ensures that continuous improvement is happening in ways that are measureable, evidence-based, and that contribute to the strategic goals of the organization. (EASIER SAID THAN DONE!)
    4. 4. What are Key Performance Indicators? Performance measures (or KPIs the terms are more often used interchangeably these days) are objective evidence of the degree to which a performance result is happening over time. They are feedback about your progress in improving aspects of your organization. (Source: http://staceybarr.com)
    5. 5. Why use KPIs? • They allow us to set targets and improve our site/projects over time • They allow us to take action to improve • They let us measure the viability of time consuming activities – should we keep doing what we are doing? • They help us answer the question - What is the point? Why are we doing this? Are we delivering on our promises?
    6. 6. Key Learning(s)… • This is not something you should be left to do on your own as the “web person” or project manager. You should brainstorm every element of this process with others, including board and management • There is no one magic tool that can do this. You need to come at with combination of approaches and modify them regularly. • We need to share our strategies and techniques with one another as we are breaking new ground here.
    7. 7. The Process 1. Why - Look at your Vision Statement (for the org) – how can your website help you deliver on this objective (along with other channels) 2. How - Strategic Plan, or Services Review Plans (one way our sector has advantages over the private sector) – Integrate key initiatives and where the org is heading to meet core goals and objectives 3. Outcomes/Results - “Defining Success” – Looking at why and how – what outcomes or results will create the collective “fist bump” – what does victory/success look like for the org and for the project?
    8. 8. What makes a good KPI? “We know we will be successful when this RESULT is achieved”
    9. 9. Types of Success Evidence 1. 2. 3. 4. 5. Financial – online purchases, tickets sold, fundraising $ - Success Evidence – “Met fundraising goals” Behavioural – Are people “doing” what you hoped they would (differently than without your site/project) - Success Evidence – “Increase in registrations for and repeat participation/viewings of webinars over time” Experiential – Is the process better, are they able to complete their task more easily? – Success Evidence: Time required to submit an online request is reduced, client intake interview less time consuming Attitudinal – Getting a new perspective on how things are done or what steps to take – Evidence of Success “Organizations perceive you to be collaborative and responsive to feedback” Technical – slow site – needs to be faster - Evidence of Success – “Average page load time reduced” – Fewer “page not found” errors
    10. 10. Examples of KPI Metrics • % of content greater than 1 year old – (insert appropriate time frame based on sector/topic) • % of content voted up or down or rated • % Average # of edits prior to publish • % Average time to respond to email questions/enquiries • % of users able to complete their task • % of users reporting satisfaction and/or that they would recommend the site to others over time
    11. 11. Key learning(s)… • It is worth the time making a few customizations or configurations to your website or online project early on to make sure are actually able to collect what you plan to measure . Google Analytics is great, but it’s probably not enough and nor is a simple user survey. Ask your developers about this early on if you can. • Raw numbers of stats are less helpful and not easily actionable. It is better to establish a way to measure percentages and shifts in percentage patterns. • Try to measure monthly averages or annual benchmarks – comparisons over time are key.
    12. 12. How to get started… • • • • • Think about a project in your organization… Think of what the goal of the project is? Think of ONE result of that you would like to see for the project. Think of one piece of evidence you might use to prove whether you have achieved this goal. Draft that into a KPI (Key Performance Indicator)
    13. 13. Website Sample Project Goal Result Measurement CLEONet.ca –a PLEI site for service providers was rebranded and launched as Your Legal Rights, a site aimed at the general public in Ontario Expand site audience from service providers to also reach “General Public” Within 1 year of the launch of YLR, we should see a 30 – 50 % increase in visits over the last year of CLEONet.ca - Google Analytics Visits and Page Views Within 1 year of launch we should see an increase in the % of users who report using the site to find information for a friend or family member rather than in their work or to help a client. - User survey asking question of audience/purpose on CLEONet and then on YLR for key periods - % shifts in breakdown of purpose over time.
    14. 14. Project Sample Project Goal Result Measurement PLEI Webinars for service providers in Ontario Increase the reach of PLEI from local inperson workshops to wider online availability through YLR Within 1 year of project launch we should deliver 10 -12 webinars reaching over 100 people live and over 1000 watching the archived versions. - # of webinars conducted and archived to the site - # of live participants - # of views of archived webinars - Each of the metrics looked at as % from year to year Within 1 year of project launch, identify and partner with 4-6 organizations to develop and deliver webinars - # of organizations presenting webinars with CLEO Surveys on whether webinars met user’s learning needs
    15. 15. Project Sample Project Goal Result Measurement At Your Legal Rights we want to make sure we are responding to outreach emails in a timely and helpful manner YLR should be an authoritative PLEI site for Ontario and must therefore provide information that is relevant. % User survey respondents that agree/strongly agree that YLR’s response was relevant to their inquiry Follow up survey sent to people who received a response from YLR’s outreach team. (Monthly) Count of survey respondents that “Agree” or “Strongly Agree” that YLR’s response to their inquiry was relevant Count of responses to the question vs. count of outreach emails sent Trends examined over time
    16. 16. Benchmarking and Baselines Benchmarking vs. Baseline • • Benchmarking – comparing one’s business process against competitors (industry standards – difficult in this sector to know) Baseline – initial known value which is used for comparison with later data (don’t forget to establish/retain stats before re-launching or redesigning your site) Measurement – planning development processes testing • • • Baseline from launch – For example: last 3 months of CLEONet to first three months of Yourlegalrights.ca, then year to year etc. Numbers of likes on your FB page or followers on Twitter over time Target your baseline to your audience (service providers in Ontario, general public) – Note that Google Analytics does Country and City only
    17. 17. Featured Tools What are users doing on your site and how? • Google Analytics – Tracks use and can follow “goals” or “events” set by you based on KPIs • Heat maps – Crazyegg.com and Clicktale as well as some online usability testing tools show you who is clicking what, how quickly and how often • Clicktale – records user sessions on web site – free account for 45 recorded sessions a month – shows you what/where users clicked – should help you see recurring problem areas • Google Adwords – Great tool to track keywords and compare your page rankings and click-rates
    18. 18. Featured Tools Online User Testing • Usability testing – You can pay third party sites/services to set up task-based tests your site with unknown people based on demographics you select, or you can do it yourself with tools like VerifyApp based on invitations to volunteers (cheaper than running them yourselves, especially if you serve a large Geographic area) • A/B testing – Google’s A/B testing tool lets you split traffic in half so one half of users see one site/page and the others see a second version – reports will tell you that the new site reaches more goals – Try this with your email bulletins with MailChimp
    19. 19. Featured Tools Voice of the User • 4Q/IPerceptions – Survey that is used widely and asks 4 basic questions to random visitors specific to the purpose of their visit and their task completion and site satisfaction – This tool has a variety of monthly plans with add-on features • Online Polls – asking a single question at key intervals let’s you get quick answers to questions about your users – Examples – Poll Daddy and Poll Everywhere
    20. 20. Google Analytics – Basics Bounce rate is the percentage of visits that go only one page before exiting a site: If the point of our work is to get people to find information quickly and leave our site(s) to get what they need, how can we know if a bounce is a success or whether they have lost interest or found the site useless/frustrating ?
    21. 21. Custom Dashboards Set widgets for the stats you care about most – In this case, visits, bounce rates, sources and goals
    22. 22. Use Google Analytics to Calculate Percentage Shifts Over Time
    23. 23. Set “Goals” in Google Analytics • If there are things you really want your users to “do” on your site, set them up as “Goals” in Google Analytics. Each time a user completes a Goal, a conversion is logged in your Google Analytics account. • Give each goal a monetary value merely to rank them in relationship to one another. If a visit to a particular area of your site is more important/strategic than another give it a higher $ value. This allows you to answer the question “Which pages/sections are worth our time?” • Improve a section of your site and see the Goal value rise and fall “at a glance”
    24. 24. Examples of Goals Types and Reports
    25. 25. Heat Maps and Screen Recordings
    26. 26. Google Adwords - Keywords Adwords is a great tool to suggest keywords and help you know which words are commonly used in Ontario. It also tells you your site’s average position in Google search results.
    27. 27. User testing Best Practices • Should be task-based rather than “do you like our site?” • Should not be “guided” or assisted in any way • Results should be observable and you only need enough “testers” to identify trends or repeated problems • Should be done early enough that you can make changes
    28. 28. In-Person User Testing • Should be facilitated or led by a neutral 3rd party that can provide tasks and instructions and summarize findings • Ideally, testers should verbalize their process, their confusion, their opinions as they go • And again, ideally you should be able to watch and observe these results to make your final conclusions
    29. 29. Virtual or Online User Testing • Because we are trying to reach a vast geographic area with limited funds, we elected to conduct our user testing for Your Legal Rights online in two phases • Phase 1: Testing with wireframes – Designfree blue prints • Phase 2: Re-testing with design mock-ups
    30. 30. Online User Testing: A Case Study • In April and then in July 2011 we conducted sets of online usability tests for the Your Legal Rights website using a tool called VerifyApp. • In April, we presented 15 volunteers with a set of test based on 3 – 6 wireframes. We then analyzed results and made revisions to our design plans. In July and then conducted a set of linked tests with 3 - 6 design mock ups including an option for users to annotate the page with comments or suggestions. For these later tests, we put out a wider call and had closer to 25 volunteers.
    31. 31. Wireframe Test: Where would you click to find information on eviction?
    32. 32. Mock-up Test: Where would you click to find information on eviction?
    33. 33. Sample Test Result Showing Heat Map Where would you click to find information on Eviction?
    34. 34. Examples of Comments Reports
    35. 35. You can use these tools to test accessibility, navigation, emotional responses, user’s memory, and A/B testing Samples here are from http://verifyapp.com/
    36. 36. A/B Testing • Create two “competing” versions of a page and then have Google create a test where 50% of users see one and 50% see the other and then examine results. Which page had a higher bounce rate? Which page led to the next level you were hoping users would find? • Create two “competing” designs of your email newsletter or e-bulletin and see which one has a higher open or click-through rate. (MailChimp)
    37. 37. User Experience Surveys • Create a post-visit survey and set an invitation rate (20 to 50% of visitors) for key time periods throughout the year. • Ask users what their primary task was, whether they completed it, what their “next step” is, and if they would recommend your site. They can also leave comments
    38. 38. IPerceptions (formerly 4Q)
    39. 39. Rate of return based on Invitation Rate We set the Invitation rate to 20% for desktop, 10% for mobile and 20% for tablet and leave it in place for 4-6 months each year to compare and benchmark success
    40. 40. User Experience Survey
    41. 41. Comments by Purpose of Visit Challenge: People keep “anonymously” telling me what they wish they had found and I can’t help them find it! Anecdotal Observation: Now that our site is not for service providers, users can be more negative
    42. 42. Social Media – Followers/Users Twitter As of July 31, 2012 we have sent out 3900 tweets and have 1,560 followers. As of March 31, 2012 we have sent out 5,035 Tweets to 1,998 Followers This represents a 28.07% change in the # of followers to @legalrightsON on Twitter. Facebook As of the end of July 2012, the Your Legal Rights Facebook page had 600 fans or “likes”. As of the end of March 2013, the Your Legal Rights Facebook page had 2,031 fans or “likes”. This represents a 238.49% increase.
    43. 43. Social Media– Measuring Engagement Twitter - “Re-tweets” and “Mentions” – Hard to measure this over time, better to log it on a schedule for benchmarking or baseline purposes. Lots of tools give you a “glance” of the last 90 days but for our sector, we need longer timeframes. Facebook - Facebook Insights Reports for Pages shows “Impressions” which refers to the number of people who saw your postings as well as user engagement in terms of the number of people who clicked on any of your postings or created stories based on content posted to your Facebook page.
    44. 44. Social Media – Facebook Insights
    45. 45. Social Media – Klout Score
    46. 46. Tweetreach https://tweetreach.com/reports/7661919?v=2
    47. 47. Discussion Questions? How do we figure out who the audience is and why they are visiting our sites or using our PLEI? It is more than “purpose” and can’t really be captured in demographic data… How do we know what success really looks like without any “industry benchmarks” or with such goals so difficult to measure because of client privacy? When our sites cover vast ranges of topics and given the gaps in what legal info is needed, what insights can we bring to the reasons certain topics or keywords are most visited/searched?
    48. 48. Let’s do a sample together… • • • • • Think about a project in your organization… Think of what the goal of the project is? Think of ONE result of that you would like to see for the project. Think of one piece of evidence you might use to prove whether you have achieved this goal. Draft that into a KPI (Key Performance Indicator)
    49. 49. Keep in Touch! Fiona MacCool Your Legal Rights Project Manager Tel: 416-408-4420 ext. 829 Email: fionamaccool@yourlegalrights.on.ca Web site: http://www.yourlegalrights.on.ca Twitter Feed: http://twitter.com/legalrightsON Facebook: http://www.facebook.com/yourlegalrights

    ×