• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Communications Audit

Communications Audit



Presentation to library schools in Tallinn and Stuttgart

Presentation to library schools in Tallinn and Stuttgart



Total Views
Views on SlideShare
Embed Views



5 Embeds 54

http://www.slideshare.net 49
http://auditkomunikasi.blogspot.com 2
http://translate.googleusercontent.com 1
http://webcache.googleusercontent.com 1
http://www.lmodules.com 1


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Are we hearing them? Responding to their concerns in a timely fashion? First presented at Internet Librarian International in 2006

Communications Audit Communications Audit Presentation Transcript

  • Demystifying the communications audit Barbie E. Keiser US Consulate – Stuttgart November 17, 2008
  • Because the world changes so quickly, sooner or later we all need to rebuild or refine …
    • An intranet, extranet/public website, or e-newsletter
    • The Library-related portions of an organization’s intranet or public website
    • Intranet or extranet-delivered services and content
    • Communications programs (internal or external)
    • Web based client relationship management
  • What worked well last year may no longer be ideal
    • Input may "happen our way," indicating that improvements are needed
    • But it would be risky to wait for – or rely solely on such input!
    • We need to engage in planned and well-executed audits
  • "Audit" is not a scary word
    • But if it seems to be, just choose another name!
      • Strategic Planning Review
      • Resource Assessment
      • Business Process Planning
      • Communications Checkup
    • Translation: Regularly repeated, s ystematic examination of current practices and future needs with the focus on the client
  • The Audit is supplemented by ongoing monitoring:
    • Mechanisms to catch evidence of:
      • Shifts in client needs due to changes in internal operations, changes in their target markets, and developments in industries in which they operate
      • Opportunities created through the application of advanced technology
    • The objective is to put in place a "360 °" system so as never to be caught off guard
  • The methods we choose to employ in conducting the Audit depend on:
    • Existing client relationships
    • Corporate culture
    • Time and money tradeoffs
      • Benefits of a comprehensive effort
      • Options for a phased-approach
    • Skill sets of staff involved
    • Available time frame
    • Complexity of what is being examined
  • Key consideration
    • How can we engage informants in a non-burdensome way?
    • How do we choose the right method for each target user group?
  • Tell-it-to-me … or Easter Eggs?
    • Common techniques involve a mix:
      • Interviews
      • Discussion (focus) groups
      • Surveys
      • Spot checks
    • But what people say isn’t always “the goods”
    • We may need more “evidentiary” methods
      • Logs (behavior trails)
      • Stats (behavior trends)
      • Observed behavior as-it-happens
  • All efforts are designed to help us:
    • Understand where we need / want / are able to go in order to meet evolving client needs
    • Build a business case for new or redirected efforts/investments
    • Create a plan for how we get there from here
  • Focus for today
    • Designing a process that will yield maximum insight into client perceptions and priorities
      • Involving the least effort - for us and for our clients
      • Resulting in the greatest degree of confidence in the findings
  • Case example 1
    • The Corporate Library’s Intranet presence is not cutting it!
    • It may once have been the cat’s meow, but time has gone by and “grafting” has resulted in a difficult-to-use site
  • We know some challenges, based on symptoms
    • Usage has dropped off significantly
    • Users are confused
    • Navigation is not intuitive
    • Users "miss" important announcements and new sources
    • Users go elsewhere for information
  • But we need to dig deeper
    • Our audit must not only point toward solutions for known concerns (for staff and users), but be designed to uncover details that may be "hiding" from us
  • What are the elements to be examined?
    • What specific "defects" must we address?
    • What content and services would be priority offerings in the eyes of our clientele?
    • What functionality do clients consider essential vs. "nice to have"?
    • Any "sacred cows" we can drop?
    • What are "good" library intranets doing that we aren’t?
      • How are those intranets maintained and by whom?
    • Are there good models we could use for inspiration?
  • Audit design preamble
    • Balance what we already know and what we don't yet know - avoid skewing results
    • Who can tell us? Who should find out?
      • Who on our staff should be involved in the process?
      • What outside assistance will we need?
  • Typical process
    • Review documented (web logs) & anecdotal evidence, and any marketing collateral distributed
    • Identify "regular user" / fan base informants
    • Identify "non-user who should be user" informants
    • Identify individuals others view as role models
    • Devise interaction structures
      • Two-on-one interviews
      • Discussion/focus groups
    • Devise direct observation events
    • Test interview/survey instruments
    • Set up logistics
  • A note on communication
    • Must "ace" the invitation: WIIFM
    • Explain the why
    • If possible, make the effort fun, not arduous
    • Accommodate participants' schedules
    • Gently remind
    • Demonstrate later that input got results
  • Tell me … or may I watch?
    • Common techniques involve questions
    • But what people say isn’t always "the goods"
    • We may need other methods - such as observation of what people actually do in a specific situation
  • "Easter Eggs"
    • A specific task answerable through the intranet site or other vehicle being examined
    • Forcing respondents to not just say “looks nice” but to dive in and answer a specific question
    • What path used to solve question?
    • Where found answer - or abandoned?
  • At-the-elbow "Process Deconstruction"
    • An attempt to trace a logical decision path
    • "Show me a typical task you need to accomplish"
    • What do you do first? Why?
    • Stop – why did you look there ?
    • Stop – then what made you decide to look here ?
    • Why did you not check here first?
  • Benefits of observation
    • Discovery of "hidden knowledge" not documented and thus unavailable to new users
    • Reality check to guard against owners' sense that "it's obvious"
    • Insights useful in further communication and design
  • Case example 2
    • A Library Consortium’s Communications Program is not as effective as it could be
    • Too many vehicles are employed
    • Many services are under-utilized
  • Project design
    • Initial orientation meeting
    • Survey of members
    • Focus groups
    • Needs assessment and usability-lite test for the Consortium’s website
    • Audit and benchmarking
    • Synthesis of data
  • Survey
    • Identified survey content, designed survey, and coordinated Consortium’s review of draft survey
    • Identified survey pretest participants, completed survey pretest, and revised questionnaire
    • Developed and implemented Web-based survey
      • Addressed security & privacy issues by observing professional protocols for information collection
      • Attained buy-in and announced survey
    • Hosted and monitored Web-based collection tools and systems
    • Monitored survey completion and followed-up
    • Analyzed data
      • Overall (Example)
      • Type of library (Example)
      • Portraits of _____ Library (Example)
  • Focus groups
    • Obtain information and clarification on:
      • Which issues are a priority
      • What features are important
      • Extent to which members will have influence
      • Ways to measure success
      • Role of interactivity
    • Conduct exercises to discern:
      • Awareness of the services offered to and valued by the participants
      • Adequacy of communication about those services
      • Availability and accessibility of services
      • Opportunities for improvement
    • Present results (to organization, participants, and members)
  • "Usability-lite" test of the Website
    • Does our website succeed in communicating clearly? Are users having trouble?
    • Usability-lite tests will help you determine:
      • Actuals / Optimals
      • Drivers / Incentives
      • Barriers / Potential solutions
    • A combination of telephone interviews (for pre-screening candidates) and in-person interviews in the participants’ normal work environment
      • Ask participants to “think aloud” as they explore the website
      • Personae and specific situations
      • Ask some follow-up questions
  • Audit and benchmarking: Objective website review
    • Reviewed the site from a member’s perspective vis-à-vis stated goals
    • Analyzed extant data (e.g. logfile data)
    • Evaluated typical navigation
    • Examined usability/human factors
    • Assessed the calls to action and flow of copy
    • Identified interactive techniques
    • Provided recommendations for:
      • Navigation, technical, and usability functions
      • Marketing copy
      • Interactive techniques
      • Access to other information systems and services
  • The result: User- and usage-centered design
  • Before…
  • After…
  • Tips for interviews and focus groups
    • Explain if/how an audio/video recording of the discussion will be utilized
    • Have a set of questions but be flexible
    • Assure complete confidentiality
    • Let them talk
    • Be comfortable with silence, but ready to "prime the pump“
    • Elicit elaboration by validating: Interesting, you are not the first to say so
    • Use "others-find" technique (you too?)
    • Be aware of interpersonal dynamics and politics
    • Recognize that some may not want to "look bad" & may tailor comments to what is thought "correct"
  • Tips for surveys
    • Short - Fast – Easy – did we mention short !
    • Clear, unambiguous questions in sections that flow
    • Ranking of personal priorities (What means more to you?) vs. "easy middle choice"
    • "How much do you love us on a 1-10" yields less valuable insight (no one wants to offend)
    • Minimize the number of open-ended questions
    • Do you agree with these statements made by your peers?
    • People shouldn’t agonize over responses, so provide an "out" using Don’t know/Not sure options
  • Tips for the direct-observation portion of a web usability test
    • Explain that the findings from the evaluation will be used "for good purpose"- no need to be polite
    • Explain that you will be collecting data by taking written notes
    • Stress that the website is being tested, they aren't
    • Remind interviewees to articulate their thoughts
    • Stay neutral
    • Help users in distress
    • Ask if they have any questions before the interview begins
  • What do we have in hand when it's all done?
    • Ideally, the elements for our business case
    • The justification for action and investment
    • The credibility that "the users spoke"
  • Audit Report sets out ( in one place )…
    • Drivers for the Audit (why did we)
    • Goals (what looked for)
    • Methodology / Informants (how did we)
    • Findings (broken down by major topic area) – factual, dispassionate
    • Conclusions that take the findings and group them into themes
    • Recommendations flowing from the findings (not just a crazy idea)
    • Business Case that includes a “how we will” implement change & risks of not implementing
  • Some of what we uncovered may be similar to your own experiences
    • Overall communications strategy: NSLS is responsive to member requests, but with some interesting effects
    • Target market: Probability of messages being opened, read, and acted upon increases dramatically as the targeting becomes more precise
    • Design/layout: Members normally find what they need on the NSLS website, but not easily, and they often forget where pages used in the past are located
    • Orientation/navigation: Consistency and clarity
  • Some of what we uncovered may be similar to your own experiences (con’t)
    • Search: Search functionality provided by local Google and sitemap
    • Content: Targeted content; relationships established; sourcing of content
    • Networking: Member "control" over physical and virtual (listservs vs. CoPs)
    • Continuing education: Professional development opportunities with series of courses for career paths
  • Access to technology
    • Issues surrounding technology focused on convenience in accessing NSLS services rather than not having access to required technology to make full use of them
    • Self-described as not being “explorers,” merely accepting what is on the screen in the way in which it appears
    • Preference for accessing NSLS while working, and not at home (nights or weekends)
  • Moving from findings and conclusions to recommended action
    • Document findings
      • Known challenges and potential solutions
      • Existing technical barriers and other constraints
    • Synthesize/interpret findings to draw conclusions
      • What works
      • Areas for improvement
      • Opportunities for growth
    • Make recommendations
      • Implement immediately
      • Implement in the near-term
      • Longer-term initiatives
  • What have we learned today?
    • Why organizations should conduct audits
    • Options for soliciting client input
    • The importance of communicating with our clients (and especially those who give time to the audit process)
    • Alternative ways to say “no”
    • Evolution, not revolution
      • No surprises
  • Questions?
  • Thank You … feel free to be in touch!
    • Barbie Keiser
    • [email_address]