2. 2
Table of Contents
Table of Contents ........................................................................................................................................ 2
Technology Introduction........................................................................................................................... 3
How it works.................................................................................................................................................. 3
Auditing............................................................................................................................................................ 3
Simulations...................................................................................................................................................... 4
ObservePoint Best Practices.................................................................................................................... 5
Promote the Data Quality Owner......................................................................................................... 5
Define Governance for Data Quality ................................................................................................... 6
Deploy a Data Quality Assurance Platform...................................................................................... 7
Audit Data Collection Regularly ............................................................................................................ 8
Audit data collection in QA, staging and production.................................................................. 9
Set Proactive Alerts...................................................................................................................................11
The Data Quality Management Process...........................................................................................12
Confirm Complex Integrations.............................................................................................................14
Leverage Audits for Documentation .................................................................................................15
Prevent Data Leakage..............................................................................................................................16
Conclusion.....................................................................................................................................................17
Appendix A...................................................................................................................................................18
3. 3
Technology Introduction
How it works
The ObservePoint technology was built to mimic a living site visitor as he or
she browses through your website. ObservePoint’s Auditing and Simulation
system work by crawling specified sites and aggregate data into the
ObservePoint interface. ObservePoint Audits have the ability to use
simultaneous connections to quickly audit your site in which the user can
control.
ObservePoint also provides a technology called Silent Mode, which allows
Audits and Simulations to run undetected within measurement platforms, such
as Adobe Analytics, Webtrends or Google Analytics, by intercepting the
network request. This will prevent ObservePoint page views and visits causing
data inflation. Silent Mode can be enabled/disabled by the user on a per
Audit or Simulation basis. As a best practice, it is recommended that this
technology be used when the ObservePoint IP Range has not been excluded
or filtered from measurement platforms.
Auditing
Audits are used to measuring data quality. Audits improve the accuracy and
completeness of web analytics data, testing and optimization, advertising, tag
management, video measurement, voice of the customer, etc.; ensuring
decisions are made with truthful data.
4. 4
• Audits are performed on live websites (production and pre-production);
no implementation is required
• All tags on the web site are identified and catalogued by page
• Reports show tag presence, allowing you to see pages missing tags
• Each of the tag's variables and each variable’s parameters are shown
• Audits often expose superfluous or old tags that should be removed or
updated
• Exposes JavaScript errors that prevent the browser from executing tags
• Syntax checks identify truncation of values or dropping tag data
• Analyze the network requests to identify the order in which tags execute
• Detect duplicate or multiple image requests that result in data inflation
• Measure page load time performance of each page and tag
• Find broken links, redirects, and internal server errors
• Receive an alert when tag configuration is changed
Simulations
Simulations are interactions with focused, high-value web content to simulate
a human visitor. Simulations frequently confirm site functionality and detect
tagging errors.
Confirms tag presence before, during, and after updates
Monitors critical content on a regular basis
Test complex flows in sequence to ensure accurate linkage and
functionality
5. 5
Ensures heartbeat tags in video content are functioning correctly
Alerts teams via email and/or SMS when an error is detected
ObservePoint Best Practices
Promote the Data Quality Owner
The first step towards data quality is to assign ownership to data quality. A
data quality owner should have the correct background and authority to
influence organizational practices.
Important background for the data quality owner:
Strong Technical Background: Competency with the technology and
be familiar with the different platforms each digital property is built
upon
Seniority: A person with context, influence, and trust improves the
odds you’ll have a better marketing technology ROI
Understand Business Value of Data: Understand and create dialogue
about the requirements, purposes, and methods of data collection
Data Orientation: A data-oriented steward understands the effects of
breakage, knows where to look for problems, and will choose effective
corrective actions
6. 6
Define Governance for Data Quality
Data collection and governance best practices revolve around three key
processes:
New Tag Deployments: Create clear, consistent business processes for
adding data collection. Integrate with development and content
deployment cycles.
Verify Tags: Confirm variables are properly populating and code
functions as expected under all conditions. Ensure data collection is as
consistent as possible across browsers, devices, geographies, and does
not hurt the user experience.
Conduct Audits: Deploy a process to confirm that tag deployments are
maintained over time. Websites have enough moving parts and enough
people touching code that tagging problems are common and
frequent.
Tag auditing solutions are designed to improve accuracy and mitigate the
challenges associated with tag-sourced data. Tag Management Systems are
terrific tool for managing tag deployments, but they do not reduce the need
for Quality Assurance processes.
7. 7
Deploy a Data Quality Assurance Platform
Data Quality Assurance offers solution-specific data confirmation, complex tag
verification, and proactive alerting for all digital marketing channels and
content types.
Data Quality Matters: Web Analytics, Advertising, Testing tools, Data
Management Platforms and Tag Management Systems are only as
good as the data they process. Data Quality Assurance confirms that
the source of this data – the tags on your website, in your video, and in
your apps – are deployed correctly and completely.
Measure and Improve Data Quality: A solid Data Quality Assurance
process evolves and is maintained over time to accommodate and
capture changes in real-time, ensuring accurate data collection; the
value of which is immeasurable considering the impact of decisions
made based on your digital data. Regular and consistent monitoring of
your digital data are hallmarks of a successful process.
Improve ROI in All Digital: Consistently complete, correct, and
compliant data improves the ROI of your entire digital marketing
technology stack.
8. 8
Audit Data Collection Regularly
Launching scheduled site audits ensures that validation of your data collection
environment becomes part of your organization’s digital DNA. Start with daily,
weekly, and monthly audits and then larger quarterly discovery audits.
Daily/Weekly “difference” audits detect what has changed: Daily
audits are key in detecting variable anomalies on your key pages. Most
websites change on a continuous basis. Your tagging should keep up
with the change, and this means that you should be testing early and
often. For highly effective “difference” audits, only audit those pages
that have changed. Simply supply ObservePoint a list of changed URLs
to confirm they are tagged properly.
Monthly/Quarterly Discovery Audits validate baseline data quality:
Monthly/Quarterly Discovery Audits confirm data collection is occurring
as planned and creates lists to repair the problems quickly. Regular site
audits promote visibility into data pollution by providing validation and
documentation necessary to avoid tagging problems. Larger audits help
to identify systemic problems such as bad code versions and patterns
of data leakage.
9. 9
Audit data collection in QA, staging
and production
Website content is constantly added, changed, and retired, and this creates
tracking challenges. Data Quality Assurance helps you manage these changes
while still being productive with your other duties.
“We have 40 or 50 developers dedicated to the website and we roll code
twice a week. [Before ObservePoint], we didn’t have an easy process to
validate page tagging on such a frequent basis. I needed to hand check new
or altered pages with each roll-out, which could easily lead to missed errors in
the tags.” —Tim Munsell, lead Web Analyst at DaveRamsey.com
A threefold auditing strategy is prescribed for all new content:
1. Test frequently during the development process — Audit multiple times
during any large-scale deployment effort. This instills data collection
best practices throughout your company. Share the results with
developers and project managers when appropriate.
2. Test often during quality assurance and testing phases — Closely
monitor the data being passed to your analytical and other systems (i.e.
optimization systems) after your development project has passed into
your QA environment.
3. Test immediately following deployment to confirm data is being
collected as planned — Confirm data collection has successfully
migrated from the development and staging environments to the
production environment. When problems are identified in this phase,
act quickly to correct them. Assign resources to post-deployment
efforts tasked with ensuring data collection is correct.
10. 10
4. Continue to monitor critical content on a high-frequency basis —
Changes often occur without your knowledge. Capturing and
understanding these changes as they occur can prevent long periods of
inaccurate data collection.
The Pay-off
The deployment process is not considered complete until all aspects of the
code are deemed functional in a production environment including data
collection code. Commitment to data accuracy and utility within the broader
enterprise ensures quality reporting and limits the occurrence of data
anomalies and the need to explain them.
11. 11
Set Proactive Alerts
Breakage of web measurement code is harmful to business processes and
early detection is challenging. Problems can go unseen until the forest fire is
raging unless you’ve configured a proactive alerting system. Proactive alerting
helps you detect tagging and site errors before they become data disasters.
What is a proactive alerting system? Proactive alerting is a continuous
background testing process that verifies the deployment and function of tags.
ObservePoint Data Quality Assurance verifies the functionality of tags by
interacting with the site, triggering events as an average user would. Tests
embody a typical user path or behavior. When a test fails, you’re notified
immediately.
Proactive, Powerful and Practical: When tagging on any page fails or is out
of compliance with the test criteria, ObservePoint sends alerts.
End Result: With proactive alerts, ObservePoint becomes a key part of
measuring and improving data quality.
12. 12
The Data Quality Management Process
Analyzing audit and simulation data is where you will make data quality
improvement recommendations, and confirm changes. There are two modes
in the data quality assurance process.
Agile Mode
In agile mode, you are cleaning up tagging issues that have already
happened. It is a necessary task, but you will see immediate benefits. Your
Data Quality Assurance program helps you get faster results that are more
accurate and allows you to quickly take corrective action.
Proactive Mode
In proactive mode, you are preventing future tagging issues. This happens
through regularly scheduled audits and simulations in established core
content areas, ad-hoc audits in beta environments, and simulations in
coordination with campaigns.
Recognize Triggers
The first step to improving data quality is recognizing the signs of poor data
before anyone else does. These triggers are your signal to perform quality
assurance. Triggers come in two types: external and internal.
External triggers are items you do not have control or insight into but
impact digital data. Examples are website code updates, a question about
data quality from a coworker, or a breach. Agility in responding to external
triggers is key to maintaining data quality.
Internal triggers are items in which you maintain control and insight; such as
the deployment of new tags or a planned launch of new web content. Internal
13. 13
triggers allow you to be proactive and set baseline quality assurance
standards and supporting processes.
Target the Right Content
You must focus your QA efforts on the right content. It is vital to know the
priorities of your business and how those are mapped to your measurement.
Identifying whether you are acting based on internal or external triggers helps
you focus your efforts and configure your tools to get the most value with
the least amount of work.
Make Impactful Recommendations
After audit results are generated, you must analyze the data. This might mean
regression testing, in-context QA, or going back to documentation. Your job
is to communicate clearly and efficiently to stakeholders what needs to be
repaired.
Ideally, you will combine proactive and agile methods of data quality
assurance. Where you have control and a steady baseline, regularly scheduled
audits detect anomalies that can creep in leaving you to deal with external
triggers, which often incur additional cost and overhead. ObservePoint Data
Quality Management makes all of this faster and more accurate than any
other solution.
14. 14
Confirm Complex Integrations
Digital marketers have become accustomed to complex, dynamic technology
integrations that create data in specific ways based on visitor behavior.
Testing the proper functionality of this tooling has historically been a
laborious process requiring specialized browser-based tools, careful
interpretation of raw data, and manual use of excel files. By using the
ObservePoint technologies, you allow yourself to verify complex integrations
within an automated background process.
15. 15
Leverage Audits for Documentation
Best practices suggest a regular review of your Solution Design Document
(SDD), revising it as necessary, to maintain an accurate reflection of what data
is being collected. Technical specifications supporting the SDD should also be
maintained to ensure the data is being collected in the proper manner.
Although a great deal of time is spent creating these documents about data
collection and technology integrations, in many cases, they are seldom
reviewed and revised. Meanwhile online properties change rapidly and often.
As a result, when someone has a valid question about why or how data is
being collected there is little real, reliable documentation available.
Leverage Audits for Documentation
ObservePoint Data Quality Assurance generates documentation about the
current state of digital integrations. According to Tim Munsell, lead Web
Analyst at DaveRamsey.com, “The report we get from ObservePoint is a more
useful and accurate reflection of our implementation than our original
documentation. We are able to export our audit to Excel and essentially work
backwards, making sure that the data collected on each page on our site
matches our business requirements.”
An audit report is a true-to-life assessment of what tags are actually doing on
your site. Data exports from ObservePoint are extremely dynamic and have a
variety of uses to address specific needs faster than an SDD can be generated
or updated.
16. 16
Prevent Data Leakage
Data Leakage is a growing problem within the industry. Data Leakage, put
simply, is the unauthorized transmission of data (or information) from within
an organization to an internal or external destination or recipient. Deploying
data collection systems without a clear plan for maintaining accuracy, validity,
and security of data collected is undoubtedly a poor practice. ObservePoint
detects hundreds of technologies, many of which you may be unaware of.
With this data, you can reduce the risks of data leakage.
Consider these data leakage scenarios:
1. An employee leaves the company to work for a direct competitor but
maintains access to traffic and revenue data through unknown deployment of
analytical tools.
2. An agency deploys tools with questionable PII collection and handling
practices.
3. A Tag Management System (TMS) is deployed in an effort to
consolidate the chain of authority for controlling site tags, but others
circumvent this process and deploy tags outside of the TMS to meet their
own needs.
4. A third-party vendor deploys tracking that sells data to other third
parties, potentially exposing your data to direct competitors.
5. A policy bans particular technologies on all sites, yet these tags
continue to appear.
ObservePoint will expose any third party vendors for you to validate internally
and, once identified, allow you to deal with them accordingly.
17. 17
Conclusion
Digital data is a treasure trove of information for conducting business online.
Effectively leveraging this data requires it to be complete, accurate, and truly
reflective of visitor behavior. Using ObservePoint technologies, you will begin
to formulate your own Digital Quality Assurance program that fits the needs
of your online business. To summarize the necessary parts of your Digital
Quality Assurance program you will need:
A team or individual with the technical acumen and internal authority
to research, execute, and promote a Digital Quality Assurance program
Keep data pure by defining processes for implementing new tags,
verifying existing tags, and maintaining clean data by running constant
audits and simulations.
Begin your Digital Quality Assurance program by performing a large
Discovery Audit to get a baseline. Use smaller high-frequency audits
(between 100-1000 pages) to verify the data quality of your site
templates, high value pages and critical user paths.
Audit development, QA and staging environments before production
launches with audits and monitoring after production pushes.
Set proactive alerts using Simulations within vital processes such as the
shopping cart, forms, and other critical paths.
Document the Data Quality Management process using the agile and
proactive modes and include internal and external triggers.
Leverage audit data to create and maintain your Solution Design
Document.
Prevent data leakage by verifying 3rd party tag access to your site and
have internal checks and balances for those that have access to site
data.
18. 18
Appendix A
Data Quality Assurance Checklist
Select a Digital Quality Assurance Owner
Define tag verification process
Implement the ObservePoint Platform
Filter the ObservePoint IP Range within your Digital Marketing
vendors and internal systems (development, QA, Staging, etc.)
See your Customer Success Manager for more information.
Set up a small 1000 page audit to find what pages, sub sections,
unrelated URLs & domains, etc., should be excluded from your
Audits.
Create a large Discovery Audit to look for wide spread tag
anomalies and other site issues. Schedule the audit monthly or
quarterly to verify overall site improvements and progression.
List your high value pages (including templates) to be audited on
a daily basis. Page quantity will be from 100-1000.
Set up audits on your QA and Staging environments to find
implementation issues before production launch. Run the audit
on the same production pages post-launch to ensure
implementation quality.
Set proactive alerts using ObservePoint Simulations on key
landing pages, and critical high-value paths (checkout process,
forms, site login, etc)
19. 19
Use Audits to verify and update site documentation for each tag that is
implemented.
Use ObservePoint technologies to find all third party tags and
technologies currently implemented and verify if they are legitimate as
well as validate the data the tags are collecting. This is also a good
time to confirm user access to site data, both internally and externally.
