1. . ...........................................................................................................
...................................................................................................... ................................................................................................................................
................................................................................................................................
................................................................................................................................
...................................................................................................................................................
..................................................................................
.................................................................................
CASE STUDY
THE GLOBAL LEADER IN CLOUD-BASED
SOFTWARE FOR THE UTILITY INDUSTRY
“Pepperdata ensures that our production Hadoop
environment remains reliable enough to handle all of
our diverse workloads so we can meet our SLAs with
confidence.”
- Eric Chang, Data Infrastructure Technology Lead, Opower
Opower is a leading provider of cloud-based software to the utility
industry and is transforming the way utilities relate to their customers.
By combining data management, insightful analytics, and behavioral
science, Opower’s customer engagement platform positions utilities
as trusted energy advisorvs to the customers they serve. To date,
the Opower platform has created enough energy savings through
behavior change to power all the homes in a city of 1 million
people for a year.
Opower uses Hadoop as the infrastructure foundation for its data
warehouse and its operational data store. The Software as a Service
(SaaS) company relies on Hadoop to deliver big data analytics to
its more than 95 utility partners, including 28 of the 50 largest
U.S. electric utilities, and over 50 million household and business
customers in nine countries. The multi-tenant production environment
runs mixed workloads that include Hive queries, MapReduce jobs,
and HBase serving data in real time to end users.
KEY CHALLENGES
Eric Chang, Data Infrastructure Technology Lead at Opower, recalls
the early days of their Hadoop deployment. “As with many initial
Hadoop deployments, stability was a big challenge for us.” Jobs
would compete with each other for physical resources. When cluster
capacity was exceeded, the Hadoop cluster would experience
cascading failures and critical applications would become unavailable.
Annual Revenue
$88.7mm (2013)
Business Needs
Big data platform to support cloud-
based analytics for more than 95
utility partners and their customers
Scalability to handle hundreds of
billions of data points from energy
meters, third-party data feeds, and
event data
Key Challenges
Maintaining stable environment to
predictably meet SLAs
Diagnosing performance problems in a
timely manner
Minimizing hardware expenses
Solution & Results
Pepperdata Supervisor real-time
cluster optimizer:
• Hive queries, HBase, and
MapReduce jobs run reliably on
the same multi-tenant production
cluster
• SLAs are met with confidence
• Increased throughput, fewer
hardware resources required

2. CASE STUDY
pepperdata.com
Chang and his team took a number of steps to
mitigate these issues through extensive capacity
planning, cluster tuning, prioritized job scheduling,
and the careful curation and testing of jobs before
release to production. In spite of adopting these best
practices, Chang saw a need to do more.
The volume of data from energy meters, third party
data feeds, and event data was growing rapidly. At
the same time, the number of users and diversity
of workloads continued to increase. In the face of
these challenges, meeting SLAs and cost-effectively
managing hardware expenses were of critical
importance.
MORE VISIBILITY, CONTROL, AND CAPACITY
Opower began using Pepperdata Supervisor in the
spring of 2014. Installation took less than an hour,
with no modifications to Opower’s schedulers,
workflow, or jobs.
Opower’s Hadoop administrators now have the ability
to monitor every facet of cluster performance in
real time. Visibility into CPU, memory, disk I/O, and
network usage by job, task, user, and group makes it
easier to proactively identify potential performance
problems before they occur and take preventive
actions.
When performance bottlenecks do occur, Opower’s
Hadoop administrators are able to quickly diagnose
and fix the problem. Troubleshooting activities that
used to take days are now typically completed in a
matter of minutes.
Opower is also using Pepperdata Supervisor to
dynamically adjust job resources to reflect their
service level priorities, ensuring that their clusters are
devoting sufficient resources to appropriate jobs. As a
result, critical jobs now run faster, more reliably, and
more efficiently on Opower’s existing servers, allowing
the infrastructure team to scale services with fewer
hardware resources.
Copyright © 2014 Pepperdata, Inc. All rights reserved. PEPPERDATA and the logo
are registered trademarks of Pepperdata, Inc. The other trademarks are trademarks of
Pepperdata, Inc.
Pepperdata’s products and services may be covered by U.S. Patent Nos. 8,706,798 and
8,849,891, as well as other patents that are pending.