EnerNOC is a leading provider of energy intelligence software and demand response solutions. It has over 1,300 employees globally serving over 6,000 companies. EnerNOC is increasingly moving its infrastructure and applications to AWS to take advantage of cloud capabilities. It uses Splunk extensively to gain visibility into its complex and distributed AWS environment for monitoring, alerting, and troubleshooting. The Splunk App for AWS integrates data from various AWS services to provide a single pane of glass view into AWS activity, configurations, costs, and security.

1. Splunk for Visibility into
AWS
Jim Nichols, Principal Engineer - Cloud Architect and Splunk
Champion at EnerNOC

2. EnerNOC at a Glance
• A leading provider of SaaS-based Energy
Intelligence Software (EIS) and related
solutions
• Market leader in demand response (DR)
• Global company (over 1,300 employees in
countries across North America, APAC,
Europe) with HQ in Boston, MA

3. EnerNOC’s Energy Intelligence Software
For enterprises: platform and solutions focus on the 3 drivers of energy expense
How you buy it
Budgets and Procurement
Utility Bill Management (UBM)
How much you use
Visibility and Reporting
Facility Optimization
Project Tracking
When you use it
Demand Response
Demand Management

4. Increase Use of Enterprise Energy Intelligence
Software
More than 70,000 sites and devices
stream data into EnerNOC’s energy
intelligence software platform
Over 6,000 companies globally
rely on EnerNOC to drive energy savings
EnerNOC Is Transforming Energy Management Across Industries

5. Increase Use of Enterprise Energy Intelligence
Software
Demonstrated expertise trusted by the largest companies in the world:

6. Utility and Grid Operator Partnerships
EnerNOC has Extensive Expertise Working With Utilities and Grid Operators Globally
We also operate in wholesale markets:

7. The Case for Demand Response
Balancing supply and demand on the electricity grid is difficult and expensive. End users that provide
a balancing resource are compensated for the service.
Annual Electricity Demand As a Percent of Available Capacity
50%
100%
Winter Spring Summer Fall
75%
25%
90%

8. EnerNOC EIS Platform
Data Streaming and Processing at Scale
Cloud Hosted Web
& Application
Servers
EnerNOC Hosted
Web & Application
Servers
4000/1000 Users
Daily/Hourly
100 Countries
EnerNOC
Database Servers
Cloud Hosted
Database Servers
32TB Persisted

9. About Jim
Multiple DevOps Roles @ EnerNOC since 2007
• Performance (site reliability), Architecture, Operations, Development…
• Charted with establishing Cloud Operations team in 2014
#1 Goal – Make sure production is up, running fast, and reliable
Brought Splunk in to look at web logs in March 2012
• Now have ~150+ indexes, 100GB/day, 1000’s of forwarders
EnerNOC’s Splunk Champion

10. Energy Intelligence in the Cloud
Great success with fast data on-premises: Oracle, Stored Procs, 3-tier Java
Strong business requirements for Big Data and Analytics, Rapid Development
Major shift in strategy 18-24 months ago: All-in with AWS
• All new development in AWS
• Migrating technology of acquisitions into AWS
• Moving on premises workloads to the cloud
Efficiently made the pivot
• Start of 2015: 30% in AWS, 70% in Data Center(s)
• Start of 2016: 80% in AWS, 20% in Data Center(s)
• Goal 2017: 95% AWS

11. Our AWS Environment
• AWS Usage
• Diverse Technology: MongoDB and Cassandra
12-25 AWS
Accounts
All public
AWS regions
1000+ EC2
Instances
150+ Engineers in 7+
Countries in AWS
90+ VPCs

12. Monitoring the Transition
Bootstrap Cloud Operations team:
• Basic Operations
• Architecture and Design
• Security
• Operational Best Practices
High degree of parallel activity
“Long” time Splunk customer (5+ years)
• DevOps
• Alerting and Monitoring
• Fast and real-time data processing
• Insight into many disparate systems in once place

13. Opportunity for Splunk
Leverage our mature
Splunk practice in use
for production DevOps
Bring data from all
accounts to a single,
secure location
Enrich the application log
data with AWS specific
data (CloudWatch)
Add additional layer of security
on top of what AWS provides
Application specific
dashboards with AWS data
Centralized alerting on
AWS specific metrics

14. Splunk App for AWS
Splunk App for AWS v3, and then v4
Splunk IAM user in all accounts with limited read-only permissions
AWS CloudTrail in all regions
Setup each account, then inputs
• Just pulled in CloudTrail, billing and a few s3 buckets (ELB logs, other data)
• Since v4, pull in AWS Config, CloudWatch, S3 ELB
Start slowly, add accounts and inputs over time

15. Splunk in Action
• Started of looking at the dashboards, get a feel for how the environment looks
• Enable included alerts, or created own based off searches from dashboard panels

16. IAM Examples
IAM Unauthorized and Detailed Activity
• Monitor for Unauthorized IAM Actions
• For malicious activity
• Non-conformant activity
• Build tool errors
IAM Access Key Create/Update/Delete
• Initially to avoid production outages and monitor key creation
• Alert now allows us to enforce best practice of using IAM Instance Roles

20. Operational Examples
Develop insight into the AWS environment:
• What changes do we typically see?
• Who are the active users/keys/IAM roles?
Troubleshooting:
• Build Errors
• Capacity Limits
• AWS system errors

24. Next Steps
• Get in touch – jnichols@enernoc.com, splunk-admins@enernoc.com, or LinkedIn
• Boston Splunk Users Group

25. Securing your data on AWS

26. $6.53M 56% 70%
Your data and IP are your most valuable assets
https://www.csid.com/resources/stats/data-breaches/
Increase in theft of hard
intellectual property
http://www.pwc.com/gx/en/issues/cyber-
security/information-security-survey.html
Of consumers indicated
they’d avoid businesses
following a security breach
https://www.csid.com/resources/stats/data-
breaches/
Average cost of a
data breach

27. In June 2015, IDC released a report which found that most customers can be more secure
in AWS than their on-premises environment. How?
AWS can be more secure than your existing environment
Automating logging
and monitoring
Simplifying
resource access
Making it easy to
encrypt properly
Enforcing strong
authentication

28. AWS Foundation Services
Compute Storage Database Networking
AWS Global Infrastructure
Regions
Availability Zones
Edge Locations
Identity &
Access Control
Network
Security
Customer applications & content
You get to
define your
controls IN
the Cloud
AWS takes
care of the
security OF
the Cloud
You
Inventory
& Config
Data
Encryption
AWS and you share responsibility for security

29. Key AWS Certifications and Assurance Programs

30. The AWS infrastructure is protected by extensive network and security
monitoring systems:
• Network access is monitored by AWS
security managers daily
• AWS CloudTrail lets you monitor
and record all API calls
• Amazon Inspector automatically assesses
applications for vulnerabilities
Constantly monitored

31. The AWS infrastructure footprint protects your data from costly downtime
• 33 Availability Zones in 12 regions for
multi-synchronous geographic redundancy
• Retain control of where your data resides
for compliance with regulatory requirements
• Mitigate the risk of DDoS attacks using
services like AutoScaling, Route 53
Highly available

32. AWS enables you to improve your security using many of your existing
tools and practices
• Integrate your existing Active Directory
• Use dedicated connections as a secure,
low-latency extension of your data center
• Provide and manage your own encryption
keys if you choose
Integrated with your existing resources

33. Security Requires Visibility:
Splunk App for AWS
Praveen Rangnath
Sr. Dir. of Cloud Product Marketing

34. Make machine data accessible,
usable and valuable to everyone.

35. Proven at 11,000+ Customers in 110+ Countries
More Than 80 of the Fortune 100

36. Splunk’s AWS Credentials
• AWS Advanced Technology Partner
• AWS Big Data competency
• AWS Security competency
• AWS MSP Technology Provider
• AWS Marketplace Partner (BYOL provider)
• AWS IoT Launch partner for IoT analytics
• AWS Security by Design Program Partner
• 1st partner with published Blueprints for AWS Lambda
• 1st partner to pass SaaS extension for Well Architected framework

37. EC2
EMR
Kinesis
R53
VPC
ELB
S3
CloudFron
t
CloudTrail
CloudWatch
Redshift
SNS
API Gateway
Config
RDS
CF
IAM
Lambda
Explore Analyze Dashboard Alert Act
AWS Data Sources
Customer Requirement: End-to-End AWS Visibility
Splunk App for AWS

38. Splunk App for AWS: The Data
AWS CloudTrail
• Service that delivers logs of admin activity
on AWS infrastructure
• Examples:
• Start/Stop/Create instance
• Change of User roles/rights
• Modification of Network Configuration
• Delivers log files to customers; no UI,
display, analysis, search
AWS Config
• Provides resource inventory
• Provides configuration history & change
information
• Enables security & governance
Amazon Cloudwatch Metrics
• IP traffic information to/from VPC network interfaces
• Data stored and accessible from Amazon
CloudWatch Logs
Amazon Cloudwatch VPC Flow Logs
• IP traffic information to/from VPC network interfaces
• Data stored and accessible from Amazon
CloudWatch Logs
AWS Access Logs
• Elastic Load Balancing (ELB)
• Cloudfront CDN
• S3
AWS Billing
• Current Month via CloudWatch metrics
• Monthly Detailed Billing

39. Splunk App for AWS: What Customers Gain
In-depth visibility into all
activity in an AWS account
Real-time monitoring of
AWS resource changes
Full audit-trail reporting
across multiple AWS
accounts and regions
Ability to prove both
real-time and historical
compliance
Visual representation of
the AWS Topology and
resource relationships
Monitoring of VPC
traffic utilization for
security insights
Dashboards showing
historical and real-time
views/comparisons
Real-time visibility into
AWS billing

40. Splunk App for AWS - Overview
• The overview page shows you on
one screen information about:
• Configuration changes
• Compute
• Storage
• Billing
• ELB
• Cloudfront
• Security
• Notable CloudTrail Activity is
highlighted on the map.
• Drill down on any event and gain
detailed information.

41. AWS Topology
• Topology view gives you a holistic view
of your current or historical AWS
deployment using AWS Config
• Maps out relationships between all the
components, giving you a clear view into
the environment
• Clickable layers adds additional visual
queues for high CPU or network traffic
• Snapshot feature allows for topology to
be saved for future use

42. AWS Usage
• In one glance, instantly see your
EC2 usage and EBS Volume data
info via CloudWatch metrics
• Click through dashboards for details
on individual EC2 instances and
EBS Volumes
• Drill down into raw search for even
more detailed views on your
instances.

43. VPC Flow Data - Security
• Utilizes VPC Flow Logs from
CloudWatch for Security Analysis
• Drill down into rejected vs. accepted
traffic
• View top Source Country and City
information
• See top source / destination and IP
addresses and ports

44. AWS Billing & Capacity Planning
• Utilizes Billing logs from
CloudWatch for Month-to-Date
billing and End-of-Month
projections
• Detailed Historical Billing
Dashboard available using Monthly
AWS Detailed billing reports
• Capacity Planner gives additional
clarity on AWS On-Demand
instance spending

45. Index Untapped Data: Any Source, Type, Volume
Online
Services Web
Services
Servers
Security GPS
Locatio
n
Storage
Desktops
Networks
Messaging
Telecoms
Online
Shopping
Cart
Web
Clickstream
s
Databases
Energy
Meters
RFID
On-
Premises
Private
Cloud
Public
Cloud
End-to-End Visibility
Application Delivery
Security, Compliance
and Fraud
IT Operations
Business Analytics
Industrial Data and
the Internet of Things
CloudTrail
Config
Lambda
EC2
True Goal: Complete Infrastructure Visibility

46. Portfolio of Cloud Solutions
100% Uptime SLA
SOC2 Type II certified
ISO 27001 certified
Apps for Cloud Services
AWS, ServiceNow,
Akamai, SFDC …
AWS Integrations
AWS Lambda: First partner blueprint
AWS IoT: Featured analytics platform
ES released as software
and SaaS
Enterprise Security
ITSI released as
software and SaaS
IT Service Intelligence

47. Easy to Get Started for FREE
Splunk App for AWS