SlideShare a Scribd company logo
1 of 31
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 1© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved.
Data Sciences Solution for APS
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 2
Goals and Objective
 Objective
 Identify a pattern of data that will more effectively identify smart
meters that have permanently stopped communicating vs
intermittent and temporary conditions i.e. power outage,
tampering, etc.
 Goal
 Eliminate the back-office manual investigation and 10 day delay
 Replace the meter before any billing impact
 Avoid replacing smart meters not communicating due to
temporary condition
 Allow field workers to increase SLA for the meter replacement
and uniform their workload
 Cost Benefits
 Potential reduction of back-office investigator by 1 FTE = $53K
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 3© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved.
Week#3 , Week#4 & Week#5
Hypothesis#1: Is same meter read patterns observed across meters?
Hypothesis#2: Is meter following a sequential pattern of events?
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 4
Proposed Timeline
Project Plan
Deliverables
Key Activities
Execution
Timeline
(week #)
• Due diligence, Determine
Business Objectives
• Assess As-is Situation(Process
Flow, Data Flow, Frequency
Distribution, Data Granularity)
• Mapping of events, reads and meter details file
• Integration of data sources by joining keys
• Data Exploration, Aggregation and Cleaning
• Missing value treatment, outlier treatment etc.
• Univariate/Bivariate analysis, Correlation and
Multi-collinearity check
• Data preprocessing for predicting meter change
• Building prediction model
• Model validation, build visualizations
• Provide recommendations and
present insights
• Project Plan Document
• Process Flow
• Initial Descriptive Analysis
document of Meter reads
• Exploratory Analysis document Meter reads
pattern, meter reads error code sequencing,
meter events sequencing patterns,
manufacturing details of the meter
• Model result
• Final presentation with insights
1st
Week
3rd
Week
2nd
Week
4th
Week
7th
Week
8th
Week
Business Understanding
and Data Gathering
Exploratory Data Analysis
and Data Preparation
Model Building and Insight
Generation
5th
Week
6th
Week
9th
Week
We are in 5th Week
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 5
Patterns in Register Reads of Residential Meters
Permanently Stopped Communicating(Unit of Measure - kwh)NormalizedMeterReadinginKwh
Register Reads for past 30 days from LCD
Register Reads for past 30 days from LCD
NormalizedMeterReadinginKwh
Meter reading
increases with time
Meter reading
increases with time
78%
Meters fall in this cluster
19%
Meters fall in this cluster
Meter reading constant
with time
3%
Meters fall in this cluster
Key Patterns:
 Positive Trend: 97% Meters(397 out of 410) that have stopped
communicating permanently show an increasing trend of normalized
meter reading
 Constant reading: 3% Meters(13 out of 410) that have stopped
communicating permanently show constant normalized meter
reading
Machine Learning algorithm:
 Time series clustering
Cluster#1 – Positive Trend Cluster#2 – Positive Trend
Cluster#2 – Constant Reading
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 6
Patterns in Interval Reads of Residential Meters
Permanently Stopped Communicating(Unit of Measure - Vrmsh)
NormalizedMeterReadinginVrmsh
Interval Reads for past 30 days from LCD
67%
Meters fall in this cluster
33%
Meters fall in this cluster
Key Patterns:
 Between 2.5 and -2.5: 67% Meters(279 out of
417) that have stopped communicating
permanently have normalized meter reading
fluctuating between -2.5 and 2.5.
 Between 2.5 to -2.5 & beyond: 33% Meters(138
out of 417) that have stopped communicating
permanently have normalized meter reading
fluctuating between 2.5 to -2.5 & beyond.
Machine Learning algorithm:
 Time series clustering
Cluster#1 – Value between 2.5 and -2.5
Cluster#1 – Value between 2.5 and -2.5 & beyond
Fluctuation Range ~5
Units
Reading with ‘0’ value
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 7
Patterns in Register Reads of Residential Meters
Communicating(Unit of Measure - kwh)
NormalizedMeterReadinginKwh
Register Reads for 60 days
Meter reading
increases with time
61%
Meters fall in this cluster
Meter reading constant
with time
39%
Meters fall in this cluster
Key Patterns:
 Constant Reading: 39% Meters(59 out of 151) that are still
communicating shows constant normalized meter reading.
 Positive Trend: 61% Meters(92 out of 151) that are still
communicating show an increasing trend of normalized meter
reading
Machine Learning algorithm:
 Time series clustering
Cluster#1 – Constant Reading
Cluster#2 – Positive Trend
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 8
Patterns in Interval Reads of Residential Meters
Communicating(Unit of Measure - Vrmsh)
NormalizedMeterReadinginVrmsh
Interval reads for 60 days
Key Patterns:
 Between 1.5 and -1.5: 26% Meters(39 out of 156) that have
stopped communicating permanently have normalized meter
reading fluctuating between 1.5 and -1.5.
 Between 1.5 to -1.5 & beyond: 77% Meters(117 out of 156)
that have stopped communicating permanently have
normalized meter reading fluctuating between 1.5 to -1.5 &
beyond.
Machine Learning algorithm:
 Time series clustering
Fluctuation Range ~ 3
Units
Cluster#1 – Value between 1.5 to -1.5
Cluster#1 – Value between 1.5 to -1.5 & beyond
Reading with ‘0’ value
25%
Meters fall in this cluster
75%
Meters fall in this cluster
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 9
Data Dictionary-Meter Events(Applied to both MC and Non MC)
Column Name Description Comments
action_code Meter Status MC for Meter Change and Non MC for Communicating meters
Cutoffdatetime Last Communication Date Used to analyze past patterns
mtr_evnt_log_id Unique value applied to each meter event record in UIQ Unique Values for all rows
ami_dvc_name
Unique identifier for devices in UIQ.
Manufacture code concatenated with a serial number
Identification of meters
mtr_evnt_id Numerical code representing the type of meter event. Key to join to the meter event type table to get the event description
mtr_evnt_tmstmp Timestamp representing when the event occurred Used for time sequencing of the events
sag_or_swell_cnt Sag or Swell Count Important event parameter
mtr_evnt_name Meter Event Description Used for differentiating the events. Details from meter event type table
Variables that would be used for analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 10
Data Dictionary-Meter Events(Applied to both MC and Non MC)
Column Name Description Comments
ami_proxy_dvc_name AMI Proxy Device Name Same as ami_dvc_name.
evnt_argus Event Arguments
Additional events details
evnt_txt
Event Text is a textual representation of the EVENT
ARGUMNETS
Additional events details
location_util_id Location Utility Id
service_pt_util_id Service Point Utility Id
mtr_evnt_insrt_tmstmp Timestamp representing when the event was recorded in UIQ Not relevant for the analysis
mtr_evnt_cnt Column is no longer used All rows are blank
etl_prcs_id
Unique identifier assigned to each ETL batch session
This unique identifier will be applied at the row level for every
insert or update.
Not relevant for the analysis
etl_prcs_type.
Attribute that will indicate the exact type of action that was
performed at the row level of the target table Insert (I), Update
(U), etc
Not relevant for the analysis
src_admin_st Ami Device Status
Values like Active, Disconnected, Inactive, Investigate,
Maintenance, New, Retired, Service Disconnect
src_operal_st Ami Device Network Status
Values like Active, Disconnected, Discovered, Inactive, Init
Failed, Initializing, Installed, Investigate, Maintenance, New,
Removed, Retired, Service Disconnect, Unreachable.
mtr_evnt_sevrty Meter Event Severity Details from meter event type table
mtr_evnt_ctgy Meter Event Category 1
Details from meter event type table
Variables that would NOT be used for analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 11
*Support **Confidence
Meter Events File Analysis – Meter Change ( Illustration )
{Last Gasp} => {NIC Power Restore} 0.480839416 0.981378026
{NIC Power Restore} => {Last Gasp} 0.480839416 0.524378109
{Last Gasp} => {NIC Power Down} 0.474452555 0.968342644
{NIC Power Down} => {Last Gasp} 0.474452555 0.594965675
{Last Gasp} => {NIC First Time Set After Boot} 0.474452555 0.968342644
{NIC First Time Set After Boot} => {Last Gasp} 0.474452555 0.539979232
{Last Gasp} => {NIC Power Restore Log Entry} 0.474452555 0.968342644
{NIC Power Restore Log Entry} => {Last Gasp} 0.474452555 0.539979232
Top event sequence based rules : For {Last Gasp}
Initial Observations:
 {Last Gasp} => {NIC Power Restore} are happening in Tandem ( confidence is 0.98)
 {NIC Power Restore} => {Last Gasp} : Cases where NIC Power Restore has happened previously – the likelihood of {Last Gasp} taking place in 0.52
 {Last Gasp} => {NIC Power Down} : - the likelihood of {Last Gasp} leading to {NIC Power Down} is 0.96
In this we have filtered the events
for {Last Gasp} and noted what are
the preceding / subsequent events
with occurrence likelihood
*Support : This says how frequent an ‘event’ is, as measured by the proportion of line items in which an event appears.
**Confidence: This says how likely event Y is to occur when another X has happened, expressed as {X -> Y}.
Machine Learning Algorithm:
 Apriori
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 12
Support Confidence
Meter Events File Analysis – Non Meter Change ( Illustration )
Top event sequence based rules : For {Last Gasp}
{Last Gasp} => {NIC Power Restore} 0.7195 0.9064
{NIC Power Restore} => {Last Gasp} 0.7195 0.7908
{Last Gasp} => {NIC First Time Set After Boot} 0.6732 0.8481
{NIC First Time Set After Boot} => {Last Gasp} 0.6732 0.8080
{Last Gasp} => {NIC Power Restore Log Entry} 0.6732 0.8481
{NIC Power Restore Log Entry} => {Last Gasp} 0.6732 0.8080
{Last Gasp,NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.6732 1.0000
{Last Gasp,NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.6732 1.0000
{NIC First Time Set After Boot,NIC Power Restore Log Entry} => {Last Gasp} 0.6732 0.8080
{Last Gasp,NIC First Time Set After Boot} => {NIC Power Restore} 0.6732 1.0000
{Last Gasp,NIC Power Restore} => {NIC First Time Set After Boot} 0.6732 0.9357
Initial Observations:
 {Last Gasp} => {NIC Power Restore} are happening in Tandem ( confidence is 0.90)
 {NIC Power Restore} => {Last Gasp} : Cases where NIC Power Restore has happened previously – the likelihood of {Last Gasp} taking place in 0.79
 {NIC First Time Set After Boot} => {Last Gasp} : Confidence 0.80
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 13
Support Confidence
Meter Events File Analysis – Meter Change
Support : This says how
frequent an ‘event’ is, as
measured by the proportion of
line items in which an event
appears.
Confidence: This says how likely
event Y is to occur when
another X has happened,
expressed as {X -> Y}.
Top event sequence based rules
Top 5 Rules
Initial Observations:
 NIC First Time Set After Boot and NIC Power Restore Log Entry
are happening in Tandem ( confidence is 1)
 ‘NIC First Time Set After Boot’ and ‘NIC Power Restore’ are
happening in Tandem ( confidence is 0.99)
Further Steps :
 High support in Rules might not indicate the importance
 Specifically focus on key ‘events’ and filter out the preceding
events leading to the ‘final event’
{NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.87865 1.0000
{NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.87865 1.0000
{NIC First Time Set After Boot} => {NIC Power Restore} 0.87773 0.9989
{NIC Power Restore} => {NIC First Time Set After Boot} 0.87773 0.9572
{NIC Power Restore Log Entry} => {NIC Power Restore} 0.87773 0.9989
{NIC Power Restore} => {NIC Power Restore Log Entry} 0.87773 0.9572
{NIC First Time Set After Boot,NIC Power Restore Log Entry} => {NIC Power Restore} 0.87773 0.9989
{NIC First Time Set After Boot,NIC Power Restore} => {NIC Power Restore Log Entry} 0.87773 1.0000
{NIC Power Restore, NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.87773 1.0000
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 14
Support Confidence
Meter Events File Analysis – Non Meter Change
{NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.83320 1.0000
{NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.83320 1.0000
{NIC First Time Set After Boot} => {NIC Power Restore} 0.83244 0.9990
{NIC Power Restore} => {NIC First Time Set After Boot} 0.83244 0.9150
{NIC Power Restore Log Entry} => {NIC Power Restore} 0.83244 0.9990
{NIC Power Restore} => {NIC Power Restore Log Entry} 0.83244 0.9150
{NIC First Time Set After Boot,NIC Power Restore Log Entry} => {NIC Power Restore} 0.83244 0.9990
{NIC First Time Set After Boot,NIC Power Restore} => {NIC Power Restore Log Entry} 0.83244 1.0000
{NIC Power Restore,NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.83244 1.0000
Top event sequence based rules
Top 5 Rules
Initial Observations:
 NIC First Time Set After Boot and NIC Power Restore Log Entry
are happening in Tandem ( confidence is 1)
 ‘NIC First Time Set After Boot’ and ‘NIC Power Restore’ are
happening in Tandem ( confidence is 0.99)
Further Steps :
 High support in Rules might not indicate the importance
 Specifically focus on key ‘events’ and filter out the preceding
events leading to the ‘final event’
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 15
Next Steps
Survival Analysis
 Time to failure
 Variables are associated with time until the occurrence of an event
 Estimated effort 2 weeks
Classification Model
 Assigns probability of failure to device type
 Variables are calculated for a particular duration of time
 Estimated effort 2 weeks
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 16© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved.
Week#2-Exploratory Data Analysis of Meter
Reads
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 17
Proposed Timeline
Project Plan
Deliverables
Key Activities
Execution
Timeline
(week #)
• Due diligence, Determine
Business Objectives
• Assess As-is Situation(Process
Flow, Data Flow, Frequency
Distribution, Data Granularity)
• Mapping of events, reads and meter details file
• Integration of data sources by joining keys
• Data Exploration, Aggregation and Cleaning
• Missing value treatment, outlier treatment etc.
• Univariate/Bivariate analysis, Correlation and
Multi-collinearity check
• Data preprocessing for predicting meter change
• Building prediction model
• Model validation, build visualizations
• Provide recommendations and
present insights
• Project Plan Document
• Process Flow
• Initial Descriptive Analysis
document of Meter reads
• Exploratory Analysis document Meter reads
pattern, meter reads error code sequencing,
meter events sequencing patterns,
manufacturing details of the meter
• Model result
• Final presentation with insights
1st
Week
3rd
Week
2nd
Week
4th
Week
7th
Week
8th
Week
Business Understanding
and Data Gathering
Exploratory Data Analysis
and Data Preparation
Model Building and Insight
Generation
5th
Week
6th
Week
9th
Week
Completed 2nd week
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 18
Initial Data Sets Created
File Name No. of Rows
Unique
Records
Additional Comments
Meter Scrub Details Report 127,198 5,854 Current Meter Status if working properly or not
MC_MeterDetails 1,234 1,234 Meter Manufacturer's detail
NonMC_MeterDetails_Latest 1,610 1,610 Meter Manufacturer's detail
Meter_Event_Type 427 427 Event Details - Like type of event
Meter_Events_NonMC 18,218 350 Meter Events from 26th to 29th August
Meter_Events_MC_Raw 4,9644 1,033 Meter Events from 26th to 29th August
MeterReads_MC_26th
360,820 454 Past two month read July & August for meters with LCD as 26
th
August.
MeterReads_MC_27th
224,589 282 Past two month read July & August for meters with LCD as 27
th
August.
MeterReads_MC_28th
199,325 247 Past two month read July & August for meters with LCD as 28
th
August.
MeterReads_MC_29th
142,859 239 Past two month read July & August for meters with LCD as 29
th
August.
Read_flag_lookup 49 49 Error code of the meter reads file.
MeterReads_NonMC 677,089 216 Past two months of data July & August’17 of meters working fine.
Data consist of
 1,222 meters that require meter change
 1,610 meters that does not require meter change
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 19
Process Flow
MeterReads_MC
MeterReads_NonMC
Read_Error Code_Flag
(Example: RMS_VOLTAGE_ERROR
INTERVAL_GMI_CLOCK_BACKWARD
Meter Read Sequencing
& Read Error Details
Meter Manufacturing
Details
Reads Patterns Analyzed
for MC and Non MC
Meter Reads Data
Meter Event Sequencing
& Meter Event Details
Meter Read Sequencing
& Read Error Details
Analysis Universe
Meter_Events_NonMC
Meter_Events_MC_Raw
Meter_Event_Type
Meter Event Sequencing
& Meter Event Details
Combining using Event ID
Meter Events Data
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 20© 2017, Cognizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved.
Exploratory Data Analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 21
Data Dictionary-Meter Reads(Applied to both MC and Non MC)
Column Name Description Comments
ami_dvc_name
Unique identifier for devices in UIQ. It is a manufacture code concatenated with
a serial number
Identification of meters
action_code Meter Status
’MC’ for Meter Change and Other values for
communicating meters
Cutoffdatetime Last Communication Date Used for analyzing read patterns before LCD
read_time Timestamp of meter read Used for meter reads sequencing
uiq_chnl_num
Uiq Channel Number is the channel number from UIQ meter program and
export with the meter readings
Used for analyzing patterns of all the different
channels
read_type Code which indicates whether it is an interval or register read
Commercial Register, Commercial Interval,
Residential Register, Residential Interval
uom Unit of Measure (ie.KWH KWD, KVAR, KVRMS, etc) Differentiate the reading type
mrdg Meter Reading
mtr_flag_code
Code used to join to the read flag lookup table to identify flags associataed with
the meter reading....
Flag would be same for all the channels of a meter
at a particular time.
Variables used for analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 22
Data Dictionary-Meter Reads(Applied to both MC and Non MC)
Column Name Description Comments
rgst_read_src
Register Read Type Description can be a self read or a meter ping
for information. From which tables its getting read.
Not relevant for the analysis
read_src Read Source. Product that captures reading from Smart meters Not relevant for the analysis
read_chnl_flag Read Channel Flag Description >95% rows are blank
crtl_peak_pricing_sson Critical Peak Pricing Season numeric code Either blank or Zero
etl_btch_id
Unique identifier assigned to each ETL batch session This unique
identifier will be applied at the row level for every insert or update.
Not relevant for the analysis
etl_prcs_seq Number that indicates the order of processing within an ETL batch
Not relevant for the analysis
Variables NOT used for analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 23
Register Read for Residential Meters(Starting with ‘G’)
(Compare Pattern Meter Change Vs Non Meter Change)
Observations:
 Meter device#G0201018172 which require change
shows gradual Increase in the reading over a period
of 30 days
 Meter device#G0209150804 which does not require
change shows constant reading for past 60 days
Action Items
 Validate the meter reads patterns for the sample
population of 1000+ meters
 Analyze last 60 days of reading for the meters which
require change
MC#G0201018172
Non MC#G0209150804
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 24
Interval Reads for Residential Meters (Starting with ‘G’)
(Compare Pattern Meter Change Vs Non Meter Change)
Observations
 Meter device#G0201018172 which require change
show irregular spikes observed in past 30 days from
LCD
 Meter device#G0209150804 which does not require
change shows regular peaks observed in past 60
days of data
MC#G0201018172
Non MC#G0209150804
Action Items
 Validate the meter reads patterns for the sample
population of 1000+ meters
 Analyze last 60 days of reading for the meters which
require change
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 25
Register Read for Residential Meters (Starting with ‘G’)
(Compare Pattern Meter Change Vs Non Meter Change)
Observation
 Meter device#G0202018083 which require change
shows gradual increase in the reading over a period
of 30 days
 Meter device#G0209156046 which does not require
change shows constant reading for past 60 days
MC#G0202018083
Non MC#G0209156046
Action Items
 Validate the meter reads patterns for the sample
population of 1000+ meters
 Analyze last 60 days of reading for the meters which
require change
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 26
Interval Reads for Residential Meters (Starting with ‘G’)
(Compare Pattern Meter Change Vs Non Meter Change)
Observations
 Meter device#G0202018083 which require change
shows irregular Spikes in past 30 days from LCD
 Meter device#G0209156046 which does not require
change shows regular peaks observed in past 60
days of data
MC#G0202018083
Non MC#G0209156046
Action Items
 Validate the meter reads patterns for the sample
population of 1000+ meters
 Analyze last 60 days of reading for the meters which
require change
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 27
Data Dictionary-*Meter Events(Applied to both MC and Non MC)
Column Name Description Comments
action_code Meter Status MC for Meter Change and Non MC for Communicating meters
Cutoffdatetime Last Communication Date Used to analyze past patterns
mtr_evnt_log_id Unique value applied to each meter event record in UIQ Unique Values for all rows
ami_dvc_name
Unique identifier for devices in UIQ.
Manufacture code concatenated with a serial number
Identification of meters
mtr_evnt_id Numerical code representing the type of meter event. Key to join to the meter event type table to get the event description
mtr_evnt_tmstmp Timestamp representing when the event occurred Used for time sequencing of the events
src_admin_st Ami Device Status
Values like Active, Disconnected, Inactive, Investigate, Maintenance,
New, Retired, Service Disconnect
src_operal_st Ami Device Network Status
Values like Active, Disconnected, Discovered, Inactive, Init Failed,
Initializing, Installed, Investigate, Maintenance, New, Removed,
Retired, Service Disconnect, Unreachable.
sag_or_swell_cnt Sag or Swell Count Important event parameter
mtr_evnt_name Meter Event Description Used for differentiating the events. Details from meter event type table
mtr_evnt_sevrty Meter Event Severity Details from meter event type table
mtr_evnt_ctgy Meter Event Category 1
Details from meter event type table
Variables that would be used for analysis
Note:
*Will be revised after exploratory data analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 28
Data Dictionary-*Meter Events(Applied to both MC and Non MC)
Column Name Description Comments
ami_proxy_dvc_name AMI Proxy Device Name Same as ami_dvc_name.
evnt_argus Event Arguments
Additional events details
evnt_txt Event Text is a textual representation of the EVENT ARGUMNETS Additional events details
location_util_id Location Utility Id
service_pt_util_id Service Point Utility Id
mtr_evnt_insrt_tmstmp Timestamp representing when the event was recorded in UIQ Not relevant for the analysis
mtr_evnt_cnt Column is no longer used All rows are blank
etl_prcs_id
Unique identifier assigned to each ETL batch session
This unique identifier will be applied at the row level for every insert or
update.
Not relevant for the analysis
etl_prcs_type.
Attribute that will indicate the exact type of action that was
performed at the row level of the target table Insert (I), Update (U), etc
Not relevant for the analysis
Variables that would NOT be used for analysis
Note:
*Will be revised after exploratory data analysis
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 29
Summary
Observations:
 Meters which require change shows
o Gradual increase in the register read for past 30 days from LCD
o Irregular spikes in interval read for past 30 days from LCD
 Meter which does not require change
o Constant register read for past 60 days(July & August month)
o Regular peaks for past 60 days(July & August month)
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 30
Week#3 & Week#4 – Activities
Hypothesis#1: Is same meter read patterns observed across meters?
 Explore last 60 days of reading for the meters which require change
 Frequency distribution of error codes of meter reads
Hypothesis#2: Is meter following a sequential pattern of events?
 Event A -> Event B -> Event C
Illustrative:
NIC Insufficient Privilege Last GaspSag Count
© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 31© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved.
THANK YOU

More Related Content

Similar to Meter anomaly detection

Transformer Asset Management & Analytics
Transformer Asset Management & AnalyticsTransformer Asset Management & Analytics
Transformer Asset Management & AnalyticsGENUINE AUTOMATION
 
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...Predictive Maintenance Systems, Technologies & Equipment Management Softwares...
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...proudmehta
 
Digitized Meter and Invoice Generation
Digitized Meter and Invoice GenerationDigitized Meter and Invoice Generation
Digitized Meter and Invoice GenerationIRJET Journal
 
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...IRJET Journal
 
Using Grid data analytics to protect revenue, reduce network losses and impro...
Using Grid data analytics to protect revenue, reduce network losses and impro...Using Grid data analytics to protect revenue, reduce network losses and impro...
Using Grid data analytics to protect revenue, reduce network losses and impro...Schneider Electric
 
Managing the Meter Shop of the Future Through Better Tools and Information
Managing the Meter Shop of the Future Through Better Tools and InformationManaging the Meter Shop of the Future Through Better Tools and Information
Managing the Meter Shop of the Future Through Better Tools and InformationTESCO - The Eastern Specialty Company
 
Condition-based Maintenance with sensor arrays and telematics
Condition-based Maintenance with sensor arrays and telematicsCondition-based Maintenance with sensor arrays and telematics
Condition-based Maintenance with sensor arrays and telematicsGopalakrishna Palem
 
IRJET - Eloquent Salvation and Productive Outsourcing of Big Data
IRJET -  	  Eloquent Salvation and Productive Outsourcing of Big DataIRJET -  	  Eloquent Salvation and Productive Outsourcing of Big Data
IRJET - Eloquent Salvation and Productive Outsourcing of Big DataIRJET Journal
 
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the Tank
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the TankDigital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the Tank
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the TankIRJET Journal
 
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICS
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICSCONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICS
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICSijmnct
 
Business Analytics and Optimization Introduction (part 2)
Business Analytics and Optimization Introduction (part 2)Business Analytics and Optimization Introduction (part 2)
Business Analytics and Optimization Introduction (part 2)Raul Chong
 
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-Computing
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-ComputingRedefining-Smart-Grid-Architectural-Thinking-Using-Stream-Computing
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-ComputingAjoy Kumar
 
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)FELIDAE SYSTEMS
 
Reduce the Internal and External Interruptions and Improving Communications u...
Reduce the Internal and External Interruptions and Improving Communications u...Reduce the Internal and External Interruptions and Improving Communications u...
Reduce the Internal and External Interruptions and Improving Communications u...IRJET Journal
 
A Review on GSM Based Monthly Electrical Energy Billing Through SMS
A Review on GSM Based Monthly Electrical Energy Billing Through SMSA Review on GSM Based Monthly Electrical Energy Billing Through SMS
A Review on GSM Based Monthly Electrical Energy Billing Through SMSIRJET Journal
 
Smart meter parameters for smart grid
Smart meter parameters for smart gridSmart meter parameters for smart grid
Smart meter parameters for smart grideSAT Journals
 
Supporting the Smart Grid with IMDBS
Supporting the Smart Grid with IMDBSSupporting the Smart Grid with IMDBS
Supporting the Smart Grid with IMDBSStephen Dillon
 
Qi Square: Digitizing Energy Efficiency Through Virtual Audits
Qi Square: Digitizing Energy Efficiency Through Virtual AuditsQi Square: Digitizing Energy Efficiency Through Virtual Audits
Qi Square: Digitizing Energy Efficiency Through Virtual AuditsNilesh Y. Jadhav
 

Similar to Meter anomaly detection (20)

Transformer Asset Management & Analytics
Transformer Asset Management & AnalyticsTransformer Asset Management & Analytics
Transformer Asset Management & Analytics
 
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...Predictive Maintenance Systems, Technologies & Equipment Management Softwares...
Predictive Maintenance Systems, Technologies & Equipment Management Softwares...
 
Digitized Meter and Invoice Generation
Digitized Meter and Invoice GenerationDigitized Meter and Invoice Generation
Digitized Meter and Invoice Generation
 
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...
Comparative Analysis of Machine Learning Algorithms for their Effectiveness i...
 
Managing Vendor Issues on AMI Design
Managing Vendor Issues on AMI DesignManaging Vendor Issues on AMI Design
Managing Vendor Issues on AMI Design
 
Using Grid data analytics to protect revenue, reduce network losses and impro...
Using Grid data analytics to protect revenue, reduce network losses and impro...Using Grid data analytics to protect revenue, reduce network losses and impro...
Using Grid data analytics to protect revenue, reduce network losses and impro...
 
Managing the Meter Shop of the Future Through Better Tools and Information
Managing the Meter Shop of the Future Through Better Tools and InformationManaging the Meter Shop of the Future Through Better Tools and Information
Managing the Meter Shop of the Future Through Better Tools and Information
 
Condition-based Maintenance with sensor arrays and telematics
Condition-based Maintenance with sensor arrays and telematicsCondition-based Maintenance with sensor arrays and telematics
Condition-based Maintenance with sensor arrays and telematics
 
IRJET - Eloquent Salvation and Productive Outsourcing of Big Data
IRJET -  	  Eloquent Salvation and Productive Outsourcing of Big DataIRJET -  	  Eloquent Salvation and Productive Outsourcing of Big Data
IRJET - Eloquent Salvation and Productive Outsourcing of Big Data
 
Challenges for Meter Shop Operations of the Future
Challenges for Meter Shop Operations of the FutureChallenges for Meter Shop Operations of the Future
Challenges for Meter Shop Operations of the Future
 
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the Tank
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the TankDigital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the Tank
Digital Gasoline Indicator which Shows Accurate Fuel/Gasoline in the Tank
 
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICS
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICSCONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICS
CONDITION-BASED MAINTENANCE USING SENSOR ARRAYS AND TELEMATICS
 
Business Analytics and Optimization Introduction (part 2)
Business Analytics and Optimization Introduction (part 2)Business Analytics and Optimization Introduction (part 2)
Business Analytics and Optimization Introduction (part 2)
 
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-Computing
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-ComputingRedefining-Smart-Grid-Architectural-Thinking-Using-Stream-Computing
Redefining-Smart-Grid-Architectural-Thinking-Using-Stream-Computing
 
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)
INTERNET OF THINGS (ONLINE ENERGY MANAGEMENT)
 
Reduce the Internal and External Interruptions and Improving Communications u...
Reduce the Internal and External Interruptions and Improving Communications u...Reduce the Internal and External Interruptions and Improving Communications u...
Reduce the Internal and External Interruptions and Improving Communications u...
 
A Review on GSM Based Monthly Electrical Energy Billing Through SMS
A Review on GSM Based Monthly Electrical Energy Billing Through SMSA Review on GSM Based Monthly Electrical Energy Billing Through SMS
A Review on GSM Based Monthly Electrical Energy Billing Through SMS
 
Smart meter parameters for smart grid
Smart meter parameters for smart gridSmart meter parameters for smart grid
Smart meter parameters for smart grid
 
Supporting the Smart Grid with IMDBS
Supporting the Smart Grid with IMDBSSupporting the Smart Grid with IMDBS
Supporting the Smart Grid with IMDBS
 
Qi Square: Digitizing Energy Efficiency Through Virtual Audits
Qi Square: Digitizing Energy Efficiency Through Virtual AuditsQi Square: Digitizing Energy Efficiency Through Virtual Audits
Qi Square: Digitizing Energy Efficiency Through Virtual Audits
 

Recently uploaded

Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...
Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...
Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...dannyijwest
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayEpec Engineered Technologies
 
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptxS1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptxSCMS School of Architecture
 
Computer Networks Basics of Network Devices
Computer Networks  Basics of Network DevicesComputer Networks  Basics of Network Devices
Computer Networks Basics of Network DevicesChandrakantDivate1
 
School management system project Report.pdf
School management system project Report.pdfSchool management system project Report.pdf
School management system project Report.pdfKamal Acharya
 
fitting shop and tools used in fitting shop .ppt
fitting shop and tools used in fitting shop .pptfitting shop and tools used in fitting shop .ppt
fitting shop and tools used in fitting shop .pptAfnanAhmad53
 
Introduction to Geographic Information Systems
Introduction to Geographic Information SystemsIntroduction to Geographic Information Systems
Introduction to Geographic Information SystemsAnge Felix NSANZIYERA
 
8086 Microprocessor Architecture: 16-bit microprocessor
8086 Microprocessor Architecture: 16-bit microprocessor8086 Microprocessor Architecture: 16-bit microprocessor
8086 Microprocessor Architecture: 16-bit microprocessorAshwiniTodkar4
 
Databricks Generative AI Fundamentals .pdf
Databricks Generative AI Fundamentals  .pdfDatabricks Generative AI Fundamentals  .pdf
Databricks Generative AI Fundamentals .pdfVinayVadlagattu
 
Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)Ramkumar k
 
Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2ChandrakantDivate1
 
Post office management system project ..pdf
Post office management system project ..pdfPost office management system project ..pdf
Post office management system project ..pdfKamal Acharya
 
Augmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxAugmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxMustafa Ahmed
 
Introduction to Serverless with AWS Lambda
Introduction to Serverless with AWS LambdaIntroduction to Serverless with AWS Lambda
Introduction to Serverless with AWS LambdaOmar Fathy
 
Computer Graphics Introduction To Curves
Computer Graphics Introduction To CurvesComputer Graphics Introduction To Curves
Computer Graphics Introduction To CurvesChandrakantDivate1
 
Worksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxWorksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxMustafa Ahmed
 
Compressing and Sparsifying LLM in GenAI Applications
Compressing and Sparsifying LLM in GenAI ApplicationsCompressing and Sparsifying LLM in GenAI Applications
Compressing and Sparsifying LLM in GenAI ApplicationsMFatihSIRA
 
Electromagnetic relays used for power system .pptx
Electromagnetic relays used for power system .pptxElectromagnetic relays used for power system .pptx
Electromagnetic relays used for power system .pptxNANDHAKUMARA10
 

Recently uploaded (20)

Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...
Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...
Cybercrimes in the Darknet and Their Detections: A Comprehensive Analysis and...
 
Standard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power PlayStandard vs Custom Battery Packs - Decoding the Power Play
Standard vs Custom Battery Packs - Decoding the Power Play
 
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptxS1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
S1S2 B.Arch MGU - HOA1&2 Module 3 -Temple Architecture of Kerala.pptx
 
Computer Networks Basics of Network Devices
Computer Networks  Basics of Network DevicesComputer Networks  Basics of Network Devices
Computer Networks Basics of Network Devices
 
School management system project Report.pdf
School management system project Report.pdfSchool management system project Report.pdf
School management system project Report.pdf
 
fitting shop and tools used in fitting shop .ppt
fitting shop and tools used in fitting shop .pptfitting shop and tools used in fitting shop .ppt
fitting shop and tools used in fitting shop .ppt
 
Signal Processing and Linear System Analysis
Signal Processing and Linear System AnalysisSignal Processing and Linear System Analysis
Signal Processing and Linear System Analysis
 
Introduction to Geographic Information Systems
Introduction to Geographic Information SystemsIntroduction to Geographic Information Systems
Introduction to Geographic Information Systems
 
8086 Microprocessor Architecture: 16-bit microprocessor
8086 Microprocessor Architecture: 16-bit microprocessor8086 Microprocessor Architecture: 16-bit microprocessor
8086 Microprocessor Architecture: 16-bit microprocessor
 
Databricks Generative AI Fundamentals .pdf
Databricks Generative AI Fundamentals  .pdfDatabricks Generative AI Fundamentals  .pdf
Databricks Generative AI Fundamentals .pdf
 
Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)Theory of Time 2024 (Universal Theory for Everything)
Theory of Time 2024 (Universal Theory for Everything)
 
Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2Fundamentals of Internet of Things (IoT) Part-2
Fundamentals of Internet of Things (IoT) Part-2
 
Post office management system project ..pdf
Post office management system project ..pdfPost office management system project ..pdf
Post office management system project ..pdf
 
Augmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptxAugmented Reality (AR) with Augin Software.pptx
Augmented Reality (AR) with Augin Software.pptx
 
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak HamilCara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
Cara Menggugurkan Sperma Yang Masuk Rahim Biyar Tidak Hamil
 
Introduction to Serverless with AWS Lambda
Introduction to Serverless with AWS LambdaIntroduction to Serverless with AWS Lambda
Introduction to Serverless with AWS Lambda
 
Computer Graphics Introduction To Curves
Computer Graphics Introduction To CurvesComputer Graphics Introduction To Curves
Computer Graphics Introduction To Curves
 
Worksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptxWorksharing and 3D Modeling with Revit.pptx
Worksharing and 3D Modeling with Revit.pptx
 
Compressing and Sparsifying LLM in GenAI Applications
Compressing and Sparsifying LLM in GenAI ApplicationsCompressing and Sparsifying LLM in GenAI Applications
Compressing and Sparsifying LLM in GenAI Applications
 
Electromagnetic relays used for power system .pptx
Electromagnetic relays used for power system .pptxElectromagnetic relays used for power system .pptx
Electromagnetic relays used for power system .pptx
 

Meter anomaly detection

  • 1. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 1© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. Data Sciences Solution for APS
  • 2. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 2 Goals and Objective  Objective  Identify a pattern of data that will more effectively identify smart meters that have permanently stopped communicating vs intermittent and temporary conditions i.e. power outage, tampering, etc.  Goal  Eliminate the back-office manual investigation and 10 day delay  Replace the meter before any billing impact  Avoid replacing smart meters not communicating due to temporary condition  Allow field workers to increase SLA for the meter replacement and uniform their workload  Cost Benefits  Potential reduction of back-office investigator by 1 FTE = $53K
  • 3. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 3© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. Week#3 , Week#4 & Week#5 Hypothesis#1: Is same meter read patterns observed across meters? Hypothesis#2: Is meter following a sequential pattern of events?
  • 4. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 4 Proposed Timeline Project Plan Deliverables Key Activities Execution Timeline (week #) • Due diligence, Determine Business Objectives • Assess As-is Situation(Process Flow, Data Flow, Frequency Distribution, Data Granularity) • Mapping of events, reads and meter details file • Integration of data sources by joining keys • Data Exploration, Aggregation and Cleaning • Missing value treatment, outlier treatment etc. • Univariate/Bivariate analysis, Correlation and Multi-collinearity check • Data preprocessing for predicting meter change • Building prediction model • Model validation, build visualizations • Provide recommendations and present insights • Project Plan Document • Process Flow • Initial Descriptive Analysis document of Meter reads • Exploratory Analysis document Meter reads pattern, meter reads error code sequencing, meter events sequencing patterns, manufacturing details of the meter • Model result • Final presentation with insights 1st Week 3rd Week 2nd Week 4th Week 7th Week 8th Week Business Understanding and Data Gathering Exploratory Data Analysis and Data Preparation Model Building and Insight Generation 5th Week 6th Week 9th Week We are in 5th Week
  • 5. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 5 Patterns in Register Reads of Residential Meters Permanently Stopped Communicating(Unit of Measure - kwh)NormalizedMeterReadinginKwh Register Reads for past 30 days from LCD Register Reads for past 30 days from LCD NormalizedMeterReadinginKwh Meter reading increases with time Meter reading increases with time 78% Meters fall in this cluster 19% Meters fall in this cluster Meter reading constant with time 3% Meters fall in this cluster Key Patterns:  Positive Trend: 97% Meters(397 out of 410) that have stopped communicating permanently show an increasing trend of normalized meter reading  Constant reading: 3% Meters(13 out of 410) that have stopped communicating permanently show constant normalized meter reading Machine Learning algorithm:  Time series clustering Cluster#1 – Positive Trend Cluster#2 – Positive Trend Cluster#2 – Constant Reading
  • 6. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 6 Patterns in Interval Reads of Residential Meters Permanently Stopped Communicating(Unit of Measure - Vrmsh) NormalizedMeterReadinginVrmsh Interval Reads for past 30 days from LCD 67% Meters fall in this cluster 33% Meters fall in this cluster Key Patterns:  Between 2.5 and -2.5: 67% Meters(279 out of 417) that have stopped communicating permanently have normalized meter reading fluctuating between -2.5 and 2.5.  Between 2.5 to -2.5 & beyond: 33% Meters(138 out of 417) that have stopped communicating permanently have normalized meter reading fluctuating between 2.5 to -2.5 & beyond. Machine Learning algorithm:  Time series clustering Cluster#1 – Value between 2.5 and -2.5 Cluster#1 – Value between 2.5 and -2.5 & beyond Fluctuation Range ~5 Units Reading with ‘0’ value
  • 7. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 7 Patterns in Register Reads of Residential Meters Communicating(Unit of Measure - kwh) NormalizedMeterReadinginKwh Register Reads for 60 days Meter reading increases with time 61% Meters fall in this cluster Meter reading constant with time 39% Meters fall in this cluster Key Patterns:  Constant Reading: 39% Meters(59 out of 151) that are still communicating shows constant normalized meter reading.  Positive Trend: 61% Meters(92 out of 151) that are still communicating show an increasing trend of normalized meter reading Machine Learning algorithm:  Time series clustering Cluster#1 – Constant Reading Cluster#2 – Positive Trend
  • 8. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 8 Patterns in Interval Reads of Residential Meters Communicating(Unit of Measure - Vrmsh) NormalizedMeterReadinginVrmsh Interval reads for 60 days Key Patterns:  Between 1.5 and -1.5: 26% Meters(39 out of 156) that have stopped communicating permanently have normalized meter reading fluctuating between 1.5 and -1.5.  Between 1.5 to -1.5 & beyond: 77% Meters(117 out of 156) that have stopped communicating permanently have normalized meter reading fluctuating between 1.5 to -1.5 & beyond. Machine Learning algorithm:  Time series clustering Fluctuation Range ~ 3 Units Cluster#1 – Value between 1.5 to -1.5 Cluster#1 – Value between 1.5 to -1.5 & beyond Reading with ‘0’ value 25% Meters fall in this cluster 75% Meters fall in this cluster
  • 9. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 9 Data Dictionary-Meter Events(Applied to both MC and Non MC) Column Name Description Comments action_code Meter Status MC for Meter Change and Non MC for Communicating meters Cutoffdatetime Last Communication Date Used to analyze past patterns mtr_evnt_log_id Unique value applied to each meter event record in UIQ Unique Values for all rows ami_dvc_name Unique identifier for devices in UIQ. Manufacture code concatenated with a serial number Identification of meters mtr_evnt_id Numerical code representing the type of meter event. Key to join to the meter event type table to get the event description mtr_evnt_tmstmp Timestamp representing when the event occurred Used for time sequencing of the events sag_or_swell_cnt Sag or Swell Count Important event parameter mtr_evnt_name Meter Event Description Used for differentiating the events. Details from meter event type table Variables that would be used for analysis
  • 10. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 10 Data Dictionary-Meter Events(Applied to both MC and Non MC) Column Name Description Comments ami_proxy_dvc_name AMI Proxy Device Name Same as ami_dvc_name. evnt_argus Event Arguments Additional events details evnt_txt Event Text is a textual representation of the EVENT ARGUMNETS Additional events details location_util_id Location Utility Id service_pt_util_id Service Point Utility Id mtr_evnt_insrt_tmstmp Timestamp representing when the event was recorded in UIQ Not relevant for the analysis mtr_evnt_cnt Column is no longer used All rows are blank etl_prcs_id Unique identifier assigned to each ETL batch session This unique identifier will be applied at the row level for every insert or update. Not relevant for the analysis etl_prcs_type. Attribute that will indicate the exact type of action that was performed at the row level of the target table Insert (I), Update (U), etc Not relevant for the analysis src_admin_st Ami Device Status Values like Active, Disconnected, Inactive, Investigate, Maintenance, New, Retired, Service Disconnect src_operal_st Ami Device Network Status Values like Active, Disconnected, Discovered, Inactive, Init Failed, Initializing, Installed, Investigate, Maintenance, New, Removed, Retired, Service Disconnect, Unreachable. mtr_evnt_sevrty Meter Event Severity Details from meter event type table mtr_evnt_ctgy Meter Event Category 1 Details from meter event type table Variables that would NOT be used for analysis
  • 11. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 11 *Support **Confidence Meter Events File Analysis – Meter Change ( Illustration ) {Last Gasp} => {NIC Power Restore} 0.480839416 0.981378026 {NIC Power Restore} => {Last Gasp} 0.480839416 0.524378109 {Last Gasp} => {NIC Power Down} 0.474452555 0.968342644 {NIC Power Down} => {Last Gasp} 0.474452555 0.594965675 {Last Gasp} => {NIC First Time Set After Boot} 0.474452555 0.968342644 {NIC First Time Set After Boot} => {Last Gasp} 0.474452555 0.539979232 {Last Gasp} => {NIC Power Restore Log Entry} 0.474452555 0.968342644 {NIC Power Restore Log Entry} => {Last Gasp} 0.474452555 0.539979232 Top event sequence based rules : For {Last Gasp} Initial Observations:  {Last Gasp} => {NIC Power Restore} are happening in Tandem ( confidence is 0.98)  {NIC Power Restore} => {Last Gasp} : Cases where NIC Power Restore has happened previously – the likelihood of {Last Gasp} taking place in 0.52  {Last Gasp} => {NIC Power Down} : - the likelihood of {Last Gasp} leading to {NIC Power Down} is 0.96 In this we have filtered the events for {Last Gasp} and noted what are the preceding / subsequent events with occurrence likelihood *Support : This says how frequent an ‘event’ is, as measured by the proportion of line items in which an event appears. **Confidence: This says how likely event Y is to occur when another X has happened, expressed as {X -> Y}. Machine Learning Algorithm:  Apriori
  • 12. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 12 Support Confidence Meter Events File Analysis – Non Meter Change ( Illustration ) Top event sequence based rules : For {Last Gasp} {Last Gasp} => {NIC Power Restore} 0.7195 0.9064 {NIC Power Restore} => {Last Gasp} 0.7195 0.7908 {Last Gasp} => {NIC First Time Set After Boot} 0.6732 0.8481 {NIC First Time Set After Boot} => {Last Gasp} 0.6732 0.8080 {Last Gasp} => {NIC Power Restore Log Entry} 0.6732 0.8481 {NIC Power Restore Log Entry} => {Last Gasp} 0.6732 0.8080 {Last Gasp,NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.6732 1.0000 {Last Gasp,NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.6732 1.0000 {NIC First Time Set After Boot,NIC Power Restore Log Entry} => {Last Gasp} 0.6732 0.8080 {Last Gasp,NIC First Time Set After Boot} => {NIC Power Restore} 0.6732 1.0000 {Last Gasp,NIC Power Restore} => {NIC First Time Set After Boot} 0.6732 0.9357 Initial Observations:  {Last Gasp} => {NIC Power Restore} are happening in Tandem ( confidence is 0.90)  {NIC Power Restore} => {Last Gasp} : Cases where NIC Power Restore has happened previously – the likelihood of {Last Gasp} taking place in 0.79  {NIC First Time Set After Boot} => {Last Gasp} : Confidence 0.80
  • 13. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 13 Support Confidence Meter Events File Analysis – Meter Change Support : This says how frequent an ‘event’ is, as measured by the proportion of line items in which an event appears. Confidence: This says how likely event Y is to occur when another X has happened, expressed as {X -> Y}. Top event sequence based rules Top 5 Rules Initial Observations:  NIC First Time Set After Boot and NIC Power Restore Log Entry are happening in Tandem ( confidence is 1)  ‘NIC First Time Set After Boot’ and ‘NIC Power Restore’ are happening in Tandem ( confidence is 0.99) Further Steps :  High support in Rules might not indicate the importance  Specifically focus on key ‘events’ and filter out the preceding events leading to the ‘final event’ {NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.87865 1.0000 {NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.87865 1.0000 {NIC First Time Set After Boot} => {NIC Power Restore} 0.87773 0.9989 {NIC Power Restore} => {NIC First Time Set After Boot} 0.87773 0.9572 {NIC Power Restore Log Entry} => {NIC Power Restore} 0.87773 0.9989 {NIC Power Restore} => {NIC Power Restore Log Entry} 0.87773 0.9572 {NIC First Time Set After Boot,NIC Power Restore Log Entry} => {NIC Power Restore} 0.87773 0.9989 {NIC First Time Set After Boot,NIC Power Restore} => {NIC Power Restore Log Entry} 0.87773 1.0000 {NIC Power Restore, NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.87773 1.0000
  • 14. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 14 Support Confidence Meter Events File Analysis – Non Meter Change {NIC First Time Set After Boot} => {NIC Power Restore Log Entry} 0.83320 1.0000 {NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.83320 1.0000 {NIC First Time Set After Boot} => {NIC Power Restore} 0.83244 0.9990 {NIC Power Restore} => {NIC First Time Set After Boot} 0.83244 0.9150 {NIC Power Restore Log Entry} => {NIC Power Restore} 0.83244 0.9990 {NIC Power Restore} => {NIC Power Restore Log Entry} 0.83244 0.9150 {NIC First Time Set After Boot,NIC Power Restore Log Entry} => {NIC Power Restore} 0.83244 0.9990 {NIC First Time Set After Boot,NIC Power Restore} => {NIC Power Restore Log Entry} 0.83244 1.0000 {NIC Power Restore,NIC Power Restore Log Entry} => {NIC First Time Set After Boot} 0.83244 1.0000 Top event sequence based rules Top 5 Rules Initial Observations:  NIC First Time Set After Boot and NIC Power Restore Log Entry are happening in Tandem ( confidence is 1)  ‘NIC First Time Set After Boot’ and ‘NIC Power Restore’ are happening in Tandem ( confidence is 0.99) Further Steps :  High support in Rules might not indicate the importance  Specifically focus on key ‘events’ and filter out the preceding events leading to the ‘final event’
  • 15. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 15 Next Steps Survival Analysis  Time to failure  Variables are associated with time until the occurrence of an event  Estimated effort 2 weeks Classification Model  Assigns probability of failure to device type  Variables are calculated for a particular duration of time  Estimated effort 2 weeks
  • 16. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 16© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. Week#2-Exploratory Data Analysis of Meter Reads
  • 17. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 17 Proposed Timeline Project Plan Deliverables Key Activities Execution Timeline (week #) • Due diligence, Determine Business Objectives • Assess As-is Situation(Process Flow, Data Flow, Frequency Distribution, Data Granularity) • Mapping of events, reads and meter details file • Integration of data sources by joining keys • Data Exploration, Aggregation and Cleaning • Missing value treatment, outlier treatment etc. • Univariate/Bivariate analysis, Correlation and Multi-collinearity check • Data preprocessing for predicting meter change • Building prediction model • Model validation, build visualizations • Provide recommendations and present insights • Project Plan Document • Process Flow • Initial Descriptive Analysis document of Meter reads • Exploratory Analysis document Meter reads pattern, meter reads error code sequencing, meter events sequencing patterns, manufacturing details of the meter • Model result • Final presentation with insights 1st Week 3rd Week 2nd Week 4th Week 7th Week 8th Week Business Understanding and Data Gathering Exploratory Data Analysis and Data Preparation Model Building and Insight Generation 5th Week 6th Week 9th Week Completed 2nd week
  • 18. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 18 Initial Data Sets Created File Name No. of Rows Unique Records Additional Comments Meter Scrub Details Report 127,198 5,854 Current Meter Status if working properly or not MC_MeterDetails 1,234 1,234 Meter Manufacturer's detail NonMC_MeterDetails_Latest 1,610 1,610 Meter Manufacturer's detail Meter_Event_Type 427 427 Event Details - Like type of event Meter_Events_NonMC 18,218 350 Meter Events from 26th to 29th August Meter_Events_MC_Raw 4,9644 1,033 Meter Events from 26th to 29th August MeterReads_MC_26th 360,820 454 Past two month read July & August for meters with LCD as 26 th August. MeterReads_MC_27th 224,589 282 Past two month read July & August for meters with LCD as 27 th August. MeterReads_MC_28th 199,325 247 Past two month read July & August for meters with LCD as 28 th August. MeterReads_MC_29th 142,859 239 Past two month read July & August for meters with LCD as 29 th August. Read_flag_lookup 49 49 Error code of the meter reads file. MeterReads_NonMC 677,089 216 Past two months of data July & August’17 of meters working fine. Data consist of  1,222 meters that require meter change  1,610 meters that does not require meter change
  • 19. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 19 Process Flow MeterReads_MC MeterReads_NonMC Read_Error Code_Flag (Example: RMS_VOLTAGE_ERROR INTERVAL_GMI_CLOCK_BACKWARD Meter Read Sequencing & Read Error Details Meter Manufacturing Details Reads Patterns Analyzed for MC and Non MC Meter Reads Data Meter Event Sequencing & Meter Event Details Meter Read Sequencing & Read Error Details Analysis Universe Meter_Events_NonMC Meter_Events_MC_Raw Meter_Event_Type Meter Event Sequencing & Meter Event Details Combining using Event ID Meter Events Data
  • 20. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 20© 2017, Cognizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. Exploratory Data Analysis
  • 21. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 21 Data Dictionary-Meter Reads(Applied to both MC and Non MC) Column Name Description Comments ami_dvc_name Unique identifier for devices in UIQ. It is a manufacture code concatenated with a serial number Identification of meters action_code Meter Status ’MC’ for Meter Change and Other values for communicating meters Cutoffdatetime Last Communication Date Used for analyzing read patterns before LCD read_time Timestamp of meter read Used for meter reads sequencing uiq_chnl_num Uiq Channel Number is the channel number from UIQ meter program and export with the meter readings Used for analyzing patterns of all the different channels read_type Code which indicates whether it is an interval or register read Commercial Register, Commercial Interval, Residential Register, Residential Interval uom Unit of Measure (ie.KWH KWD, KVAR, KVRMS, etc) Differentiate the reading type mrdg Meter Reading mtr_flag_code Code used to join to the read flag lookup table to identify flags associataed with the meter reading.... Flag would be same for all the channels of a meter at a particular time. Variables used for analysis
  • 22. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 22 Data Dictionary-Meter Reads(Applied to both MC and Non MC) Column Name Description Comments rgst_read_src Register Read Type Description can be a self read or a meter ping for information. From which tables its getting read. Not relevant for the analysis read_src Read Source. Product that captures reading from Smart meters Not relevant for the analysis read_chnl_flag Read Channel Flag Description >95% rows are blank crtl_peak_pricing_sson Critical Peak Pricing Season numeric code Either blank or Zero etl_btch_id Unique identifier assigned to each ETL batch session This unique identifier will be applied at the row level for every insert or update. Not relevant for the analysis etl_prcs_seq Number that indicates the order of processing within an ETL batch Not relevant for the analysis Variables NOT used for analysis
  • 23. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 23 Register Read for Residential Meters(Starting with ‘G’) (Compare Pattern Meter Change Vs Non Meter Change) Observations:  Meter device#G0201018172 which require change shows gradual Increase in the reading over a period of 30 days  Meter device#G0209150804 which does not require change shows constant reading for past 60 days Action Items  Validate the meter reads patterns for the sample population of 1000+ meters  Analyze last 60 days of reading for the meters which require change MC#G0201018172 Non MC#G0209150804
  • 24. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 24 Interval Reads for Residential Meters (Starting with ‘G’) (Compare Pattern Meter Change Vs Non Meter Change) Observations  Meter device#G0201018172 which require change show irregular spikes observed in past 30 days from LCD  Meter device#G0209150804 which does not require change shows regular peaks observed in past 60 days of data MC#G0201018172 Non MC#G0209150804 Action Items  Validate the meter reads patterns for the sample population of 1000+ meters  Analyze last 60 days of reading for the meters which require change
  • 25. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 25 Register Read for Residential Meters (Starting with ‘G’) (Compare Pattern Meter Change Vs Non Meter Change) Observation  Meter device#G0202018083 which require change shows gradual increase in the reading over a period of 30 days  Meter device#G0209156046 which does not require change shows constant reading for past 60 days MC#G0202018083 Non MC#G0209156046 Action Items  Validate the meter reads patterns for the sample population of 1000+ meters  Analyze last 60 days of reading for the meters which require change
  • 26. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 26 Interval Reads for Residential Meters (Starting with ‘G’) (Compare Pattern Meter Change Vs Non Meter Change) Observations  Meter device#G0202018083 which require change shows irregular Spikes in past 30 days from LCD  Meter device#G0209156046 which does not require change shows regular peaks observed in past 60 days of data MC#G0202018083 Non MC#G0209156046 Action Items  Validate the meter reads patterns for the sample population of 1000+ meters  Analyze last 60 days of reading for the meters which require change
  • 27. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 27 Data Dictionary-*Meter Events(Applied to both MC and Non MC) Column Name Description Comments action_code Meter Status MC for Meter Change and Non MC for Communicating meters Cutoffdatetime Last Communication Date Used to analyze past patterns mtr_evnt_log_id Unique value applied to each meter event record in UIQ Unique Values for all rows ami_dvc_name Unique identifier for devices in UIQ. Manufacture code concatenated with a serial number Identification of meters mtr_evnt_id Numerical code representing the type of meter event. Key to join to the meter event type table to get the event description mtr_evnt_tmstmp Timestamp representing when the event occurred Used for time sequencing of the events src_admin_st Ami Device Status Values like Active, Disconnected, Inactive, Investigate, Maintenance, New, Retired, Service Disconnect src_operal_st Ami Device Network Status Values like Active, Disconnected, Discovered, Inactive, Init Failed, Initializing, Installed, Investigate, Maintenance, New, Removed, Retired, Service Disconnect, Unreachable. sag_or_swell_cnt Sag or Swell Count Important event parameter mtr_evnt_name Meter Event Description Used for differentiating the events. Details from meter event type table mtr_evnt_sevrty Meter Event Severity Details from meter event type table mtr_evnt_ctgy Meter Event Category 1 Details from meter event type table Variables that would be used for analysis Note: *Will be revised after exploratory data analysis
  • 28. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 28 Data Dictionary-*Meter Events(Applied to both MC and Non MC) Column Name Description Comments ami_proxy_dvc_name AMI Proxy Device Name Same as ami_dvc_name. evnt_argus Event Arguments Additional events details evnt_txt Event Text is a textual representation of the EVENT ARGUMNETS Additional events details location_util_id Location Utility Id service_pt_util_id Service Point Utility Id mtr_evnt_insrt_tmstmp Timestamp representing when the event was recorded in UIQ Not relevant for the analysis mtr_evnt_cnt Column is no longer used All rows are blank etl_prcs_id Unique identifier assigned to each ETL batch session This unique identifier will be applied at the row level for every insert or update. Not relevant for the analysis etl_prcs_type. Attribute that will indicate the exact type of action that was performed at the row level of the target table Insert (I), Update (U), etc Not relevant for the analysis Variables that would NOT be used for analysis Note: *Will be revised after exploratory data analysis
  • 29. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 29 Summary Observations:  Meters which require change shows o Gradual increase in the register read for past 30 days from LCD o Irregular spikes in interval read for past 30 days from LCD  Meter which does not require change o Constant register read for past 60 days(July & August month) o Regular peaks for past 60 days(July & August month)
  • 30. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 30 Week#3 & Week#4 – Activities Hypothesis#1: Is same meter read patterns observed across meters?  Explore last 60 days of reading for the meters which require change  Frequency distribution of error codes of meter reads Hypothesis#2: Is meter following a sequential pattern of events?  Event A -> Event B -> Event C Illustrative: NIC Insufficient Privilege Last GaspSag Count
  • 31. © 2019-2020, Cognizant Technology Solutions. All Rights Reserved. 31© 2016, Cgnizant Technology Solutions. All Rights Reserved.© 2019-2020, Cognizant Technology Solutions. All Rights Reserved. THANK YOU