The document discusses energy disaggregation, which is the task of taking a whole-home energy signal and separating it into its component appliances. It describes using sparse coding techniques like non-negative sparse coding and discriminative disaggregation to find basis functions and activation coefficients to model each appliance's energy usage and disaggregate a new whole-home signal without providing the individual appliance signals. The goal is to estimate the activation coefficients to separate out the energy used by each appliance.

1. RiskEcon® Lab @ NYU Courant Institute
https://wp.nyu.edu/riskeconlab/
email: riskeconlab@cims.nyu.edu
©2011-2020 RiskEcon® Lab. All rights reserved.
Energy Disaggregation
Problem Formulation
Sparse Coding Pre-training
.
Case Study
Discriminative Disaggregation Prospective Insurance Use Cases
Source: RiskEcon® Lab @ NYU Courant Institute, Other sources
Contributors: Mengheng Xue, Samantha Kappagoda, David K A Mordecai
Energy Disaggregation with Discriminative Sparse Coding
InsurTech Alliance Science & Engineering Expo: January 30, 2020
RiskEcon® Lab for Decision Metrics @ NYU Courant Institute of Mathematical Sciences
Definitions: the task of taking a whole-home energy signal and separating it
into it s component appliances.
Energy Disaggregation
The task of taking a whole-home energy signal and separating it into its
component appliances.
Goal: disaggregate new whole-home data ¯X0
into components X0
1, . . . , X0
k
Goal: disaggregate a new whole-home signal ¯X0
2 RT⇥m
without providing the
algorithm its individual components
January 8, 2020 1 / 2
Definitions: the task of taking a whole-home energy signal and separating it
into it s component appliances.
Energy Disaggregation
The task of taking a whole-home energy signal and separating it into its
component appliances.
Goal: disaggregate new whole-home data ¯X0
into components X0
1, . . . , X0
k
Goal: disaggregate a new whole-home signal ¯X0
2 RT⇥m
without providing the
algorithm its individual components
January 8, 2020 1 / 3
non-negative sparse coding
maximize
Ai 0,Bi 0
kXi Bi Ai k2
F +
X
p,q
(Ai )pq subject to kB
(j)
i k2  1, j = 1, . . . , n
discriminative disaggregation
ˆA0
1:k = arg min
A1:k 0
¯X0
[B1 · · · Bk ]
2
6
4
A0
1
...
A0
k
3
7
5
2
F
+
X
i,p,q
(A0
i )pq
prediction model
ˆX0
i = Bi
ˆA0
i
Goal: find basis functions (or dictionary) Bi 2 RT⇥n
and activations Ai 2 Rn⇥m
of each individual class Xi using Xi ⇡ BAi
January 8, 2020 2 / 3
min
Ai 0,Bi 0
kXi Bi Ai k2
F +
X
p,q
(Ai )pq subject to kB
(j)
i k2  1, j = 1, . . . , n
discriminative disaggregation
ˆA0
1:k = arg min
A1:k 0
¯X0
[B1 · · · Bk ]
2
6
4
A0
1
...
A0
k
3
7
5
2
F
+
X
i,p,q
(A0
i )pq
prediction model
ˆX0
i = Bi
ˆA0
i
Goal: find basis functions (or dictionary) Bi 2 RT⇥n
and activations Ai 2 Rn⇥m
of each individual class Xi using Xi ⇡ Bi Ai
January 8, 2020 2 / 3
• require over-complete representations of data: n m, T
• impose constraint that Ai be sparse to guarantee unique solution<latexit sha1_base64="GTfiVMjTbu1DVkhRV59ygC3z+Ow=">AAADA3icbVJLbxMxEPYurxIeTeHIxSJB4gBREoRAnFpx4Vikpq3UjSKvd3Zj1Y+NPS4Kqxz5NdwQV34I4s8wm6aiD0ay9M3DM/N9dl5rFXA4/J2kt27fuXtv637nwcNHj7e7O08Og4tewkQ67fxxLgJoZWGCCjUc1x6EyTUc5acf2/zRGfignD3AZQ1TIyqrSiUFUmjW/dPJcqiUbUodw1xDiauLiEIw6iuQ3yLuYRGVB95k5Djq+Vo6U2tAWFGOhgawuO4auCt5IVB84H3Ls6ri5hU/6G/6KFO7AFxSHXqhLHKcC+T9zAic52Wzt5qpPs+Bh1p4KkTHqyi8sAjAo1WLSCmnYzupk4EtLi3aev+IzLq94WC4Nn4TjDagxza2P9tJulnhZDTERGoRwsloXOO0ER6V1O2AGKAW8lRUcELQCgNh2qxfYcVfUKTgpfN0iNU6evlGI0wIS5NTZUs1XM+1wf/m5qDPoH0XDxa+kOpGEM+sFEbpZQGliBpXTRbKC3x1TyzfTxtl64hg5fmaZdStru2H4AU9qkS9JCCkV8SUyznpLZG+TYc0HF1X7CY4HA9Gbwbjz+Pe7tuNmlvsGXvOXrIRe8d22Se2zyZMJntJldTJIv2Wfk9/pD/PS9Nkc+cpu2Lpr78uAPfr</latexit>
• Ai, Bi be non-negative since energy usage by nature
• achieve sparsity of activations Ai by adding `1 regularization<latexit sha1_base64="ta7FloVmkdx551R9JpTw6sVmNCE=">AAADE3icbVLNbhMxEPYufyX8NIUjF4sEiQNE2SAExwIXjq1E2krdaDXrHW+s2t6V7U21XUWCh+BpuCGuPADPwEtgp6napoxk6Ztv/mec11JYNx7/ieJbt+/cvbd1v/fg4aPH2/2dJwe2agzDKatkZY5ysCiFxqkTTuJRbRBULvEwP/kU7IcLNFZU+otra5wpKLXggoHzVNb/20tzLIXuuGzsXCJ3ywtGOFTiDL0eEB2mCtw8592HZSaGry71j0GnOVJd6dcaS595gdQKzZCiRlO2tLFQIs1bqsE1Buk6JbC5wOBbg+/QtbTinvPhq+bsRskQD0UhdOkNKGWWDKnBspFgxNkqwudFXVxpPGiXg2X9wXg0Xgm9CZI1GJC17GU7UT8tKtYo1I5JsPY4SWo368A4wWQo0FisgZ346Y491KDQzrrVVZb0hWcKyivjn3Z0xV6N6EBZ26rce4Yx7aYtkP+1zVEuMNzJ+PWeskop8HOmHJSQbYEcGumWXWr5Bb7ep+PvZ53QdeNQs/M2eSOpq2j4ILQQBpmTrQfAjPCTUjYH4+/iv1HP7zDZ3NhNcDAZJW9Gk/3JYPfteptb5Bl5Tl6ShLwju+Qz2SNTwqL96DT6Gn2Lv8c/4p/xr3PXOFrHPCXXJP79D4Hd/Tc=</latexit>
non-negative sparse coding
min
Ai 0,Bi 0
kXi Bi Ai k2
F +
X
p,q
(Ai )pq subject to kB
(j)
i k2  1, j = 1, . . . , n
discriminative disaggregation
ˆA0
1:k = arg min
A0
1:k
0
¯X0
[B1 · · · Bk ]
2
6
4
A0
1
...
A0
k
3
7
5
2
F
+
X
i,p,q
(A0
i )pq
prediction model
ˆX0
i = Bi
ˆA0
i
Goal: find basis functions (or dictionary) Bi 2 RT⇥n
and activations Ai 2 Rn⇥m
of each individual class Xi using Xi ⇡ Bi Ai
January 8, 2020 2 / 3
Goal: estimate the activation coefficient for each unknown appliance (ˆA0
i )
January 8, 2020 3 / 3
• intuition: Bi is better at reconstructing the ith portion of aggregate
signal than the other bases Bj for j 6= i.<latexit sha1_base64="ckA/z8RtdrEZo3+uYwp4DTRYox4=">AAAC/XicbVJNbxMxEPUuXyV8pXDkMiKLxCnKBiEQXCq4cCwSaSt1o8jrjHedeO2VPVsUVhG/hhviyi/hwL/Bm6aiaZmD9fxmRn7zxnmtlafR6E8U37h56/advbu9e/cfPHzU33985G3jBE6E1dad5NyjVgYnpEjjSe2QV7nG43z5ocsfn6HzyprPtKpxWvHCKKkEp0DN+r97WY6FMq3UjS81SlpfMIqwUl8x3DsEylCjuqa3kGQVpzKX7fv1TCWgPORIhA44gUNhjSfXCFKmACoREpVQCbV1XTdYCbwoHBacELwqDNehiptNqQ2Hg24gv/PKIgFpHSSLzCCoZAi9DM38ksTu9m+EWX8wGo42AddBugUDto3D2X7Uz+ZWNBUaEpp7f5qmNU1bHjQL3T3QeKy5WPICTwM0vEI/bTf+r+F5YOYbgdIagg17uaPllferKg+V3Uj+aq4j/5srUZ9htxGHBr8IW1U8zJlJXim9mqPkjaZ1m3l5gXd1knwzbZWpG0IjzmXKJrhtofsKMFdhWaRXAXDhwm4FiJI7LsIqfS94mF517Do4Gg/Tl8Pxp/Hg4NXWzT32lD1jL1jKXrMD9pEdsgkT0buIR4toGX+Lv8c/4p/npXG07XnCdiL+9RfEsfQD</latexit>
• A0
i(i = 1, . . . , k): the activation matrix for the ith appliance’s base
matrix (Bi)<latexit sha1_base64="c+keiWufeEsKjpTvfmJilmQfXOI=">AAAC33icbVJdb9MwFHXC1ygf6+ARIVm0aJ00VU3RNISENNgLDyANiW6TlqpynOvGqu1E9k2hRH3mDfHKL+G38G9wshb2wX06Pvfe+JzjJIWSDgeD30F44+at23c27rbu3X/wcLO99ejY5aXlMOK5yu1pwhwoaWCEEhWcFhaYThScJLPDun8yB+tkbj7hooCxZlMjheQMPTVp/2rFCUylqYQqXaZA4HLNSAQtv4I/14h2Y80wS0T1Zrk9kbQn6Wsa7dJYpTm6XTrb6b6imAFlHOW8+Tr1C1Z+oSK3Tacru5hRVnhfzHDYdrRWvp7q/b3g7XIiuzutGEx6QUR9+idy0u4M+oOm6HUQrUCHrOposhW04zTnpQaDXDHnzqKowHHFLEqu6gtKBwXjMzaFMw8N0+DGVZPwkj73TNoYEblB2rAXNyqmnVvoxE/WJtzVXk3+t5eBmkOduQUDn3muNfM+Y8G0VIsUBCsVLqvYiTW+rBPFy3ElTVEiGH4uU5SKYk7rx6aptMBRLTxg3ErvlPKMWf9E/pdo+Qyjq4ldB8fDfvSiP/w47BzsrdLcIE/IM9IjEdknB+QdOSIjwoOnwWHwPvgQsvBb+D38cT4aBqudx+RShT//AMK25ew=</latexit>
non-negative sparse coding
min
Ai 0,Bi 0
kXi Bi Ai k2
F +
X
p,q
(Ai )pq subject to kB
(j)
i k2  1, j = 1, . . . , n
discriminative disaggregation
ˆA0
1:k = arg min
A0
1:k
0
¯X0
[B1 · · · Bk ]
2
6
4
A0
1
...
A0
k
3
7
5
2
F
+
X
i,p,q
(A0
i )pq
prediction model
ˆX0
i = Bi
ˆA0
i
Goal: find basis functions (or dictionary) Bi 2 RT⇥n
and activations Ai 2 Rn⇥m
of each individual class Xi using Xi ⇡ Bi Ai
January 8, 2020 2 / 3
Definition: task of taking a whole-home energy signal
and separating it into its component appliances.<latexit sha1_base64="jpIaM2g7J4ikgXtsSIlrlWGE4yw=">AAACwXicbZHbbtNAEIbX5lTMKYVLblZElbghJEEIxFWlcsFlkUhbqY6i8XrWXmUP1u66xbLyIDwaTwPjNEg9MJK1v2Zm1/98UzRahTid/k7Se/cfPHy09zh78vTZ8xej/ZcnwbVe4EI47fxZAQG1sriIKmo8azyCKTSeFuujoX56gT4oZ3/ErsGlgcoqqQRESq1Gv7K8wErZXuo21Bpl3GR5xJ+xkP1XlMqqoe/LhkcIa+4knWtlKw78snYa39XOIM/Qoq86HlRlQfM852BLHrABT7+hbhW5stHRGbhwpnEWbeQcGhoRrMAw4VmWoy2v2ViNxtPJdBv8rpjtxJjt4ni1n4zy0onW0NtCQwjns2kTlz34qIRGmqsdLIk1VHhO0oLBsOy3DDf8gDIll87TR9622es3ejAhdKagTgOxDrdrQ/K/tRr1BQ5UPVG6pOkNwelzCUbprkQJrY6bPg/yn77pM8rPy17Zpo1oxZVN2WpOMId18lJ5FFF3JEB4WpbgoibsItLSM2I4u03srjiZT2YfJvPv8/Hhxx3NPfaavWFv2Yx9YofsGztmCybYn+QgmSTv06NUpU3qr1rTZHfnFbsRaf8Xdb/cyA==</latexit>
Why it matters?<latexit sha1_base64="6DlPGK+lIBtyL+pDMBnn14Z9+lU=">AAACQ3icbVDLShxBFK02MdHWJK0u3RSRgKuh2yBmFyGbLBUcR5gehtvVt53Cquqm6rZJ08wi/5B/yDb+QTZ+hN/gTrINWD2j4CMHCg7n3EPde7JKSUdxfBUsvHi5+Or10nK4svrm7btobf3YlbUV2BelKu1JBg6VNNgnSQpPKougM4WD7OxL5w/O0TpZmiNqKhxpODWykALIS+NoMwxTwu+UFe1g0nBJXAORD3yehuNoK+7FM/DnJLkjW/vRn6ufQ7V3MF4LojQvRa3RkFDg3DCJKxq1YEkKhdMwrR1WIM7gFIeeGtDoRu3siin/4JWcF6X1zxCfqQ8TLWjnGp35Sb/jxD31OvG/3gTVOZL/3aLBb6LUGkzepgVoqZocC6gVTdvUFff88Z5UfBq10lQ1oRHzNYtacSp5VyjPpUVBqvEEhJX+Ui4mYEF0LXYdJk8be06Od3rJx97OoS9zl82xxDbZe7bNErbH9tlXdsD6TLAf7Bf7zS6Cy+A6uAn+zkcXgrvMBnuE4N8tVP60Zw==</latexit>
utilities:
• identify/verify appliance in Demand Response
• understand customer behavior
• improve capacity planning<latexit sha1_base64="baUmd9djXCRlY26ci3Bpi57tfT4=">AAAC/XicbVJNbxMxEPUuX2X5SuHIxSJC4kLYBFUguFSCA8eCSFupG0Wz3tnExB8rezZoWUX8Gm6IK7+EA/8Gb5JS2jKS5ec3Hvv5efJKSU9p+juKr1y9dv3Gzs3k1u07d+/1du8fels7gWNhlXXHOXhU0uCYJCk8rhyCzhUe5Ys3Xf5oic5Laz5SU+FEw8zIUgqgQE17v5KkJqkkSfSvkmzpKxDYPh2ke1yaVZLlOJOmLVXt5wpL+stIQi2/YFh3iMsCDcmyeRbuChOHKogHIzCcwt+iBlPwD+grazxuS2pTBF3UZUTtyWp0PMc5LKV1p6fqytklcgFBlaSGVwqMkWbGkwxNcSZiszxTOe3100G6Dn4ZDLegz7ZxMN2NellhRa3DM4QC70+GaUWTFhxJobpn1h6DiAXM8CRAAxr9pF37v+KPA1Pw0rowDPE1+29FC9r7Rudhpwaa+4u5jvxvbo5qiZ3pDg1+FlZ3RrZZCVqqpsASakWrNvPlKT6vk8qXk1aaqiY0YiOzrBUny7tW4IV0KEg1AYBwoQUEF3NwICh8TBI8HF507DI4HA2Gzwej96P+/t7WzR32kD1iT9iQvWD77B07YGMmotcRRJ+iRfw1/hZ/j39stsbRtuYBOxfxzz9J3/WH</latexit>
customers:
• understand bill and plan monthly budget
• identify/repair/replace energy hogs
• make financial decision when to use device<latexit sha1_base64="3pvcU4HJmV3CdrJrriT4n5y1J6Y=">AAADD3icbVLLbtQwFHXCq4TXFJZsLEZIbBgygyoQq0ps2CAViWkrNaORY98kV/Ujsp2pQpiP4GvYIbZ8Al/Ab+DMBEpbrmT5+Nzj+N5zk9cSnU/Tn1F87fqNm7d2bid37t67/2C0+/DQmcZymHMjjT3OmQOJGuYevYTj2gJTuYSj/PRtnz9agXVo9Eff1rBQrNRYIGc+UMvRryThjfNGBc2bJFu5mnHonk/SPYp6nWQ5lKi7QjauklD4vwx6UPgJwrlHtNEifMAzLWiOUtIe1JJpqoz2lWxp3ogS/KBGAdpj0b6wUDO0/SbDsxQ02LKllSndoFTsFGiBmmmOTFIBHPtW6FkFmnpDGweBXCGHJAMtzsvaHs/rXo7G6STdBL0KpgMYkyEOlrvRKBOGNyqUyiVz7mSa1n7RMeuRy77x8HYw65SVcBKgZgrcottMZE2fBkbQwtiwtKcb9t8bHVPOtSoPSsV85S7nevK/uQrkCvox2GDWGTdKBau7rGAKZSugYI306y5zxR98sU5fvF50qOvGg+bbMotG9k72PwcVaIH7MC6BjFsMnVJeMcu4D9NNgofTy45dBYezyfTlZPZhNt7fG9zcIY/JE/KMTMkrsk/ekQMyJzx6H7noc7SOv8Rf42/x9600joY7j8iFiH/8Bns3/F0=</latexit>
• lots of methods one common denominator, Data
– low-resolution smart meter data, e.g., 60-min
• perform unsupervised on test, and supervised on scaled houses
Assume we have 1 : k appliances with
• individual energy reading of ith appliances (e.g. heater): Xi 2 RT ⇥m
,
where T hourly week data for m houses, e.g.,
X1 =
2
6
6
6
6
6
4
App x
(j)
1 · · · x
(m)
1
1h 0.8kWh · ·
2h 0.7kWh · ·
...
... · ·
168h 0.1kWh · · · · · ·
3
7
7
7
7
7
5
• one aggregated test power consumption reading: ¯X0
P
1:k X0
i<latexit sha1_base64="0jFMEOMHwHwnBW9niEQNmPRn1Qs=">AAAFuHicjVTNbtw2EJZjb5tu/+z22BwGteI4xWaxctHEMBAgQXvoMS3s2EDkbClqJLErkipJ7Xoj6BnyfH2RnjvUyvZ60wDlRcP508w33zCpSmHdZPL31r3tncEnn97/bPj5F19+9fXu3jevra4NxzOuS20uEmaxFArPnHAlXlQGmUxKPE9mP3v7+RyNFVqdumWFl5LlSmSCM0eq6d72+zjBXKgmK2tblJi5dthrhEMp3iHdvQSldhZ0BhJdoVMSFQLXUmoFKSothWJOmxH8whz7aIrFE4NWl7X/OVjJjPP50EBKUSPAcT4ewdPJE8o2jFGlmxkqNJk2Empla5LnwmJKlYBD60bAVAp39ZazkqRC1xYtbKb0t7W+hy+trSXCAqFgc4QwOpmFwCqaA1Oc4hfCFTD8v4AJlYq5SGtWAio0+RJoMKlQuUcxFCHlWst96HuHAhmh8fgEwlgyVyRZc9FOBcRCwUqRNL+3b5vT2AlJQbINR7Ao0FCxp6Fv05RLqh9nHaBAWEEow77/a3zj+Lpi/KvuePAD1Xz7vwie33SZkNqIq3b4sqrAnwO4etsc/vmY3A4g5qlnxUonva7zoR9EhXedjI9n5wX0jjdfsh/19mcfscfzPvOtsOkSPT0uVhVNxtFamhtnu5rwTQvdba3lfk6xwytHnROhW2B5bjCnIaQdp6DSC2In18Q3WXWs7Ye4NqI4YYZwax9B7PnAjNELiClg2hCD2ltkH01FONwg4SYLp7v7k/GkO/ChEPXCftCfV9O9rd041ZyIqxwvmbVvoknlLhvaLcFLz0aafcX4jOX4hkTFiDmXTfd0tPCw9pvieZJp5aDTrkc0TFq7lAl5+jbsps0r/9NWYDlHvxuGyL/wDwVtZxNnTIpymWLG6tK1TWyza/lunS47vmyEqmqHiq/KzOoSnAb/ikEqDHJHXE8F40ZQp8ALZhin7bFDwjDaROxD4fXROPpxfPTb0f6Ln3o07wffBd8Hh0EUPAteBL8Gr4KzgG//s/Ng5+HOweBk8McgH4iV672tPubb4M4ZmH8Bkg3ijg==</latexit>
• identify anomalous appliance to avoid
energy wastage, e.g., light bulbs burning
out too often
• detect device failure to avert disasters,
e.g., house fire
• diagnose household electrical problems,
e.g., circuit overload/short circuit
• assist occupancy monitoring<latexit sha1_base64="6Y+mn18l6ivzWaywThuXkdTE7N4=">AAADwnicdVLbbtNAEN3EXEq4pfDIy4oKxEOUOkEIHivBA49FIm2lOqrW67E9dC9md53KmHwIn8bfMM6F0gsr2R6fnd0zc86klUIf4vh3rx/duXvv/s6DwcNHj588He4+O/K2dhJm0irrTlLhQaGBWcCg4KRyIHSq4Dg9/9jtHy/AebTma2gqmGtRGMxRikDQ2W7v12CQpFCgaXNV+1JBHpZbBANo/AH030UcMzAB84YLY7VQtvZcVFSkMBJ4sFwsLGZ8kCQcDLii4RfCB1HAiMO4GI+4wqIMPK1V6untDJqCU7KtA5223OYBDFWz8JWQ0I6nHM2WOoMAMtBngcSVC1S123CCIxw9UVGboxX7iq2k+igVHfznSiQlLKWsEkurMg6KSBxpo3jlLEmo/Yj/vVCikzUGbolSWZHt+9IS9xa+nUV4TyZyK2VdkU4N19ZgsK7rfZCAyS5FXv9eunA23IvH8Wrxm8FkE+yxzTokM4dJZmWtySapiPl0Eldh3goXUKrORmqUSjwnT04pNEKDn7erIVryV4RkPLeOHkNtdei/J1qhvW90SplahNJf3+vAW/dKUAvohsrRWFxIq7WgPpNcaFRNBrmoVVi2ic+38dU6Q/5h3qKpapoOuS4zr1XnfTfPZKMj01TT+SkdUqdclsIJ2Y3DgDScXFfsZnA0HU/ejqdfpnsH7zZq7rAX7CV7wybsPTtgn9khmzHZZ/3X/f1+HH2KvkXfI79O7fc2Z56zKyv6+QfuZTfL</latexit>