Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The DuraMat Data Hub and
Analytics Capability
A Resource for Solar PV Data
Robert White1 and Anubhav Jain2
1National Renew...
Data	and	Analy*cs	overview	-	1	
Field deployment
Data Hub Data analytics
Materials
Forensics &
Characterization
Predictive...
Data	and	Analy*cs	overview	-	2	
Field deployment
Data Hub Data analytics
Materials
Forensics &
Characterization
Predictive...
Why	a	capability	dedicated	to	data?	
Data	relevant	to	solar	manufacturing	and	deployment	
is	largely	sca5ered	and	cannot	b...
Some	ways	in	which	data	is	enabling	
innova*on	in	solar	
Google	Project	Sunroof	and	PowerScout	
partner	to	iden=fy	target	...
Some	ways	in	which	data	is	enabling	
innova*on	in	solar	
Google	Project	Sunroof	and	PowerScout	
partner	to	iden=fy	target	...
The	value	of	a	common	“data	hub”	
•  Much	data	has	li5le	value	when	used	alone	and	becomes	
much	more	impacVul	when	combin...
Examples	of	data	for	“the	commons”	
Footer 4
4 4
Time series data Meteorological data Test chamber data
Materials simulati...
How	to	structure	this	resource?	
Roger	Chen,	“Data	liquidity	in	the	age	of	inference”,	h5p://bit.ly/2yt521W	
Data	heteroge...
How	to	structure	this	resource?	
Roger	Chen,	“Data	liquidity	in	the	age	of	inference”,	h5p://bit.ly/2yt521W	
Data	heteroge...
Data	Hub	Infrastructure
What	is	the	Data	Hub	
“A	collec6on	of	so8ware	components	and	
databases	all	working	in	concert	to	support	secure	
data	sha...
Elements	of	the	Data	Hub	
Modeling	and	
Simula=on	
Forensics,	
Measurement	
and	Synthesis	
Techno-Economic	
Analysis	
Docu...
Data	Management	Structure	
Project
Sub-Project
Dataset
Files and Resources
(Optional)
Example: of Data Management Structure
Project
Sub-Project
Dataset
Files and Resources
Predic've	Simula'on	
Thermomechanica...
“Dynamic,	sequen6al	data,	indexed	by	6me	and	loca6on,	
that	is	con6nuously	being	acquired	from	fielded	PV	
systems.”	
Time	...
The	DuraMAT	Data	Hub	
h5ps://datahub.duramat.org	
	
A_er	registering,	email:	robert.white@nrel.gov	
include	your	ins=tu=on...
Analy*cs
Data	Analy*cs	capability	
Visualize, model, and predict
Release open
source software
•  The	data	analy=cs	thrust	is	cross-...
Example	project:	clear	sky	detec*on	
Visualize, model, and predict
Release open
source software
rdtools	
Data from
partners
Clear	sky	detec*on	problem	
Persistently	
cloudy,	noisy	
Persistently	
cloudy,	noisy	
Borderline	
cases	
Generally	clear	
...
Clear	sky	detec*on	analy*cs	
x1
x2
x3
…
x1
x2
x3
…
x1
x2
x3
…
x1
x2
x3
…
x1
x2
x3
…
x1
x2
x3
…
+ NSRDB clear sky labels
ML...
ML	model	achieves	~98%	accuracy!	
No data cleaning Data cleaning applied
Next	steps:	
•  Evaluate	sensi=vity	of	model	to	d...
Currently	seeking	collabora*ons	
•  The	Data	Analy=cs	thrust	is	also	looking	to	extend	
our	experience	with	=me	series	and...
NREL EMN
Development Team
•  Kris=n	Munch	
•  Nick	Wunder	
•  Chris	Webber	
•  Dave	Evenson	
•  Courtney	
Pailing	
DuraMat...
Upcoming SlideShare
Loading in …5
×

The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data

277 views

Published on

Presentation given at DuraMat workshop at Sandia National Labs, Nov 8, 2017. Joint presentation with Robert White (NREL).

Published in: Science
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data

  1. 1. The DuraMat Data Hub and Analytics Capability A Resource for Solar PV Data Robert White1 and Anubhav Jain2 1National Renewable Energy Laboratory 2Lawrence Berkeley National Laboratory
  2. 2. Data and Analy*cs overview - 1 Field deployment Data Hub Data analytics Materials Forensics & Characterization Predictive simulationModule testing Techno-economic analysis DuraMAT capability areas and partners will share PV performance, materials properties, meteorological data, & other data through the Data Hub
  3. 3. Data and Analy*cs overview - 2 Field deployment Data Hub Data analytics Materials Forensics & Characterization Predictive simulationModule testing Techno-economic analysis Data analytics is a cross-cutting thrust that will work with DuraMat partners to provide software, visualization, and data mining capabilities
  4. 4. Why a capability dedicated to data? Data relevant to solar manufacturing and deployment is largely sca5ered and cannot be easily exploited. A centralized data hub enhances: •  efficiency – reduce produc=on of duplicate data •  reproducibility – collect together results from mul=ple sources for quality assurance purposes •  context – bring together mul=ple data sources (materials proper=es, accelerated tes=ng, field data, meteorological data) into an analysis •  new analysis types – machine learning algorithms require data as much as they require CPUs or memory! More data means be5er models.
  5. 5. Some ways in which data is enabling innova*on in solar Google Project Sunroof and PowerScout partner to iden=fy target consumers for solar to innovate in sales / marke=ng kWh Analy=cs uses data to develop risk assessment models for solar investments IBM’s “Wa5-Sun” uses data and machine learning to provide more reliable solar forecas=ng and power management
  6. 6. Some ways in which data is enabling innova*on in solar Google Project Sunroof and PowerScout partner to iden=fy target consumers for solar to innovate in sales / marke=ng kWh Analy=cs uses data to develop risk assessment models for solar investments IBM’s “Wa5-Sun” uses data and machine learning to provide more reliable solar forecas=ng and power management These are all “data-driven” technologies!
  7. 7. The value of a common “data hub” •  Much data has li5le value when used alone and becomes much more impacVul when combined •  If you need to collect every data point yourself, data-driven technologies become inaccessible to all •  Thus, it is in the best interest if some data is put in a “commons”, available to all Roger Chen, “Data liquidity in the age of inference”, h5p://bit.ly/2yt521W
  8. 8. Examples of data for “the commons” Footer 4 4 4 Time series data Meteorological data Test chamber data Materials simulation Materials properties Degradation analysis The Data Hub will host and connect to data from multiple sources and specializations These research areas and data sources are a small sample of what the hub plans to host
  9. 9. How to structure this resource? Roger Chen, “Data liquidity in the age of inference”, h5p://bit.ly/2yt521W Data heterogeneity / incompa=bility and lack of coordina=on are some problem with typical open data ini=a=ves For-profit data sharing brokerages are incen=vized to collect/standardize data, but may shy away from data with high acquisi=on cost Data coopera=ves enhance standardiza=on and work well for members, but are difficult to get off the ground
  10. 10. How to structure this resource? Roger Chen, “Data liquidity in the age of inference”, h5p://bit.ly/2yt521W Data heterogeneity / incompa=bility and lack of coordina=on are some problem with typical open data ini=a=ves For-profit data sharing brokerages are incen=vized to collect/standardize data, but may shy away from data with high acquisi=on cost Data coopera=ves enhance standardiza=on and work well for members, but are difficult to get off the ground The DuraMat data hub combines elements of the “open data” and “data cooperative” models, providing support for both open and private data. The hub will aid in data standardization while providing flexibility to data providers, and also contribute data with high acquisition cost as generated by DuraMat.
  11. 11. Data Hub Infrastructure
  12. 12. What is the Data Hub “A collec6on of so8ware components and databases all working in concert to support secure data sharing and distribu6on” Criteria for our Data Hub •  Manage research data. •  Provide access to historical and =me-series databases. •  Ability to search data resources and metadata. •  Provided hierarchical data security. •  Provide access to analysis tools. •  Easy (but secure) access to consor=um members and partners. •  Provide access to publically released data.
  13. 13. Elements of the Data Hub Modeling and Simula=on Forensics, Measurement and Synthesis Techno-Economic Analysis Documenta=on and Publica=ons
  14. 14. Data Management Structure Project Sub-Project Dataset Files and Resources (Optional)
  15. 15. Example: of Data Management Structure Project Sub-Project Dataset Files and Resources Predic've Simula'on Thermomechanical Cell Demonstra'on Ø  Input Ø  Geometry Ø  Mesh Ø  Output Ø  5191b5Ab5c9d.pdf Ø  cell.SLDPRT Ø  cells.SLDASM Ø  Complete.SAT Ø  EVAandStack.SLDPRT Ø  ribbon.SLDPRT Ø  ribbondead.SLDPRT
  16. 16. “Dynamic, sequen6al data, indexed by 6me and loca6on, that is con6nuously being acquired from fielded PV systems.” Time Series Data FTP Site Transla=on Scripts API Data Hub API Public Website Programma*c Access Field Sites Time-Series Database •  Slice and examine performance in *me scales from minutes to years. •  Cross compare system performance. •  System event informa*on. •  Loca*on and climate metadata. •  Weather condi*ons.
  17. 17. The DuraMAT Data Hub h5ps://datahub.duramat.org A_er registering, email: robert.white@nrel.gov include your ins=tu=on or company and which projects you will need access to.
  18. 18. Analy*cs
  19. 19. Data Analy*cs capability Visualize, model, and predict Release open source software •  The data analy=cs thrust is cross- cuang, i.e., we can work with any thrust that needs help •  Help visualize, model, and predict based on your data •  Seeking collabora*ons!
  20. 20. Example project: clear sky detec*on Visualize, model, and predict Release open source software rdtools Data from partners
  21. 21. Clear sky detec*on problem Persistently cloudy, noisy Persistently cloudy, noisy Borderline cases Generally clear Given measured solar irradiance and expected clear sky irradiance, can we automa=cally detect clear vs. cloudy sky periods? •  Important for downstream data analyses – e.g., correctly filtering out cloudy / noisy points can significantly change degrada=on rate calcula=on •  Prior algorithms require tuning based on site and on data resolu=on •  Can we infer a “universal” clear sky model based on past data? •  NSRDB has labeled historical data across en=re U.S. (2 million pixels) every 30 minutes for >15 years! Plenty of data to train a model…
  22. 22. Clear sky detec*on analy*cs x1 x2 x3 … x1 x2 x3 … x1 x2 x3 … x1 x2 x3 … x1 x2 x3 … x1 x2 x3 … + NSRDB clear sky labels ML classifier, e.g., random forest Features used (similar to pvlib) Single point features: •  GHI - GHICS •  GHI’ - GHI’CS •  Time from solar noon Window based metrics calculated every 1 hour •  GHI - GHICS average and standard devia=on •  GHI’ - GHI’CS average and standard devia=on •  GHILL - GHILL CS (difference of line length) Step 2: describe each irradia*on measurement point by a set of numerical features Step 3: train a machine learning algorithm to make predic*ons Step 1: clean data set of unambiguously mislabeled data clear points labeled as “cloudy” in NSRDB – filter from analysis
  23. 23. ML model achieves ~98% accuracy! No data cleaning Data cleaning applied Next steps: •  Evaluate sensi=vity of model to different irradiance measurement intervals •  Run analysis across en=re NSRDB data set and test applicability of a single model across mul=ple geographical loca=ons •  Comparisons of satellite vs ground-based measurements •  Benchmark against exis=ng algorithms as well as expert labeling •  Publish model and accompanying so_ware implementa=on
  24. 24. Currently seeking collabora*ons •  The Data Analy=cs thrust is also looking to extend our experience with =me series and weather data to new applica=ons •  Irradiance forecas=ng •  PV genera=on forecas=ng •  Degrada=on analysis We are also interested in other areas of research in need of data mining, machine learning, and sta=s=cal analysis! Come talk to us at the poster session or during the workshop.
  25. 25. NREL EMN Development Team •  Kris=n Munch •  Nick Wunder •  Chris Webber •  Dave Evenson •  Courtney Pailing DuraMat Data Team •  Ben Ellis •  Birk Jones •  Pedro Perez •  Stephanie Moffi5 •  Kevin Leung •  Jonathan Trinas=c Acknowledgements A Coordinated Team Effort Data Analy@cs •  Ben Ellis •  Mike Deceglie •  NSRDB / PVDAQ teams

×