This document discusses technical challenges and potential solutions for developing community microgrids. It describes MIT Lincoln Laboratory's Hardware-in-the-Loop (HILLTOP) testbed, which can be used to test microgrid controllers in real-time simulation. The testbed integrates simulated generators, energy storage systems, and other devices to evaluate controllers before full deployment. Industry feedback supports the potential for HILLTOP and similar open-source testbeds to accelerate microgrid development by reducing costs and risks.

1. MIT Lincoln Laboratory
This work is sponsored by the Department of Energy, Office of Electricity Delivery and Energy Reliability under Air Force Contract #FA8721-05-C-0002. Opinions,
interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.
Erik Limpaecher
Microgrid Hardware-in-the-Loop
Laboratory Testbed and Open Platform
(HILLTOP)
August 30, 2016

2. Community mGrids - 2
KN 30 August 2016
Status Quo:
Single-customer Microgrids
• Only economical and practical for large utility customers
– University campuses
– Hospitals
– Prisons
– Department of Defense bases
Santa Rita Jail
CERTS microgrid controller
Princeton University
15 MW cogen & thermal storage
Illinois Institute of Technology
high reliability distribution system

3. Community mGrids - 3
KN 30 August 2016
Community Microgrids
Single-Customer
Microgrids
Graphic credit: EPRI
Challenges
• Technical
1. Regional resilience analysis: site selection
2. Microgrid site design
3. Economical systems integration
4. Safe grid interconnection
5. Stable islanded operation
• Policy
– Coordination with public utilities commission
– Framework for utility involvement
– Financial pro forma for real estate developers
Community
Microgrids
• Many recent studies…no actual deployments
– Sandia “Advanced Microgrids” study (Mar 2014)
– “Microgrids …for the Commonwealth of Massachusetts” (Feb 2014)
– “Minnesota Microgrids” (Sep 2013)

4. Community mGrids - 4
KN 30 August 2016
Community Microgrids
Single-Customer
Microgrids
Graphic credit: EPRI
Challenges
• Technical
1. Regional resilience analysis: site selection
2. Microgrid site design
3. Economical systems integration
4. Safe grid interconnection
5. Stable islanded operation
• Policy
– Coordination with public utilities commission
– Framework for utility involvement
– Financial pro forma for real estate developers
Community
Microgrids
• Many recent studies…no actual deployments
– Sandia “Advanced Microgrids” study (Mar 2014)
– “Microgrids …for the Commonwealth of Massachusetts” (Feb 2014)
– “Minnesota Microgrids” (Sep 2013)

5. Community mGrids - 5
KN 30 August 2016
• High NRE to integrate systems in the field
– One vendor’s microgrid controller quote: $1M starting price
• Risk of damage to expensive equipment
– One utility-deployed microgrid: 1 year of controls testing, damaged
a 750 kW transformer, required significant engineering staff support
• Interconnection behavior unknowable to utility engineers
– Controls are implemented in proprietary software
– Microgrids are a system of systems: Exhibit emergent behavior
• “Vaporware”
– No standard list of functions or performance criteria
– Difficult to validate marketing claims
• No standards verification
– IEEE P2030.7 and P2030.8 standards are on the horizon
How Do We Accelerate Microgrid Deployment?
Reduce Integration Time, Cost, & Risk

6. Community mGrids - 6
KN 30 August 2016
Types of Power System Testbeds
Full System
Inv G
C C
Inv
Power Testbed
G
C C
Power HIL
Inv
C
G
C
Controller HIL
C
GInv
C
Simulation
GInv
CC
mC mC mC
DMS DMS
Legend
G generator
Inv battery or solar inverter
C device controller
mC microgrid controller
DMS distribution management system controller
power grid
high-bandwidth AC-AC converter
simulation or emulation boundary
hardware
virtual (simulated or emulated)
Matlab
SimPowerSystems
simulation
(not real-time)
Florida State CAPS
facility
Princeton University
cogeneration plant
ORNL DECC
Microgrid Lab
MIT-LL HILLTOP
System

7. Community mGrids - 7
KN 30 August 2016
Power Distribution
Integration Platforms and Testbeds
Full System
Inv G
C C
Inv
Power Testbed
G
C C
Power HIL
Inv
C
G
C
Controller HIL
C
GInv
C
Simulation
GInv
CC
mC mC mC
DMS DMS
Legend
G generator
Inv battery or solar inverter
C device controller
mC microgrid controller
DMS distribution management system controller
power grid
high-bandwidth AC-AC converter
simulation or emulation boundary
hardware
virtual (simulated or emulated)
Simulation
Controller
HIL
Power
HIL
Power
Testbed
Full
System
TestCoverage
Test Fidelity
Low
Low High
High
low cost ($10k-100k)
moderate cost ($100k-300k)
high cost ($500k-$millions)

8. Community mGrids - 8
KN 30 August 2016
Power Distribution
Integration Platforms and Testbeds
Full System
Inv G
C C
Inv
Power Testbed
G
C C
Power HIL
Inv
C
G
C
Controller HIL
C
GInv
C
Simulation
GInv
CC
mC mC mC
DMS DMS
Legend
G generator
Inv battery or solar inverter
C device controller
mC microgrid controller
DMS distribution management system controller
power grid
high-bandwidth AC-AC converter
simulation or emulation boundary
hardware
virtual (simulated or emulated)
low cost
moderate cost
high cost
Simulation
Controller
HIL
Power
HIL
Power
Testbed
Full
System
TestCoverage
Test Fidelity
Low
Low High
High

9. Community mGrids - 9
KN 30 August 2016
10-5
10-4
10-3
1 ms
10-2
10-1
100
1 s
Time (seconds)
HIL simulation
rate
80 mS
(12.5 kHz)
One AC
cycle
16.7 ms
(60 Hz)
User display
update rate
66.7 ms
(15 Hz)
Load profile &
irradiance data
1 s
(1 Hz)
Power converter
controller response
0.5-1 ms
(1-2 kHz)
Secondary
control
0.1-1 s
(1-10 Hz)
Power system
fault transients
0.3-1 ms
(1-3 kHz)
Genset protection
functions
0.1-0.2 s
(50-100 Hz)
• Microgrid controller HIL simulates in real-time at sub-cycle timescales
Wide Test Coverage with
Real-time Simulation
Useful for:
• Steady-state
• Dynamic analysis
• Transient analyses

10. Community mGrids - 10
KN 30 August 2016
2016 HILLTOP Block Diagram
Modbus TCP
Communication Translation
Hardware / Firmware
COM
Microgrid Controller (Unit Under Test)
MIT-LL
Interface Box
Vendor-supplied equipment
Modbus RS485
Woodward
EasyGen 3500
#1
COM DIO AIO COM DIO AIO
Simulated
Diesel Genset
Simulated
Combined Heat and
Power System
Simulated
Battery Storage &
Power Converter
Simulated Breakers
and Loads
Simulated Distribution Grid
and Microgrid Test Feeder
Simulated
PV & Inverter
COM DIO AIOCOM DIO AIOCOM DIO AIO
Simulated
Generic
Relays
Woodward
EasyGen 3500
#2
Modbus RS485
Firewall and Network Switch
Lantronix
Intellibox 2100
TCP to RS485
Lantronix
Intellibox 2100
TCP to RS485
SEL-751 Relays
(qty 3)
Simulated
Load Meters
EPC Power
Electronics
Controller #1:
Storage
EPC Power
Electronics
Controller #2:
PV
Modbus TCP
MIT-LL
Interface Box
OPAL-RT and
Typhoon HIL
Real-time Simulation
Platforms
Microgrid
Controller
Mechanical
Enclosure
Cyber Test
C2 Proxy

11. Community mGrids - 11
KN 30 August 2016
2015 & 2016 Symposium Test Feeder
FY15 Symposium FY16 Symposium

12. Community mGrids - 12
KN 30 August 2016
Microgrid Controller
Hardware-in-the-Loop Platform
MIT Lincoln Lab
Windows Server
Power Supply
Interface Box
Monitoring I/O
Analog & Digital
Opal-RT HIL
Target
Firewall and
Network Switch
Console
Woodward
easYGens
Integrated Woodward easYgen 3000
genset controllers, EPC power converter
controllers, SEL 751 relays, and SEL 2440
fast load-shed controllers

13. Community mGrids - 13
KN 30 August 2016
• Generator statistics
• High speed instrumentation
– 8kHz sampling
– Voltage and current
– Bias signals
CHIL Integration:
Diesel Genset Model Validation
Shutdown of Generator and ModelStart-up of Generator and Model
• 1 MVA
• 480Vac
• 3 phase

14. Community mGrids - 14
KN 30 August 2016
Device Controller Integration:
Woodward easYgen 3000
Interface Box
Woodward
easYgen
3000
Controller
Genset Simulation in HIL
Governor
AVR
M
MCBGCB
Load
16 Vac 16Vac
Vabc
Iabc
16 Vac
120Vac
0-1 A
120 VAC
Feeder
Tachometer
Vabc
16 Vac
0-1 A
Ia
F_ref:
60 Hz
V_ref:
480 VLL
±5V square wave
Throttle
Field
G
Legend
M Motor
G Generator
GCB Generator Circuit Breaker
MCB Mains Circuit Breaker
Signal voltage transformer
Voltage-controlled current source

15. Community mGrids - 15
KN 30 August 2016
2015 Microgrid Controls Symposium
Boston, MA

16. Community mGrids - 16
KN 30 August 2016
Microgrid Operating in Real-time

17. Community mGrids - 17
KN 30 August 2016
Microgrid Controller Symposium
Save the Date: Thursday February 16, 2017 at MIT
• Google search for “Eventbrite: Microgrid & DER Controller Symposium”
• Hosted by Mass. Clean Energy Center and DOE-OE
– Utilities, project developers, and microgrid vendors
– MIT Campus
– Agenda is posted
• Integrating 6 microgrid controller vendors
• Demonstrations of several HIL testbeds

18. Community mGrids - 18
KN 30 August 2016
Vision for the
Microgrid Controller HIL Platform
– 1 –
Development
Platform
– 2 –
Deployment
Platform
– 3 –
Standards Test
Platform
• Application of real-time sim. technology to power engineering
• Cost-effective engineering and project development
• Enables performance evaluation of commercial products
• Demonstrations at Mass. Microgrid Controls Symposium
• Perform controller and systems integration
• Pre-commission testing of advanced power system projects
• Test edge conditions and exercise the actual device controllers
• Technical risk reduction and confidence building for the utility
• Project enabler: South Boston microgrid
• Industry-standard test platform for new power systems
• Test against IEEE P2030.8 standard and utility requirements
– 4 –
Electric Power HIL Controls Collaborative (EPHCC) Shared Repository

19. Community mGrids - 19
KN 30 August 2016
Elements of the HIL
Shared Repository
Microgrid Test Feeders
Validated Device Models Validated Device Controller
Software
Interface Circuitry for
Device Controllers
Interface Code for
Device Controllers
Standard Test Stimuli
Communications Interface
Translation Code
Post-processing Scripts for
Test Results
HIL Target Platform
Conversion Scripts
Netlists
Load profiles, irradiance profiles,
grid outages, faults
Circuit schematics, bills of materiel
Targets: OPAL-RT, Typhoon HIL,
RTDS, NI, and others
Motor-generators, power
converters / inverters, and relays
Genset controllers, power
converter controllers, relay
protection functions
Modbus
TCP
Microgrid Test
Repository
Controller-in-
the-Loop
Repository
HIL Platform
Repository
50
51
27
59
25
MODBUS
52
IEC 61850
GOOSE
MMS

20. Community mGrids - 20
KN 30 August 2016
Repository Structure
• Goals:
– Multi-users
– Multi-platform
• Directory structure designed
for other simulation
platforms
• Unit tests for checking
compatibility across tools
• Documentation
Distribution
Systems
Sheriff
Banshee
Components
Cross-platform
Translation Tools
Cable
Transformer
Passive Load
Active Load
Circuit
Breaker
Motor
Relay
Genset
PV
CHP/
Thermal
ESS
Bullfrog
Root
Device
Model
Software
Controller
ABB CHIL
EPC CHIL
Vendor-X
CHIL
· Model
· Stimuli
· Post-Processing Scripts
· Model
· Stimuli
· Post-Processing Scripts
· Validation
· Unit Test
· Validation
· Unit Test

21. Community mGrids - 21
KN 30 August 2016
• Component models, distribution test feeders, and C-HIL setups
are documented, tested, and publicly available
– https://github.com/PowerSystemsHIL/
• Working with DOE’s GMLC-OL program as pilot effort to hone
organization and working processes
Repository is Publicly Available

22. Community mGrids - 22
KN 30 August 2016
Division 7
Scott van Broekhoven
Ed Corbett
Maura Custar
Elizabeth Dalli
Jason LaPenta
Erik Limpaecher
Raajiv Rekha
Bill Ross
Reynaldo Salcedo
Tammy Santora
Chris Smith
MIT
Kendall Nowocin
Summer Interns
Matt Backes, UIUC
Aidan Dowdle, MIT
Marissa Garcia, USC
Division 4
Joe Cooley
Division 6
Igor Pedan
Sejal Popat
Contributors
Collaborators
Allan Abela, EPC Power
Vijay Bhavaraju, Eaton
Ivan Celanovic, Typhoon HIL
Fran Cummings, Peregrine Group
Babak Enayati, National Grid
Mark Evlyn, Schneider
Julie Fortin-Morin, OPAL RT
Thomas Kirk, OPAL RT
Scott Manson, Schweitzer
Mirjana Marden, ARPA E
Galen Nelson, MassCEC
Luis Ortiz, Anbaric
Jim Reilly, Reilly Associates
Todd Snider, EPC Power
Travis Sheehan, Boston Redev. Authority
Tom Steber, Schneider
Brad Swing, City of Boston

23. Community mGrids - 23
KN 30 August 2016
Erik Limpaecher
Assistant Group Leader
Energy Systems, Group 73
781-981-4006 (lab)
781-999-2237 (cell)
elimpaecher@ll.mit.edu

24. Community mGrids - 24
KN 30 August 2016
BACK-UP SLIDES

25. Community mGrids - 25
KN 30 August 2016
Technical Challenges for
Community Microgrids
Industrial
load
Battery
storage
Residential
load
Substation
Microgrid
Controller
Distribution
Utility
DMS
Regional
Transmission
Operator
CHP plant 1
CHP plant N
Critical load
Solar
array
= controller
Community microgrid =>
Stable off-grid operationEconomical asset integration Safe interconnectionRequirements
Minimize cost,
eliminate batteries
Integration of multiple assets
owned by separate entities
No low-level controlChallenge
Reference design: mix of
fast-responding
generation assets
Open controller standard,
HIL testbed
Islanding detection
using only sensing
S&T solution
FY 16 Focus FY 16 Focus

26. Community mGrids - 26
KN 30 August 2016
HILLTOP Collaborations Efforts
San Nicolas Island

27. Community mGrids - 27
KN 30 August 2016
Industry Feedback on the
Electric Power HIL System
“It would reduce our risks and costs
considerably…
The ability to perform commissioning of a
microgrid’s control systems prior to
construction would also reduce the
distribution utility’s perceived technical risk,
thereby increasing our project success rate
and reducing deployment timelines and
costs.”
– Dr. Luis Ortiz, Anbaric Microgrid
“An open source HIL library would accelerate
the entire industry, but the industry isn’t
organized well enough to make this happen.
This is a perfect role for government. We
would contribute engineering support to such
an effort.”
– Schneider Electric executive

28. Community mGrids - 28
KN 30 August 2016
2015 HILLTOP Block Diagram
Modbus TCP
Modbus TCP
Modbus TCP
Connection to HIL
Demonstration Platform
COM
Microgrid Controller – Unit Under Test
Interface Box
Vendor-supplied equipment
Modbus RS485
Prime Mover
Device Controller
Woodward
EasyGen 3500
#1
OPAL-RT HIL 5607
Power
COM DIO AIO COM DIO AIO
Simulated
4 MVA Genset
Simulated
1 MVA Genset
Simulated
Battery Storage &
Power Converter
Simulated Relays,
Breakers, and
Telemetry
Simulated Grid and One Line
Diagram of the Test Feeder
(~18 Buses and 17 lines)
Simulated
PV & Inverter
COM DIO AIO COM DIO AIOCOM DIO AIOCOM DIO AIO
Simulated Battery
Power Converter
Controller
Simulated PV Inverter
Controller
Simulated Protection
Controller
Prime Mover
Device Controller
Woodward
EasyGen 3500
#2
Modbus RS485
Firewall and Network Switch
Modbus TCP
Lantronix
Intellibox 2100
TCP to RS485
Lantronix
Intellibox 2100
TCP to RS485

29. Community mGrids - 29
KN 30 August 2016
Repo. Category Fidelity Verification Component Type Dev. Environment Real-time Platform
SW device model 0 - n/a 0 - "pre-alpha," untested Cable Matlab 2011b, SimPwrSys OPAL-RT
SW controller 1 - steady-state 1 - "beta," unverified Breaker Matlab 2014, SimPwrSys Typhoon HIL
C-HIL interface 2 - dynamic 2 - textbook implementation Relay Typhoon RTDS
P-HIL interface 3 - transient 3 - agrees with literature Other LabVIEW National Instruments
Test feeder 4 - unit tested Transformer - 3ph RSCAD Speedgoat
5 - field tested Transformer < 3ph dSPACE
Load
DG - Power Electronics
DG - Machines
Component Inventory Index
• Work continues to identify additional key components and
additional industry partners

30. Community mGrids - 30
KN 30 August 2016
Industry Feedback and
FY16 HIL Development Priorities
Blue: features
considered for 2nd
Symposium
Attendee
Priority
Devel.
Priority
Development Status
1 1 Test under fault conditions
7 1 Self-healing and distribution system configurability
14 1 Test more system topologies
3 1 Integration of more DER device controllers
7 5 More formal testing / more rigorous analysis and
metrics
9 5 Compare vendors’ performance
6 5 Exercise cyber security
9 8 Integration of more microgrid controllers / vendors
2 9 Islanded operation under more dynamic conditions
9 9 More extensive demonstration sequence / functionality
4 - Integration with distribution management system (DMS)
4 - Test against industry standard(s)
9 - Support additional communication protocols
13 - Integration with ISO energy market
Not started
In progress
Done/tested
Not planned

31. Community mGrids - 31
KN 30 August 2016
Heads-up Display (screen 1)

32. Community mGrids - 32
KN 30 August 2016
CHIL Integration ‒ SEL 751 Relay
• SEL Collaboration
• Three feeder protection relays
implemented in hardware
• Hardware I/O interface to
PTs, CT’s, breakers
• Protection features:
– Overcurrent (50, 51)
– Over/under voltage (27, 59)
– Synchronism check (25)
– Grid-tied protection
– Islanded protection
• Modbus TCP interface
OPAL-RT INTERFACE WIRING DIAGRAM
Protection
Functions
Data Logging
Registers
Mapping
Preliminary
Calculations

33. Community mGrids - 33
KN 30 August 2016
• Battery and PV Inverter to
manufacturer validated
modelled inverter
• Inverter features:
– Grid Forming
– Grid Following
• Modbus over RS485
CHIL Integration – EPC Power Inverter
EPC Inverter User InterfaceManufacturer Validated Model

34. Community mGrids - 34
KN 30 August 2016
“…educating stakeholders.” – SoCalEdison engineer
“…test[ing] our project concepts.” – Celtic Energy
“…demonstrat[ing] standard functions and develop[ing] new
functionalities.” – ABB engineer
“…certification for utility interconnection.”
“…pre-commissioning testing.”
“…evaluating the safety and reliability impact on our distribution
system of NY REV projects.”
– ComEd engineer
“DER vendors won’t give us their controller code to evaluate
interconnection behavior. This platform could finally solve that
problem.” – National Grid engineer
“…pre-deployment testing of our self-healing distribution system
project, which includes more than 100 intelligent relays.”
– VP of Engineering, Eversource Energy
The microgrid controller HIL platform would
be helpful to my organization for…