AGL Solar Exchange
Enabling Kubernetes & Service Mesh
Who we are
Michael Stevenson
Digital Developer AGL Energy :
• Data Ingestion Pipeline for battery telemetry data
• DRED (Demand Response Enabled Device)
• Containerization of AGL Solar Exchange platform
2
AGL Solar Exchange – Peer To Peer (P2P)
Australian based technology consultancy
currently partnered with AGL to enable
container orchestration.
As one of Australia’s biggest energy
providers, we believe we can make the
biggest change.
Melchi Salins
Engineer :
• Cloud Platforms
• Containerization
• Microservices
What are we talking about today?
• What is AGL Solar Exchange
• Application Architecture
• Why Kubernetes?
• Key challenges faced
• Service Mesh
• Demo
Footer | Date (Format: X Month 2017) 3
AGL Solar Exchange – Peer To Peer (P2P)
What is AGL Solar Exchange
A new solar trading platform trial, designed to give Victorians a way to
• trade to save on their electricity bills
• help others save on theirs
• reward households who have gone solar
Users with a solar system can
• sell solar tokens on AGL Solar Exchange
Users without a solar system can participate by:
• buy solar tokens from a participating sellers to earn
potential savings on their AGL electricity bill
• or choose to reward others for going solar
4
AGL Solar Exchange – Peer To Peer (P2P)
Application Architecture
Multiuser containerised prototype application:
Containers: front end (angular), backend api’s (python), workers (python / go - cron / adhoc)
AWS:
EC2 Docker / RDS postgresql
django / python / go / angular / azure blob storage (via master / slave ingestion
app)
log stash / log spout / data dog / sentry.io
bastion hosts / vpn / terrafaorm / circle ci / git hub
Kubernetes:
services / pods / ingresses / jobs (cron / adhoc) / ssl termination / load balancers /
fluent bit
/ elastic search / helm
5
AGL Solar Exchange – Peer To Peer (P2P)
Why Kubernetes
• Disaster recovery – multi region deployment of Kubernetes cluster
• Scalability – cpu / memory / custom metrics
• Cloud Portability
– how realistic is this
– its possible with careful planning of application architecture
• AGL has a large Azure cloud footprint
– first step in the move back to Azure
• Evaluate containerisation technologies vs serverless and other current cloud architecture patters
6
AGL Solar Exchange – Peer To Peer (P2P)
Challenges faced
• Team was new to both Kubernetes / docker / and AWS covered a lot in a short period
• How we came up to speed
‘learn Kubernetes in 3 hours’ a life saver
Managed vs Unmanaged Kubernetes headache or amazing learning tool
Not having access to the managed service meant that we really had to get our hands dirty
• Debugging the application in Kubernetes
viewing logs from pods
bashing into a docker container, an invaluable skill
• Current application not written with Kubernetes in mind
• Challenges surrounding the tightly coupled nature of the existing application
• Strong focus is required round architecture patterns used in code deployed in containers 7
AGL Solar Exchange – Peer To Peer (P2P)
Consul Connect as a
Service Mesh
9Footer | September 2018)
What’s next?
• Services seldom live in isolation
• Security connectivity between applications is
paramount to any enterprise
• Transitioning into the cloud brings focus on
Services and not on Hosts
• Enabling cloud agnostic capability
10Footer | September 2018)
What to take into consideration
Battle
Tested
Kubernetes Security Light
weight &
easy to use
Enterprise Support Extensibility Multi Cloud Capability
11Footer | September 2018)
Consul
Consul is a distributed service mesh to
connect, secure, and configure services
across any runtime platform and public or
private cloud
Consul is a stable Enterprise ready Service
Mesh that
12Footer | September 2018)
Consul Native Support for Kubernetes
Kubernetes Service resources and
recognized and automatically imported
into Consul and is made available to
services not running in Kubernetes
Consul Connect automatically injects Envoy sidecar
to any Pod in an easy annotation driven manner.
Envoy is a feature rich & extremely performant
proxy
13Footer | September 2018)
Demo
https://www.youtube.com/watch?v=wzRU6FwXlG4

Consul connect

  • 1.
    AGL Solar Exchange EnablingKubernetes & Service Mesh
  • 2.
    Who we are MichaelStevenson Digital Developer AGL Energy : • Data Ingestion Pipeline for battery telemetry data • DRED (Demand Response Enabled Device) • Containerization of AGL Solar Exchange platform 2 AGL Solar Exchange – Peer To Peer (P2P) Australian based technology consultancy currently partnered with AGL to enable container orchestration. As one of Australia’s biggest energy providers, we believe we can make the biggest change. Melchi Salins Engineer : • Cloud Platforms • Containerization • Microservices
  • 3.
    What are wetalking about today? • What is AGL Solar Exchange • Application Architecture • Why Kubernetes? • Key challenges faced • Service Mesh • Demo Footer | Date (Format: X Month 2017) 3 AGL Solar Exchange – Peer To Peer (P2P)
  • 4.
    What is AGLSolar Exchange A new solar trading platform trial, designed to give Victorians a way to • trade to save on their electricity bills • help others save on theirs • reward households who have gone solar Users with a solar system can • sell solar tokens on AGL Solar Exchange Users without a solar system can participate by: • buy solar tokens from a participating sellers to earn potential savings on their AGL electricity bill • or choose to reward others for going solar 4 AGL Solar Exchange – Peer To Peer (P2P)
  • 5.
    Application Architecture Multiuser containerisedprototype application: Containers: front end (angular), backend api’s (python), workers (python / go - cron / adhoc) AWS: EC2 Docker / RDS postgresql django / python / go / angular / azure blob storage (via master / slave ingestion app) log stash / log spout / data dog / sentry.io bastion hosts / vpn / terrafaorm / circle ci / git hub Kubernetes: services / pods / ingresses / jobs (cron / adhoc) / ssl termination / load balancers / fluent bit / elastic search / helm 5 AGL Solar Exchange – Peer To Peer (P2P)
  • 6.
    Why Kubernetes • Disasterrecovery – multi region deployment of Kubernetes cluster • Scalability – cpu / memory / custom metrics • Cloud Portability – how realistic is this – its possible with careful planning of application architecture • AGL has a large Azure cloud footprint – first step in the move back to Azure • Evaluate containerisation technologies vs serverless and other current cloud architecture patters 6 AGL Solar Exchange – Peer To Peer (P2P)
  • 7.
    Challenges faced • Teamwas new to both Kubernetes / docker / and AWS covered a lot in a short period • How we came up to speed ‘learn Kubernetes in 3 hours’ a life saver Managed vs Unmanaged Kubernetes headache or amazing learning tool Not having access to the managed service meant that we really had to get our hands dirty • Debugging the application in Kubernetes viewing logs from pods bashing into a docker container, an invaluable skill • Current application not written with Kubernetes in mind • Challenges surrounding the tightly coupled nature of the existing application • Strong focus is required round architecture patterns used in code deployed in containers 7 AGL Solar Exchange – Peer To Peer (P2P)
  • 8.
    Consul Connect asa Service Mesh
  • 9.
    9Footer | September2018) What’s next? • Services seldom live in isolation • Security connectivity between applications is paramount to any enterprise • Transitioning into the cloud brings focus on Services and not on Hosts • Enabling cloud agnostic capability
  • 10.
    10Footer | September2018) What to take into consideration Battle Tested Kubernetes Security Light weight & easy to use Enterprise Support Extensibility Multi Cloud Capability
  • 11.
    11Footer | September2018) Consul Consul is a distributed service mesh to connect, secure, and configure services across any runtime platform and public or private cloud Consul is a stable Enterprise ready Service Mesh that
  • 12.
    12Footer | September2018) Consul Native Support for Kubernetes Kubernetes Service resources and recognized and automatically imported into Consul and is made available to services not running in Kubernetes Consul Connect automatically injects Envoy sidecar to any Pod in an easy annotation driven manner. Envoy is a feature rich & extremely performant proxy
  • 13.
    13Footer | September2018) Demo https://www.youtube.com/watch?v=wzRU6FwXlG4