introduce Istio envoy.

1. ISTIO & ENVOY – Security
For Security Subcommittee

2. Micro Service Architecture - Background
EM
IM1 IM2 IM3
IM4 IM5
Monolithic Application
SLB/ADC/API-Gateway
Internet
Image
Micro Service based
EM
IM1 IM2 IM3
IM4 IM5
Image
ES ES ES
IS1 IS1 IS1 IS1
IS4 IS4 IS5 IS5
SLB/ADC/API-Gateway
Internet
IS2 IS2 IS3 IS3
In monolithic application, entire application is scaled-out, waste of memory, no rolling
upgrades, massive integration time, big maintenance.
Micro Service architecture:
• Each individual module is a micro service
• Each module can scale-out independently
• Typically each micro service is implemented as container

3. Micro Service Architecture - Challenges
ES ES ES
IS1 IS1 IS1 IS1
IS4 IS4 IS5 IS5
SLB/ADC/API-Gateway
Internet
IS2 IS2 IS3 IS3
• Need for load balancing among internal hierarchy
• Special load balancers – Expensive
• Each consumer balancing its connections across producers.
Independent LB decision. May load one instance of producers.
• Special logic at each service.
• Need for discovery of service instances
• Special logic at the each service.
• Need for health checks
• Special logic at each service
• Need for tracing/visibility
• Special logic at each service
• Need for network firewall capability
• Special logic at each service
• Authenticate and Authorize consumers
• Special logic at each service level
• Need for Mutual TLS among producers and consumers.
• Special logic to get certificate enrolled.
• Special logic to initiate/accept TLS connections

4. Micro Service Architecture – Development Challenges
ES ES ES
IS1 IS1 IS1 IS1
IS4 IS4 IS5 IS5
SLB/ADC/API-Gateway
Internet
IS2 IS2 IS3 IS3
• Polyglot
• Any special logic at each service level need to
support multiple language bindings.
• Coordination among all development teams
• Third party micro services (binary) – Can’t put our
special logic there.
• In some instances, it was found that 50% of the
logic is related to non-application specific.
Let application service developers not worry about
micro-service tax.

5. Micro Service Architecture – Service Mesh technology
ES ES ES
IS1
IS1
IS1
IS4 IS4 IS5 IS5
Istio-ingress envoy
Internet
IS2
IS2
IS3
IS3
SC SC SC
SC
SC
SC
SC SC SC SC
SC
SC
SC
SC
ISTIO Control
Plane
Pilot/Mixer/
CA
SC (Side Car) – One for each Service instance (A container in
each POD)
- Envoy proxy
- Service discovery, Load balancing, Failure handling, Circuit
breaking (Limits), Fault injection (for troubleshooting), Health
checks
- Mutual TLS
ISTIO Control plane:
- Pilot:
- Service discovery glue between Envoy and K8S
- Traffic rule configuration
- Security
- Role based Access control (Pluggable RBAC engine,
support for local RBAC, K8S RBAC adapters, but facility
to add new adapters – Example AAF RBAC adapter)
- Certificate Authority

6. ONAP - Non-ISTIO to ISTIO
ONAP
SDC SO OOF SDNC APPC
DCAE MC VFC POLICY DB
MUSIC
Non ISTIO
AAF
ONAP
SDC SO OOF SDNC APPC
DCAE MC VFC POLICY DB
MUSIC
ISTIO with Envoy (CNCF)
• Certificate Enrolment client
• Authentication & Authorization
• Service Discovery/registration
• TLS, Visibility etc…
AAF

7. Service to Envoy proxy – Security Concerns & Mitigation
SVC1
Instance1
Envoy
K8S POD 1
K8S POD2
Linux Kernel
Name space 1
IP Tables Tproxy
Name space 2
SVC2
Instance1
Envoy
K8S POD 1
K8S POD2
Linux Kernel
Name space 1
IP tables tproxy
Name space 2
TLS
ISTIO Pilot
Pilot configures IP Tables to redirect the traffic coming from services to Envoy and Envoy to
services.
No configuration required at the service level to use Envoy as proxy
Envoy acts as transparent proxy. Ensure that Source IP of the connection is maintained on both
sides.
Even though traffic is clear between
instance and side car, it is not considered
a security issue as both side car and
service belong to same POD, hence same
network name space. Clear traffic is
never seen on the wire.

8. ISTIO CA - Features
• Can generate self signed CA root certificate
• Can take externally generated CA certificate
and private key
• Support for certificate chain (CA certificate
signed by Intermediate CA)
• CA redundancy
• Two types of user certificate issue methods
• Native K8S based for K8S orchestrated
services
• Non-K8S orchestrated services
(VMs/Baremetal)
• Certificate renewal
• SPIFFE naming convention
Envoy:
- Auto reload of renewed certificates/keys
- Integration with PKCS11 based security
- Authentication – Mutual TLS
- Authorization - ISTIO RBAC or adapters to
remote RBAC
K8S
Master
ISTIO CA
SVC SC
1. Create Service account and
Deploy service
2. POD
deploy
ment
event
3. Create user
key/certificate
(signed by CA
cert)
4. Send key/cert using
secret mount
ISTIO CA
VM/Baremetal
3. Generate Cert,
Sign using CA key
Node agent 1. Key pair
generation and
CSR
2. Send CSR
4. Send Cert

9. Casablanca – ISTIO work
• Rancher to instantiate ISTIO software components
• Status : Currently manual installation of ISTIO components is done.
• Migration path from NON-ISTIO to ISTIO
• ISTIO Side car injection related changes
• Securing the private keys of ISTIO CA and Envoy private keys.
• Provide option to use either ISTIO RBAC and AAF CA RBAC – A way to choose the option (AAF
CA RBAC is Stretch goal)
• Any glue/tools to simplify the ISTIO deployment
Any glue logic and scripts – Put it as part of MSB
project.
Goal – Entire ONAP using ISTIO all the time (No option to choose).
Fix any gaps to make that happen.

10. HW Security - Casablanca work
ISTIO CA
Go Crypto
Crypto11 (PKCS11
Interface)
Crypto11 (PKCS11
Interface)
SSHSM
TPM /SGX plugin
Hardware
• Secure CA private key
• Distribute CA private key
securely to the Hardware
(TPM/SGX) – Started
conversations with ISITO
security group.
Extend the work done on
AAF CA to ISTIO CA – Note :
Current AAF CA is only
testing purpose only.
Envoy proxy
OpenSSL
LibP11
SSHSM
TPM /SGX plugin
Hardware
• Secure user private key
• Envoy takes care of
RBAC.
• ISTIO RBAC is quite
powerful, but if lacks
features, can add new
adapter to talk to ONAP
RBAC engine (AAF)
Since Mutual TLS/RBAC is implemented in one way (C++ in case of envoy),
provides opportunity to do good job of HW security & Accelerating TLS, thereby
universal security and improving performance

11. Summary
• Service mesh technologies take away the tax of Micro-Service
architecture out of services and consolidate in side cars.
- Less error prone
- Faster development
- Easy maintenance
- Polyglot independent.
• K8S native support
• Security is one big piece of Service mesh
- Enable Mutual TLS
- Enable RBAC
- Enables security without each developer putting effort independently in their
projects
- Provides opportunities to provide HW security and acceleration with one
implementation.

12. Inter working with AAF

13. Overlap items
• CA
- AAF has CA and is being used by some services in Beijing release.
- ISTIO has its own CA.
• RBAC
- AAF has RBAC and some services in Beijing release is already using AAF
RBAC.
- ISTIO has RBAC, which is simpler (Need to see if is good enough to satisfy
ONAP requirements)
• Three choices (Assuming that ISTIO Service mesh is chosen)
• Choice 1: Use ISTIO CA and ISTIO RBAC across all ONAP services (Presented before)
• Choice 2: Use ISTIO CA and use AAF RBAC
• Choice 3: Use ISTIO CA and AAF CA together and use AAF RBAC (Being shown in next slide as it is superset)

14. Choice 3 (Keeping services that use AAF CA )
CA Key Generation Station (Secure
system)
ISTIO CA AAF CA
AAF RBAC
S1 Side car S2
ISTIO Mixer
AAF RBAC Plugin
1
2 2
3 3
4
5 6
5
7
1. ONAP Administrator creates CA key pair and self-
signed CA certificate (or it could be CA key pair
and signed by root/intermediate CA)
2. ONAP Administrator using CA CLI/GUI, upload the
CA private key/certificate and any chain in both
ISTIO CA instances and AAF CA instances.
3. When S1 or S2 is brought up by OOM,
corresponding CA will issue service certificates.
Services get its own key/certificate & CA chain.
4. TLS establishment : Based on peer ID, service (or
its side car) authenticates using CA certificate it
has.
5. Authorization: When there is HTTP operation, its
method, URL and other operation specific
information, requester ID is sent to the RBAC
engine to figure out the roles and permissions.
6. RBAC plugin talks to AAF RBAC to get the
permissions (in case of ISTIO)
7. If permissions are allowed for that operation,
operation proceeds. Otherwise, operation is
denied.
CADI
No implication (based on current understanding) on existing services that are
using CADI for authentication & authorization
Additional RBAC plugin development at the ISTIO is required

15. Recommendation
• Assumptions:
• AAF RBAC is MUST
• Recommendation for default setup
• ISTIO CA
• ISTIO Mixer with AAF RBAC plugin
• Options:
• If some deployment likes to use “ISTIO RBAC”, it should be possible.
• If some deployment likes to use their own RBAC, ISTIO provides that option.

16. s