Downloaded 16 times
![FROM alpine:3.3
# Install just the Python runtime (no dev)
RUN apk add --update
python
py-pip
&& rm -rf /var/cache/apk/*
ENV PORT 5000
EXPOSE $PORT
# Set up a working folder and install the pre-reqs
WORKDIR /app
ADD requirements.txt /app
RUN pip install -r requirements.txt
# Add the code as the last Docker layer because it changes the most
ADD static /app/static
ADD service.py /app
# Run the service
CMD [ "python", "service.py" ]
More Advanced Dockerfile
22](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-25-2048.jpg)
![What’s in a CVE?
37
{
"id": "78de8449-313e-44c9-90b1-ae8277b12f95",
"scan_time": 1510759001,
"status": "WARN",
"vulnerabilities": [
{
"type": "vulnerable_package",
"description": "busybox 1.24.1-r7 has vulnerabilities",
"corrective_action": "Upgrade to busybox 1.24.2-r1",
"fixes": [
{
"cve_ids": "CVE-2016-6301",
"summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to
cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.",
"notice": "",
"meta": {
"usn_id": ""
}
}
],
"meta": {
"package_name": "busybox"
"fix_version": "1.24.2-r1",
"installed_version": "1.24.1-r7",
}
},](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-44-2048.jpg)
![What’s in a CVE?
37
{
"id": "78de8449-313e-44c9-90b1-ae8277b12f95",
"scan_time": 1510759001,
"status": "WARN",
"vulnerabilities": [
{
"type": "vulnerable_package",
"description": "busybox 1.24.1-r7 has vulnerabilities",
"corrective_action": "Upgrade to busybox 1.24.2-r1",
"fixes": [
{
"cve_ids": "CVE-2016-6301",
"summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to
cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.",
"notice": "",
"meta": {
"usn_id": ""
}
}
],
"meta": {
"package_name": "busybox"
"fix_version": "1.24.2-r1",
"installed_version": "1.24.1-r7",
}
},
This is how we know what to patch](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-45-2048.jpg)
![Generated Dockerfile for Alpine
38
#
# Remediation Service
#
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1
#
# Patching Vulnerabilities
#
RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;
#
# Fixing Compliance Issues
#
RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi;
# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-46-2048.jpg)
![Generated Dockerfile for Alpine
38
#
# Remediation Service
#
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1
#
# Patching Vulnerabilities
#
RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;
#
# Fixing Compliance Issues
#
RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi;
# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.
References the original image](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-47-2048.jpg)
![Generated Dockerfile for Alpine
38
#
# Remediation Service
#
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1
#
# Patching Vulnerabilities
#
RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;
#
# Fixing Compliance Issues
#
RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi;
# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.
References the original image
Updates the libraries that are vulnerable](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-48-2048.jpg)
![Generated Dockerfile for Alpine
38
#
# Remediation Service
#
FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1
#
# Patching Vulnerabilities
#
RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi;
RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi;
#
# Fixing Compliance Issues
#
RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi;
# NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.
References the original image
Updates the libraries that are vulnerable
Fixes known compliance issues](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-49-2048.jpg)
![Generated Dockerfile for Ubuntu
39
#
# Remediation Service
#
FROM registry.ng.bluemix.net/rofrano/counter:latest
MAINTAINER Vulnerability Remediator 1.0
#
# Patching Vulnerabilities
#
RUN apt-get install -y bash=4.3-7ubuntu1.7;if [ $? -ne 0 ]; then apt-get install -y bash; fi;
RUN apt-get install -y libexpat1=2.1.0-4ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y libexpat1; fi;
RUN apt-get install -y libffi6=3.1~rc1+r3.0.13-12ubuntu0.2;if [ $? -ne 0 ]; then apt-get install -y libffi6; fi;
RUN apt-get install -y libgcrypt11=1.5.3-2ubuntu4.5;if [ $? -ne 0 ]; then apt-get install -y libgcrypt11; fi;
RUN apt-get install -y libgnutls26=2.12.23-12ubuntu2.8;if [ $? -ne 0 ]; then apt-get install -y libgnutls26; fi;
RUN apt-get install -y libssl1.0.0=1.0.1f-1ubuntu2.23;if [ $? -ne 0 ]; then apt-get install -y libssl1.0.0; fi;
RUN apt-get install -y libtasn1-6=3.4-3ubuntu0.5;if [ $? -ne 0 ]; then apt-get install -y libtasn1-6; fi;
RUN apt-get install -y login=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y login; fi;
RUN apt-get install -y passwd=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y passwd; fi;
RUN apt-get install -y sudo=1.8.9p5-1ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y sudo; fi;](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-50-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-59-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
Get the endpoint to be called](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-60-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
Get the endpoint to be called
Get parameters passed in](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-61-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
Get the endpoint to be called
Get parameters passed in
Construct body of request
for microservice to call](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-62-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
Get the endpoint to be called
Get parameters passed in
Construct body of request
for microservice to call
Make the call to fire off the microservice](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-63-2048.jpg)
![Example Python Cloud Function
47
def main(params):
""" Container Redeploy Function """
# Get the API key and use it in the headers
headers = get_headers(params)
api_endpoint = params.get('API_ENDPOINT')
value = params['messages'][0]['value'] # process messages
namespace = value.get('namespace', 'default') # check parameters
# create the message body
body = {
'namespace': namespace,
'old_image': value['old_image'],
'new_image': value['new_image']
}
logging.info('Redeploying: %s', value['old_image'])
result = requests.post(api_endpoint,
data=json.dumps(body),
headers=headers)
message = result.json()
return {'result': message}
Get the endpoint to be called
Get parameters passed in
Construct body of request
for microservice to call
Make the call to fire off the microservice
Return the microservice output](https://image.slidesharecdn.com/ieee-microservices-rofrano-180709160548/75/Microservices-How-loose-is-loosely-coupled-64-2048.jpg)
More Related Content
![Vagrant + Docker provider [+Puppet]](https://cdn.slidesharecdn.com/ss_thumbnails/npoggivagrant-docker-140725100147-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?
- 1. Microservices … how looseis loosely coupled? John J Rofrano SeniorTechnical Staff Member IBMT.J.Watson Research Center @johnrofrano 1
- 2. About me… • SeniorTechnicalStaff Member / Software Engineer @ IBM (34 years) • DevOps Champion @ IBM Research • Adjunct Professor @ NewYork University (NYU) Courant Institute John J Rofrano 2 CSCI-GA 2820 - DevOps and Agile Methodologies @johnrofrano
- 3. What am Igoing to cover? • Overview of Microservices • Overview of Docker Containers • Why Docker Containers becomeVulnerable • How I built a ContainerVulnerability Service using loosely coupled microservices 3
- 4.
- 5. Evolution of AppDesigns C, C++ Code and Manual Test Monolithic Programs Bare Metal Java, J2EE Manual Deployment Tiered Applications Virtual Machines Ruby on Rails, Python Django Automated Testing Service Oriented, RESTful Platform As A Service Node, Angular, Swift, Whisk Automated Deployment Stateless Microservices Docker Containers 5
- 6.
- 7. What Are Microservices? "…themicroservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.These services are built around business capabilities and independently deployable by fully automated deployment machinery." - James Lewis and Martin Fowler https://www.martinfowler.com/microservices/#what 7
- 8. Microservice Architecture • Anarchitecture style aimed to achieve flexibility, resiliency and control, based on the following principles: • Single Purpose Services that are Loosely Coupling with a Bounded Context • Independent life cycle: developed, deployed and scaled... and hopefully, fail independently • Design for resiliency and owns it’s own data • Polyglot — independent code base • Built by autonomous teams with end-to-end responsibility, doing Continuous Delivery • Communicates with other services over a well defined API 8
- 9. Monolithic vs Microservices Web/ Presentation (Apache/Nginx) eCommerce Application (WebSphere/Tomcat/PHP/Django) Database (DB2, MySQL, PostgreSQL) Customers Shopping Cart Promotions Orders Relational Store User Interface Service Catalog NoSQL Microservices-based ApplicationsMonolithic Applications • Tightly coupled • Mixed Concerns • Large Deployment units • Hard to Scale • Long release cycles • Slow on-boarding for new developers • Slower feedback loop • Loosely coupled • Minimal responsibility per service • Small Deployment units • Easy to Scale • Short release cycles • Fast on-boarding for new developers • Develop quickly with fast feedback 9
- 10. Conway’s Law • Anyorganization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. — Melvin Conway, Datamation, 1968 • e.g., if you ask an organization with 4 teams to write a compiler… you will get a 4-pass compiler! http://www.melconway.com/Home/Conways_Law.html 10
- 11. Monolithic Organization 11 Organized aroundtechnology User Interface Team Application Logic Team Database (DBA) Team Web Tier Database App Tier Organization Structure Application Structure
- 12. Microservices Organization 12 Organized aroundBusiness Domains Account Team Login Registration Users Personalization Team Warehouse Team Personalization Inventory Shipping Receiving
- 13. How do youdeploy all these Microservices? 13
- 14. TheThree Pillars ofSoftware Agility DevOps Cultural Change Automated Pipeline Everything as Code Immutable Infrastructure Loose Coupling/Binding RESTful APIs Designed to resist failures Test by break / fail fast Portability Developer Centric Ecosystem enabler Fast startup Microservices Containers (Docker) AGILITY DevOps Microservices Containers (Docker) 14
- 15.
- 16. What is Docker? •Docker is a light-weight container service that runs on Linux • File system overlay • One Process Space • One Network Interface • Shares the Linux kernel • Containers encapsulate a run-time environment • Your code, libraries, etc. • Almost no overhead • Containers spin up in milliseconds • Native performance because there is no emulation • Package only what you need 16
- 17. Benefits of Containers •Great isolation • Great manageability • Container encapsulates implementation technology • Efficient resource utilization • Fast deployment 17
- 18. How is itdifferent fromVirtual Machines? BINS/LIBS BINS/LIBS BINS/LIBS GUEST OSGUEST OSGUEST OS HYPERVISOR HOST OPERATING SYSTEM INFRASTRUCTURE APP 1 APP 2 APP 3 BINS/LIBS BINS/LIBS BINS/LIBS DOCKER ENGINE HOST OPERATING SYSTEM INFRASTRUCTURE APP 1 APP 2 APP 3 • Virtual Machines are heavy-weight emulations of real hardware • Containers are light-weight like a process • The app looks like it’s running on the Host OS VIRTUAL MACHINES DOCKER CONTAINERS 18
- 19. How is itdifferent fromVirtual Machines? BINS/LIBS BINS/LIBS BINS/LIBS GUEST OSGUEST OSGUEST OS HYPERVISOR HOST OPERATING SYSTEM INFRASTRUCTURE APP 1 APP 2 APP 3 BINS/LIBS BINS/LIBS BINS/LIBS DOCKER ENGINE HOST OPERATING SYSTEM INFRASTRUCTURE APP 1 APP 2 APP 3 • Virtual Machines are heavy-weight emulations of real hardware • Containers are light-weight like a process • The app looks like it’s running on the Host OS VIRTUAL MACHINES DOCKER CONTAINERS VM VM VM Container Container Container 18
- 20. Docker @ 20,000feet 19
- 21. Images, Layers, andCopy on Write • Each Docker image references a list of read- only layers that represent filesystem differences • Layers are stacked on top of each other to form a base for a container’s root filesystem • When you create a new container, you add a new, thin, writable layer on top of the underlying stack • All changes made to the running container - such as writing new files, modifying existing files, and deleting files - are written to this thin writable container layer 20
- 22. Creating an Imagefrom a Dockerfile • We can create a Dockefile to add our own content to the image • Create a file called Dockerfile and add the following two lines: • Build it with: docker built -t my_image . FROM nginx:alpine COPY content /usr/share/nginx/html 21
- 23. Creating an Imagefrom a Dockerfile • We can create a Dockefile to add our own content to the image • Create a file called Dockerfile and add the following two lines: • Build it with: docker built -t my_image . FROM nginx:alpine COPY content /usr/share/nginx/html Start FROM the nginx image that’s in Docker Hub 21
- 24. Creating an Imagefrom a Dockerfile • We can create a Dockefile to add our own content to the image • Create a file called Dockerfile and add the following two lines: • Build it with: docker built -t my_image . FROM nginx:alpine COPY content /usr/share/nginx/html Start FROM the nginx image that’s in Docker Hub COPY the folder called 'content' to '/usr/share/nginx/html' inside the container 21
- 25. FROM alpine:3.3 # Installjust the Python runtime (no dev) RUN apk add --update python py-pip && rm -rf /var/cache/apk/* ENV PORT 5000 EXPOSE $PORT # Set up a working folder and install the pre-reqs WORKDIR /app ADD requirements.txt /app RUN pip install -r requirements.txt # Add the code as the last Docker layer because it changes the most ADD static /app/static ADD service.py /app # Run the service CMD [ "python", "service.py" ] More Advanced Dockerfile 22
- 26. Containers Should Be… •Stateless • All state should be maintained in a DB or Object Store • Light Weight • Only one process per container • Immutable • Do not install an ssh daemon or any other means of entering the container! • Run from Docker Registry Images or Built from Dockerfiles • Treated like code, versioned, and reconstituted when needed… not built by hand! 23
- 27. Docker Adoption HasIncreased 75% in OneYear 24https://www.datadoghq.com/docker-adoption/
- 28. What does allof this have to do with building loosely coupled microservices? 25
- 29. Managing Containers inthe Cloud 26 a.k.a. ContainerVulnerability Remediation Services …the solution that I built using loosely coupled microservices
- 30.
- 31. …and they happenevery day • A container orVM that wasn’t vulnerable yesterday is suddenly vulnerable today • This requires a management practice of continuous patching of vulnerabilities • Unfortunately, patching practices forVM’s don’t work for Containers 28
- 32. Think Cloud Native •TheTwelve-Factor App describes patterns for cloud- native architectures which leverage microservices • Applications are design as a collection of stateless microservices • State is maintained in separate databases and persistent object stores • Resilience and horizontal scaling is achieved through deploying multiple instances • Failing instances are killed and re-spawned, not debugged and patched (cattle not pets) • DevOps pipelines help manage continuous delivery of services https://www.nginx.com/blog/introduction-to-microservices/ 29
- 33.
- 34. Is My ApplicationVulnerable? •What is "the" application? • It’s a loose collection of microservices ¯_(ツ)_/¯ 30
- 35. Is My ApplicationVulnerable? •What is "the" application? • It’s a loose collection of microservices ¯_(ツ)_/¯ • Where do I install my Agents? • Nowhere! Containers are immutable and single process (via best practice) 30
- 36. Is My ApplicationVulnerable? •What is "the" application? • It’s a loose collection of microservices ¯_(ツ)_/¯ • Where do I install my Agents? • Nowhere! Containers are immutable and single process (via best practice) • How do I login to make changes? • You don’t! Did I mention that Containers are immutable? • All changes made via DevOps Pipeline • If you are not involved in the DevOps pipeline, you are not involved in Change Management 30
- 37. Is My ApplicationVulnerable? •What is "the" application? • It’s a loose collection of microservices ¯_(ツ)_/¯ • Where do I install my Agents? • Nowhere! Containers are immutable and single process (via best practice) • How do I login to make changes? • You don’t! Did I mention that Containers are immutable? • All changes made via DevOps Pipeline • If you are not involved in the DevOps pipeline, you are not involved in Change Management • When is my change window? • Never! Changes are made by application teams using blue/green deployments for continuous up-time 30
- 38. How do youmanage vulnerabilities with this Explosion of Container Growth? 31 ?
- 39. Center for InternetSecurity Docker Benchmark Recommendation • Scan and rebuild the images to include security patches • Images should be scanned "frequently" for any vulnerabilities. Rebuild the images to include patches and then instantiate new containers from it. • Rationale: • Vulnerabilities are loopholes/bugs that can be exploited and security patches are updates to resolve these vulnerabilities.We can use image vulnerability scanning tools to find any kind of vulnerabilities within the images and then check for available patches to mitigate these vulnerabilities. • Remediation: • Step 1: 'docker pull' all the base images (i.e., given your set of Dockerfiles, extract all images declared in 'FROM' instructions, and re-pull them to check for an updated/patched versions). Patch the packages within the images too. • Step 2: Force a rebuild of each image with 'docker build --no-cache'. • Step 3: Restart all containers with the updated images. 32
- 40. User Story: ContainerVulnerabilityRemediation • User Story: Container Vulnerabilities • As an Application Owner • I need an automated way to patch containers • So that they won't be vulnerable to exploits • Assumptions: • There will be long running containers ( > 24 hrs) • There will be new vulnerabilities discovered every day • Manually patching images and redeploying containers is too labor intensive • Acceptance Criteria: • Given a Docker image with deployed containers • When a vulnerability has been found in the Docker image • Then a remediation of that image will be performed • And a new image will be created and push to the registry • And any containers from the old image will be redeployed using the new image 33 User Story
- 41.
- 42. ContainerVulnerability Remediation Services ArchitecturalOverview 35 Compliance Remediation Services Redeploy Container Workload Kubernetes Notify Other Systems of Record Hybrid Cloud: Integration with Customer’s Compliance Scanning Hybrid Cloud: Deployable on Local of Public Kubernetes Hybrid Cloud: Integration with Customer’s existing systems IBM Cloud Vulnerability Advisor Cloud Native: Containers that conform to 12-factor Compliance Service Vulnerability Remediation Services USN-xxxx.1
- 43. How Does ItWork? • Compliance Service • Analyzes the input fromVulnerability Advisor and publishes alerts for other services • Vulnerability Remediation Services • Maintains a knowledge base of fix procedures and compliance remediation actions • Composes new Docker file containing the remediation actions based on the knowledge base • Forwards the new Docker file to the build service which in turn produces new version of the image • Redeploy Container Workload • Redeploys the container on Kubernetes by modifying the existing deployment parameters • Other Systems of Record • Monitor the other service messages keeping ticking system and development up to date on activities 36
- 44. What’s in aCVE? 37 { "id": "78de8449-313e-44c9-90b1-ae8277b12f95", "scan_time": 1510759001, "status": "WARN", "vulnerabilities": [ { "type": "vulnerable_package", "description": "busybox 1.24.1-r7 has vulnerabilities", "corrective_action": "Upgrade to busybox 1.24.2-r1", "fixes": [ { "cve_ids": "CVE-2016-6301", "summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.", "notice": "", "meta": { "usn_id": "" } } ], "meta": { "package_name": "busybox" "fix_version": "1.24.2-r1", "installed_version": "1.24.1-r7", } },
- 45. What’s in aCVE? 37 { "id": "78de8449-313e-44c9-90b1-ae8277b12f95", "scan_time": 1510759001, "status": "WARN", "vulnerabilities": [ { "type": "vulnerable_package", "description": "busybox 1.24.1-r7 has vulnerabilities", "corrective_action": "Upgrade to busybox 1.24.2-r1", "fixes": [ { "cve_ids": "CVE-2016-6301", "summary": "The recv_and_process_client_pkt function in networking/ntpd.c in busybox allows remote attackers to cause a denial of service (CPU and bandwidth consumption) via a forged NTP packet, which triggers a communication loop.", "notice": "", "meta": { "usn_id": "" } } ], "meta": { "package_name": "busybox" "fix_version": "1.24.2-r1", "installed_version": "1.24.1-r7", } }, This is how we know what to patch
- 46. Generated Dockerfile forAlpine 38 # # Remediation Service # FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1 # # Patching Vulnerabilities # RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi; RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi; # # Fixing Compliance Issues # RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi; # NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1.
- 47. Generated Dockerfile forAlpine 38 # # Remediation Service # FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1 # # Patching Vulnerabilities # RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi; RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi; # # Fixing Compliance Issues # RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi; # NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1. References the original image
- 48. Generated Dockerfile forAlpine 38 # # Remediation Service # FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1 # # Patching Vulnerabilities # RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi; RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi; # # Fixing Compliance Issues # RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi; # NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1. References the original image Updates the libraries that are vulnerable
- 49. Generated Dockerfile forAlpine 38 # # Remediation Service # FROM registry.ng.bluemix.net/cloud_service_mgmt/comp-srv:2.1 # # Patching Vulnerabilities # RUN apk add --no-cache busybox=1.24.2-r1;if [ $? -ne 0 ]; then apk add --no-cache busybox; fi; RUN apk add --no-cache zlib=1.2.11-r0;if [ $? -ne 0 ]; then apk add --no-cache zlib; fi; # # Fixing Compliance Issues # RUN if [ -e /etc/login.defs ]; then sed -i '/^#/! s/^.*PASS_MAX_DAYS.*$/PASS_MAX_DAYSt90/g' /etc/login.defs; fi; # NO FIX FOR: Linux.2-1-d - Minimum days that must elapse between user-initiated password changes should be 1. References the original image Updates the libraries that are vulnerable Fixes known compliance issues
- 50. Generated Dockerfile forUbuntu 39 # # Remediation Service # FROM registry.ng.bluemix.net/rofrano/counter:latest MAINTAINER Vulnerability Remediator 1.0 # # Patching Vulnerabilities # RUN apt-get install -y bash=4.3-7ubuntu1.7;if [ $? -ne 0 ]; then apt-get install -y bash; fi; RUN apt-get install -y libexpat1=2.1.0-4ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y libexpat1; fi; RUN apt-get install -y libffi6=3.1~rc1+r3.0.13-12ubuntu0.2;if [ $? -ne 0 ]; then apt-get install -y libffi6; fi; RUN apt-get install -y libgcrypt11=1.5.3-2ubuntu4.5;if [ $? -ne 0 ]; then apt-get install -y libgcrypt11; fi; RUN apt-get install -y libgnutls26=2.12.23-12ubuntu2.8;if [ $? -ne 0 ]; then apt-get install -y libgnutls26; fi; RUN apt-get install -y libssl1.0.0=1.0.1f-1ubuntu2.23;if [ $? -ne 0 ]; then apt-get install -y libssl1.0.0; fi; RUN apt-get install -y libtasn1-6=3.4-3ubuntu0.5;if [ $? -ne 0 ]; then apt-get install -y libtasn1-6; fi; RUN apt-get install -y login=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y login; fi; RUN apt-get install -y passwd=1:4.1.5.1-1ubuntu9.5;if [ $? -ne 0 ]; then apt-get install -y passwd; fi; RUN apt-get install -y sudo=1.8.9p5-1ubuntu1.4;if [ $? -ne 0 ]; then apt-get install -y sudo; fi;
- 51. 40 Generate Dockerfile to build new image Vulnerability AdvisorService vulnerabilities and compliance violations subscription to advisories Flag non- compliant images Detect non-compliant running containers and initiate remediation of relevant images Policies Container Image repository Container Instance data USN-xxxx.1 Creates new Containers existing deployment specs Based on policy, we may redeploy existing containers from remediated image DevOps Pipeline Feedback ServiceNow Integration Container Image Build Service Compliance Service Image Remediation Service Remediation Knowledge Base Possible Architecture Using Fixed Workflow Container ReDeploy Service open update close
- 52. 40 Generate Dockerfile to build new image Vulnerability AdvisorService vulnerabilities and compliance violations subscription to advisories Flag non- compliant images Detect non-compliant running containers and initiate remediation of relevant images Policies Container Image repository Container Instance data USN-xxxx.1 Creates new Containers existing deployment specs Based on policy, we may redeploy existing containers from remediated image DevOps Pipeline Feedback ServiceNow Integration Container Image Build Service Compliance Service Image Remediation Service Remediation Knowledge Base Possible Architecture Using Fixed Workflow Container ReDeploy Service open update close DON’T DO THIS
- 53. Problems with FixedWorkflow • Microservices are tightly coupled • Each service knows who to send the next request to • Ticketing service must be known by all services • No way to integrate future services without modifying several services to give them knowledge • Remediating images could take 30 - 40 minutes and polling for a response is not desirable 41
- 54. A Microservice ShouldHave • High Cohesion (Bounded Context around a Business Domain) • Does stuff that needs to change together occur together? • Low Coupling (Shared Nothing withTechnology Agnostic API) • Do you avoid making otherwise independent concerns dependent? • Low Time to Comprehension (Small and Single Responsibility) • Small enough for one person to understand quickly 42
- 55. Container ReDeploy 43 Generate Dockerfile to build new image Vulnerability AdvisorService vulnerabilities and compliance violations subscription to advisories Flag non- compliant images Detect non-compliant running containers and initiate remediation of relevant images Policies Container Image repository Container Instance data USN-xxxx.1 Creates new Containers from existing deployment specs Based on policy, we may redeploy existing containers from remediated image Pub/Sub DevOps Pipeline Feedback ServiceNow Integration Container Image Build Service Compliance Service Image Remediation Service Remediation Knowledge Base Architecture Using Messaging
- 56. Compliance Remediator Compliance Scanner RunlistBuilder Compliance Service VA Policy Manager Remediation Service Container Redeploy Service ServiceNow Callout* DevOps Issue Service* Vulnerability Advisor User Experience Docker Build Service New Vulnerability Cloud Function Periodic Trigger Pub/Sub !44 Part of IBM Cloud New Vulnerability Remediator New Compliance Remediator Checklist Builder * Future Services ContainerVulnerability Remediation Services Technology Remediation Knowledge Base
- 57. Cloud Functions withOpenWhisk • OpenWhisk is a cloud-first distributed event-based programming service • It represents an event-action platform that allows you to execute code in response to an event • Provides a serverless deployment and operations model hiding infrastructural complexity • Simply provide the code you want to execute 45
- 58. Cloud Functions HighLevel Architecture 46
- 59. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message}
- 60. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message} Get the endpoint to be called
- 61. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message} Get the endpoint to be called Get parameters passed in
- 62. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message} Get the endpoint to be called Get parameters passed in Construct body of request for microservice to call
- 63. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message} Get the endpoint to be called Get parameters passed in Construct body of request for microservice to call Make the call to fire off the microservice
- 64. Example Python CloudFunction 47 def main(params): """ Container Redeploy Function """ # Get the API key and use it in the headers headers = get_headers(params) api_endpoint = params.get('API_ENDPOINT') value = params['messages'][0]['value'] # process messages namespace = value.get('namespace', 'default') # check parameters # create the message body body = { 'namespace': namespace, 'old_image': value['old_image'], 'new_image': value['new_image'] } logging.info('Redeploying: %s', value['old_image']) result = requests.post(api_endpoint, data=json.dumps(body), headers=headers) message = result.json() return {'result': message} Get the endpoint to be called Get parameters passed in Construct body of request for microservice to call Make the call to fire off the microservice Return the microservice output
- 65. Pub/Sub Message Pub/Sub Interactions 48 ComplianceService Remediation Service Image Build Service Container Redeploy Service Each microservice is independent being called asynchronously as events are published
- 66. Pub/Sub Message Pub/Sub Interactions 48 ComplianceService Remediation Service Image Build Service Container Redeploy Service Vulnerable Image Vulnerable Image Each microservice is independent being called asynchronously as events are published
- 67. Pub/Sub Message Pub/Sub Interactions 48 ComplianceService Remediation Service Image Build Service Container Redeploy Service Vulnerable Image Vulnerable Image Image Remediated Image Remediated Each microservice is independent being called asynchronously as events are published
- 68. Pub/Sub Message Pub/Sub Interactions 48 ComplianceService Remediation Service Image Build Service Container Redeploy Service Vulnerable Image Vulnerable Image Image Remediated Image Remediated Image Build Complete Image Build Complete Each microservice is independent being called asynchronously as events are published
- 69. Pub/Sub Message Pub/Sub Interactions 48 ComplianceService Remediation Service Image Build Service Container Redeploy Service Vulnerable Image Vulnerable Image Image Remediated Image Remediated Image Build Complete Image Build Complete Containers Redeployed Each microservice is independent being called asynchronously as events are published
- 70. Pub/Sub How to Adda New Service 49 Compliance Service Ticket Notification Service New services can be added by subscribing to events on the message bus
- 71. Pub/Sub How to Adda New Service 49 Compliance Service Ticket Notification Service Vulnerable Image Vulnerable Image New services can be added by subscribing to events on the message bus Create Ticket
- 72. Pub/Sub How to Adda New Service 49 Compliance Service Ticket Notification Service Vulnerable Image Vulnerable Image Image Remediated New services can be added by subscribing to events on the message bus Create Ticket Update Ticket
- 73. Pub/Sub How to Adda New Service 49 Compliance Service Ticket Notification Service Vulnerable Image Vulnerable Image Image Remediated New services can be added by subscribing to events on the message bus Create Ticket Update Ticket Close Ticket Image Redeploy Complete
- 74. Messaging Allows For: •Services that are agnostic to other downstream services (a.k.a loosely coupled) • Long running services can simply publish when they are done instead of polling for completion • New services can be created to participate in the workflow without any change to existing services 50
- 75. Summary • Using amessaging pub/sub design keeps microservices loosely coupled • This also allows other services to participate in the workflow without changing or knowledge from existing services • Docker containers make encapsulation and deployment of services easy • Using Cloud Functions (a.k.a. Serverless) complements the event driven workflow and simplifies message flows 51