Open Source Investments in Mainframe
Through the Next Generation - Showcasing
the Work of the Open Mainframe Project 2018
Summer Interns
John Mertic and the 2018 Open
Mainframe Project Interns
@OpenMFProject
Not this mainframe…
The modern mainframe ( IBM z14 )
• Up to 170, 10 core 5.2ghz processors (
read more at
https://en.wikipedia.org/wiki/IBM_z14_(
microprocessor) )
• Up to 32TB of RAIM (random array of
independent memory)
• Heavy optimization for i/o bound
workloads
• Hardware based encryption
• Fault tolerant, native hot swapping of
hardware components ( z stands for
zero downtime )
• And….runs Linux and Open Source!
Open Source on the Mainframe - A Journey of Innovation
and Technology
• Spectrum Scale™
• Oracle 12c
1999:
Linux on
S/390®
2009:
▪ z/VM v6
▪ Enterprise Linux Server
(ELS)
2000:
Integrated Facility for
Linux (IFL)
2007:
IBM Big Green Consolidation 3900
to 30 servers
2001: HiperSockets™
▪ RHEL 7.3
▪ SLES12 SP2
▪ Ubuntu 16.04
2006:
1000 Appl., 300 ISVs
2000:
Db2®, WebSphere®
1999: IBM Linux
Tech Center
2001:
Red Hat Linux
First release
2000:
SUSE Linux
2002:
major ISVs:
SAP, Oracle 9i
2014:
▪ IBM Wave
▪ OpenStack
2015:
▪ KVM for IBM z
▪ IBM Wave upd.
▪ IBM zAware for Linux
▪ IBM LinuxONE™
▪ RHEL 7.2
▪ SLES 12 SP1
• Blockchain
• Open source
ecosystem
2017:
▪ IBM z14
▪ z/VM Subcapacity
▪ IBM Wave 1.2 SP6
• IBM Cloud Private
• Db2 Warehouse
• DBaaS ref.archit.
• Spectrum Scale 5.0
• Docker Ent.Edition
▪ KVM support distros
▪ RHEL 7.4
▪ SLES 12 SP3
▪ Ubuntu 17.04
• Db2 BLU
• GDPS® Virtual Appliance
• Financial Transaction Mgr
• Open source ecosystem
2016:
▪ KVM 1.1.2
▪ z/VM 6.4
2013:
>3000 Applications 2015:
Open Mainframe
Project
Putting mainframe in the hands of the next generation
Summer Internship Program
VCU Capstone Partnership
Blockchain Hackathon for
EPSI University in France
27
Interns
sponsored
13
Supporting
academic
institutions
100+
Students
impacted
Let’s look at this year’s projects
• Areas of focus include..
– Kubernetes
– Packaging
– Kata Containers
– Cloud Foundry
– VMLINK
– Alpine Linux
Kubernetes &
LinuxONE
Asish Varanasi
Rajula Vineet Reddy
@asish_0512
@rajula96reddy
@asish_0512
@rajula96reddy
About us
• Mentor - Mike Friesenegger, Technology Strategist SUSE
• Asish Varanasi - Recently graduated from IIIT Hyderabad
• Rajula Vineet Reddy - Senior year CS grad, IIIT Bangalore
@asish_0512
@rajula96reddy
Why Kubernetes?
@asish_0512
@rajula96reddy
What have we achieved?
● Setup based on
○ 3-node cluster
○ Official binaries
○ OpenSSL
○ Systemd services
○ Docker
○ Etcd* - NOT HA
○ Flannel
● Documentation & Video
Demos for
○ RHEL 7.5
○ SLES12SP3
○ Ubuntu 18.04
@asish_0512
@rajula96reddy
Future work
• Creating a highly available cluster
• AddOns
• E2E tests
@asish_0512
@rajula96reddy
Documentation Links
1. SLES
https://openmainframeproject.github.io/LinuxOne_Kubernetes_SL
ES_Deployment_Documentation/
1. Ubuntu
https://openmainframeproject.github.io/LinuxOne_Kubernetes_C
anonical_Deployment_Documentation/
1. RHEL
https://github.com/openmainframeproject/RHEL-documentation
@asish_0512
@rajula96reddy
Video Links
1. Ubuntu - https://youtu.be/Ae8ptEXaM-8
2. SLES - https://youtu.be/qk2oMgmpsKw
3. RHEL - https://youtu.be/WN6NyJ_ZUcY
Shikhar Jaiswal
Package Hub for s390x
Jayaditya Gupta Mohit Bansal Usman Haider Wolfgang
(Mentor)
Why Packaging and Package Hub?
• Why Packaging?
– To give soul to any bare Operating System.
– To ease the process of installing software for users.
– To maintain and update the installed package easily.
• Package Hub
– FOSS packages, images and more for SLES
– OBS provides Ubuntu, Redhat, Debian support
– Arch: s390x, x86-64, ppc64le & more
– Free: Community maintained & community supported
– Does not break SLES support from SUSE
• Packaging for s390x
– Users want more FOSS packages
– Ease of use to users/customers
– zypper install vs make, cmake, build etc.
Key Achievements
• Monitoring Packages
– Zabbix, Icinga2, Icinga-web, Cacti, Pagemon, grafana
• Security Packages
– Kibana, rclone, watchdog, Agda, Amanda
• Miscellaneous Packages
– nextcloud, seafile, arp-scan, htmldoc, weechat, Haskell stack (under progress)
• Total Packages
– 20+ packages build and tested for s390x
• If interested in a particular package, contact us and we can help.
Impact on mainframe
• More Packages
– 20+ packages build and tested for s390x
– Packages from different domains
• More Users
– Availability of FOSS packages brings in more users
• User Retention
– FOSS provides latest technologies to consumers
• More Contributions
– Use of a larger Linux ecosystem
Kata Containers
Yash Jain
@Yash_D_Jain
Elton de Souza
@elton_de_souza
Secure Container
Runtime
Traditional Container
Kata Container
Kata On Z
The Background Story...
• IBM needed a runtime for its Secure
Service Container (SSC) offering.
• At first, started porting gVisor.
• Dropped later
– Not mature enough.
– Performance was poor.
– Uses ptrace.
Why port Kata?
• Uses hardware based virtualisation.
• Already supports multiple architectures.
• Has a stable v1.0 release.
• Stable
Kata on Mainframes
• Stonewall against attackers
• More efficient resource utilisation
• Better performance
• Negligible network costs for most use
cases.
• Not yet complete.
• IBM taking the work forward.
• Will not support vhost-user devices.
Port Status
Suse Cloud Foundry on Z
Sakala Venkata Krishna Rohit / Vlad Iovanov
@rohitsakala
Why SCF on Z ?
• SCF - PaaS
– Cloud Foundry -
• Focus on workloads that will be built and run in containers.
• Templated workflow which developers leverage.
• Kubernetes -
– Kubernetes -
• ubiquitous answer for operational side.
• Not primarily focused on what you are running in the container.
– Advantage of Cloud Foundry workflow by installing it into your kubernetes clusters.
Ref - https://www.suse.com/c/applying-cloud-foundry-workflow-kubernetes/
Why SCF on Z ?
● Linux On Z Benefits
○ I/O and Cache Intensive
○ Banking sector - Security/Encryption.
○ Fault tolerant mainframe hardware capable of over 90,000 I/O operations per
second.
○ Linux On Z was built for applications that need Reliability, Accessibility,
Security, Stability and Scalability (RASSS).
○ IBM z can support more virtual servers than any other in a single footprint.,
○ Few examples
■ mongodb - 17tb. spare cores on failover for scalability - not available on
x86 servers.
■ 2000 + docker spawn, better than x86.
■ 50 percent higher performance on apache spark than x86.
● Cloud is an open source project. Enabling this will have a lot of impact on
mainframes.
Ref - https://mediacenter.ibm.com/media/1_je27z8u8
Key Achievements
• Building openSuse bosh linux stemcell.
• Building openSuse fissile stemcell.
• SCF on Z
– All releases of SCF were compiled
successfully except for one.
Problems Faced
• openSUSE image for Z needs to be maintained.
• Boring ssl and google protobuf support for Z.
VMLINK
Rushal Verma
@rusrushal13
Rick Troth(Mentor)
@vmcowboy
WHY VMLINK
• extend the namespace of virtual machine
to include devices by other virtual
machines.
• automounter point: /vmlink
• just do cd /vmlink/vmid.addr
Key Achievements
• able to write the automounter script
• tested on various other disks owned by
different hosts and handled partitioning
too.
Impact on Mainframe
• VMLINK - by developers for developers
• saves time who doesn’t speak(unix and
z/vm) both languages
Open Source Investments in Mainframe Through the Next Generation - Showcasing the Work of the Open Mainframe Project 2018 Summer Interns

Open Source Investments in Mainframe Through the Next Generation - Showcasing the Work of the Open Mainframe Project 2018 Summer Interns

  • 1.
    Open Source Investmentsin Mainframe Through the Next Generation - Showcasing the Work of the Open Mainframe Project 2018 Summer Interns John Mertic and the 2018 Open Mainframe Project Interns @OpenMFProject
  • 2.
  • 3.
    The modern mainframe( IBM z14 ) • Up to 170, 10 core 5.2ghz processors ( read more at https://en.wikipedia.org/wiki/IBM_z14_( microprocessor) ) • Up to 32TB of RAIM (random array of independent memory) • Heavy optimization for i/o bound workloads • Hardware based encryption • Fault tolerant, native hot swapping of hardware components ( z stands for zero downtime ) • And….runs Linux and Open Source!
  • 4.
    Open Source onthe Mainframe - A Journey of Innovation and Technology • Spectrum Scale™ • Oracle 12c 1999: Linux on S/390® 2009: ▪ z/VM v6 ▪ Enterprise Linux Server (ELS) 2000: Integrated Facility for Linux (IFL) 2007: IBM Big Green Consolidation 3900 to 30 servers 2001: HiperSockets™ ▪ RHEL 7.3 ▪ SLES12 SP2 ▪ Ubuntu 16.04 2006: 1000 Appl., 300 ISVs 2000: Db2®, WebSphere® 1999: IBM Linux Tech Center 2001: Red Hat Linux First release 2000: SUSE Linux 2002: major ISVs: SAP, Oracle 9i 2014: ▪ IBM Wave ▪ OpenStack 2015: ▪ KVM for IBM z ▪ IBM Wave upd. ▪ IBM zAware for Linux ▪ IBM LinuxONE™ ▪ RHEL 7.2 ▪ SLES 12 SP1 • Blockchain • Open source ecosystem 2017: ▪ IBM z14 ▪ z/VM Subcapacity ▪ IBM Wave 1.2 SP6 • IBM Cloud Private • Db2 Warehouse • DBaaS ref.archit. • Spectrum Scale 5.0 • Docker Ent.Edition ▪ KVM support distros ▪ RHEL 7.4 ▪ SLES 12 SP3 ▪ Ubuntu 17.04 • Db2 BLU • GDPS® Virtual Appliance • Financial Transaction Mgr • Open source ecosystem 2016: ▪ KVM 1.1.2 ▪ z/VM 6.4 2013: >3000 Applications 2015: Open Mainframe Project
  • 5.
    Putting mainframe inthe hands of the next generation Summer Internship Program VCU Capstone Partnership Blockchain Hackathon for EPSI University in France 27 Interns sponsored 13 Supporting academic institutions 100+ Students impacted
  • 6.
    Let’s look atthis year’s projects • Areas of focus include.. – Kubernetes – Packaging – Kata Containers – Cloud Foundry – VMLINK – Alpine Linux
  • 7.
    Kubernetes & LinuxONE Asish Varanasi RajulaVineet Reddy @asish_0512 @rajula96reddy
  • 8.
    @asish_0512 @rajula96reddy About us • Mentor- Mike Friesenegger, Technology Strategist SUSE • Asish Varanasi - Recently graduated from IIIT Hyderabad • Rajula Vineet Reddy - Senior year CS grad, IIIT Bangalore
  • 9.
  • 10.
    @asish_0512 @rajula96reddy What have weachieved? ● Setup based on ○ 3-node cluster ○ Official binaries ○ OpenSSL ○ Systemd services ○ Docker ○ Etcd* - NOT HA ○ Flannel ● Documentation & Video Demos for ○ RHEL 7.5 ○ SLES12SP3 ○ Ubuntu 18.04
  • 11.
    @asish_0512 @rajula96reddy Future work • Creatinga highly available cluster • AddOns • E2E tests
  • 12.
    @asish_0512 @rajula96reddy Documentation Links 1. SLES https://openmainframeproject.github.io/LinuxOne_Kubernetes_SL ES_Deployment_Documentation/ 1.Ubuntu https://openmainframeproject.github.io/LinuxOne_Kubernetes_C anonical_Deployment_Documentation/ 1. RHEL https://github.com/openmainframeproject/RHEL-documentation
  • 13.
    @asish_0512 @rajula96reddy Video Links 1. Ubuntu- https://youtu.be/Ae8ptEXaM-8 2. SLES - https://youtu.be/qk2oMgmpsKw 3. RHEL - https://youtu.be/WN6NyJ_ZUcY
  • 14.
    Shikhar Jaiswal Package Hubfor s390x Jayaditya Gupta Mohit Bansal Usman Haider Wolfgang (Mentor)
  • 15.
    Why Packaging andPackage Hub? • Why Packaging? – To give soul to any bare Operating System. – To ease the process of installing software for users. – To maintain and update the installed package easily. • Package Hub – FOSS packages, images and more for SLES – OBS provides Ubuntu, Redhat, Debian support – Arch: s390x, x86-64, ppc64le & more – Free: Community maintained & community supported – Does not break SLES support from SUSE • Packaging for s390x – Users want more FOSS packages – Ease of use to users/customers – zypper install vs make, cmake, build etc.
  • 16.
    Key Achievements • MonitoringPackages – Zabbix, Icinga2, Icinga-web, Cacti, Pagemon, grafana • Security Packages – Kibana, rclone, watchdog, Agda, Amanda • Miscellaneous Packages – nextcloud, seafile, arp-scan, htmldoc, weechat, Haskell stack (under progress) • Total Packages – 20+ packages build and tested for s390x • If interested in a particular package, contact us and we can help.
  • 17.
    Impact on mainframe •More Packages – 20+ packages build and tested for s390x – Packages from different domains • More Users – Availability of FOSS packages brings in more users • User Retention – FOSS provides latest technologies to consumers • More Contributions – Use of a larger Linux ecosystem
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
    The Background Story... •IBM needed a runtime for its Secure Service Container (SSC) offering. • At first, started porting gVisor. • Dropped later – Not mature enough. – Performance was poor. – Uses ptrace.
  • 24.
    Why port Kata? •Uses hardware based virtualisation. • Already supports multiple architectures. • Has a stable v1.0 release. • Stable
  • 25.
    Kata on Mainframes •Stonewall against attackers • More efficient resource utilisation • Better performance • Negligible network costs for most use cases.
  • 26.
    • Not yetcomplete. • IBM taking the work forward. • Will not support vhost-user devices. Port Status
  • 27.
    Suse Cloud Foundryon Z Sakala Venkata Krishna Rohit / Vlad Iovanov @rohitsakala
  • 28.
    Why SCF onZ ? • SCF - PaaS – Cloud Foundry - • Focus on workloads that will be built and run in containers. • Templated workflow which developers leverage. • Kubernetes - – Kubernetes - • ubiquitous answer for operational side. • Not primarily focused on what you are running in the container. – Advantage of Cloud Foundry workflow by installing it into your kubernetes clusters. Ref - https://www.suse.com/c/applying-cloud-foundry-workflow-kubernetes/
  • 29.
    Why SCF onZ ? ● Linux On Z Benefits ○ I/O and Cache Intensive ○ Banking sector - Security/Encryption. ○ Fault tolerant mainframe hardware capable of over 90,000 I/O operations per second. ○ Linux On Z was built for applications that need Reliability, Accessibility, Security, Stability and Scalability (RASSS). ○ IBM z can support more virtual servers than any other in a single footprint., ○ Few examples ■ mongodb - 17tb. spare cores on failover for scalability - not available on x86 servers. ■ 2000 + docker spawn, better than x86. ■ 50 percent higher performance on apache spark than x86. ● Cloud is an open source project. Enabling this will have a lot of impact on mainframes. Ref - https://mediacenter.ibm.com/media/1_je27z8u8
  • 30.
    Key Achievements • BuildingopenSuse bosh linux stemcell. • Building openSuse fissile stemcell. • SCF on Z – All releases of SCF were compiled successfully except for one.
  • 31.
    Problems Faced • openSUSEimage for Z needs to be maintained. • Boring ssl and google protobuf support for Z.
  • 33.
  • 34.
    WHY VMLINK • extendthe namespace of virtual machine to include devices by other virtual machines. • automounter point: /vmlink • just do cd /vmlink/vmid.addr
  • 35.
    Key Achievements • ableto write the automounter script • tested on various other disks owned by different hosts and handled partitioning too.
  • 36.
    Impact on Mainframe •VMLINK - by developers for developers • saves time who doesn’t speak(unix and z/vm) both languages