Libra: a Library OS for a JVM

Libra: A Library OS for a JVM Glenn Ammons, Jonathan Appavoo, Maria Butrico, Dilma Da Silva, David Grove, Kiyokuni Kawachiya, Orran Krieger, Bryan Rosenburg, Eric Van Hensbergen, Robert Wisniewski T.J. Watson Research Center Tokyo Research Lab 13 - June - 2007 Monday, August 10, 2009

IBM Research Motivation  Customized operating system support for applications  Previous approaches – SPIN, Vino, Scout, K42 – Exokernel  Virtualization – new opportunity General- Purpose Application Application OS Library OS Abstractions Library OS Control User domain domain Multiplex Exokernel hardware Hypervisor Hardware Hardware 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Outline  J9/Libra architecture  Nutch/Lucene Query application  Initial optimizations  Experimental results  Next steps 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Proxy Process Application User Console Environment Storage J9 (JVM) J9 port layer Authority Libra Libra API File Ops Sockets Sys Svc Inferno (9p Server) Threads Memory mgmt. 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage Linux Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage Libra Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage Libra Inferno (9p Server) 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage Libra Inferno (9p Server) Threads Memory mgmt. 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage J9 (JVM) J9 port layer Libra Libra API Inferno (9p Server) Threads Memory mgmt. 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Storage J9 (JVM) J9 port layer Libra Libra API File Ops Sockets Sys Svc Inferno (9p Server) Threads Memory mgmt. 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra Architecture Control Partition (Dom0) File System User Partition (DomU) Network Proxy Process Application User Console Environment Storage J9 (JVM) J9 port layer Authority Libra Libra API File Ops Sockets Sys Svc Inferno (9p Server) Threads Memory mgmt. 9p Client Linux in channel out channel Xen 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Target workload: Nutch/Lucene Query documents query distribute docs docs docs docs query query query query 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch/Lucene Query application structure Back-end data Driver Front-end Back-end data throughput measurement Back-end data 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Libra optimizations: file caching  Index and some raw data must be in memory  Nutch query back-end relies on OS buffer cache  Going to control partition is expensive  Solution: cache files locally in Libra  Average lseek() & read() cost for back-end: – J9/Linux: 2.25 usec – J9/Libra: 0.9 usec 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Libra optimizations: socket streaming  Nutch query back-end is a streaming application  Requests buffered in control partition  Fetching them on-demand adds latency  Sending results ties up worker threads  Solution: stage socket data into/out of Libra partition – New requests are always available locally – Results are sent asynchronously in batches 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch/Lucene Query performance  Single back-end server  10 GB document set Configuration Queries / second Default 5.9 File caching 12.8 File caching & Socket 16.0 streaming 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Performance evaluation  Platform – JS21 (PowerPC) blade – IBM BladeCenter – XenPPC – Partitions with 1 core, 1920 MB of memory 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research SPECjvm98 J9/Libra, no cache  jess & jack: file cache 1.4 J9/Libra, with cache 1.2  javac: open/stat Speedup over J9/Linux 1.0  db: large pages 0.8 0.6 0.4 0.2 0 o c s trt k b s di a es ac es _d _m av u pr _j _j ga 9 _j 7 20 m 8 2 3 pe 22 22 20 o 21 _c _m 1 2 20 22 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research SPECjbb2000 1.4 Speedup over J9/Linux, 1 Warehouse 1.2 1.0 0.8 0.6 0.4 J9/Linux J9/Libra 0.2 0.0 1 2 3 4 5 6 7 8 Number of Warehouses 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research SPECjbb2000 1.4 Speedup over J9/Linux, 1 Warehouse 1.2 1.0 0.8 0.6 0.4 J9/Linux J9/Libra 0.2 J9/Libra, small pages 0.0 1 2 3 4 5 6 7 8 Number of Warehouses 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Java Grande Forum Multi-threaded Benchmarks J9/Libra vs J9/Linux 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch/Lucene Query scaling  Query cluster – 1 front-end – 15 back-end servers – 5 blades – 150 GB document set  All clusters serve the same (replicated) data 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Related Work  BEA LiquidVM  Hardware – Azul – IBM zAAP 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Next Steps  Supporting POSIX interfaces – C/C++ library code – Java-Native-Interface support – C/C++ applications  Supporting network directly on Libra  Pursuing JVM optimizations – “safe-points” to support type-accurate garbage collection – Real-Time Java support 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Why is it named Libra? Only the necessary  We chose the name “Libra” components can because our goal – to provide be chosen. well-balanced services -- aligns with the imagery associated with the constellation of Nutch the same name. stream I/O cache JVM stat Constellation Libra Libra 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Stop!  The talk ended with the previous slide  Everything else is backup stuff 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research One slide summary  Specializing OS for application is attractive  Prior attempts had trouble supporting legacy code  Our current approach leverages – Hypervisor – Library OS handles target optimizations; side car handles the rest – 9P protocol for distributed resources  We did it: we run JVMs on a user domain partition without an OS  We explore performance optimization opportunities  We profit from management simplification 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research System architecture (cont)  Using hypervisor instead of exokernel – App can use privileged execution mode and instructions – A pared-down linux can run as libOS  Apps can be like OSes or like micro-kernels  Criticism for virtualization approach – Focus on parts; no general view • In Libra control partition provides general view 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research J9/Libra implementation  J9’s portability layer  Threading and synchronization  Incremental – Started with smallest J9 • no JIT, no optional VM features, CLDC J2ME class libraries – Started with dummy implementations on port and thread libraries – Added features as required by workload – Added features back into J9  Debugging  Currently • Running full JIT and largest set of IBM class libraries for J9 version • 50 % of port “dummies” refer to signal handling, shared memory regions, socket/network 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Libra Internals  Memory management – Simple two-level management – Heap allocated when requested  File system services – Needed for loading Java class files and providing I/O – Mapped to 9P protocol  Subset of pthreads libraries – Designed for scalability but currently only supports 1 processor per partition – No preemptive time-slicing  Socket interface – Operations forwarded to gateway server – From outside, app seems to be running on the system network server 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Driver Front-end Back-end data throughput measurement Back-end data 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Q Driver Front-end Back-end data throughput measurement Back-end data 1. Front-end receives query (“foo AND bar”) 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Driver Front-end Back-end data throughput measurement Back-end data 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Driver Front-end Back-end data throughput measurement Back-end data 1. Front-end receives query (“foo AND bar”) 2. Front-end sends query to each back-end 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Driver Front-end Back-end data throughput measurement Back-end data 1. Front-end receives query (“foo AND bar”) 2. Front-end sends query to each back-end 3. Each back-end searches its data segment, ranks results 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Q Back-end data Driver Front-end Q Back-end data throughput measurement Q Back-end data 1. Front-end receives query (“foo AND bar”) 2. Front-end sends query to each back-end 3. Each back-end searches its data segment, ranks results 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data Driver Front-end Back-end data throughput measurement Back-end data 1. Front-end receives query (“foo AND bar”) 2. Front-end sends query to each back-end 3. Each back-end searches its data segment, ranks results 4. Back-end send their partial results to front-end 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Nutch query Back-end data R Driver Front-end Back-end data throughput measurement Back-end data 1. Front-end receives query (“foo AND bar”) 2. Front-end sends query to each back-end 3. Each back-end searches its data segment, ranks results 4. Back-end send their partial results to front-end 5. Front-end selects top results overall 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research File Block I/O Performance  Performance of forwarding standard read/write  128 MB file with varying buffer size  Data present on Linux cache 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Management simplification: Many Become One 9.2.22.200 9.2.22.100 9.2.22.150 xio cpus OS disks OS OS 9.2.22.40 9.2.22.36 9.2.22.125 OS OS OS 9.2.22.63 9.2.22.121 OS OS 9.2.22.140 9.2.22.134 9.2.22.160 OS OS OS 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Management simplification: Many Become One 9.2.22.200 9.2.22.100 9.2.22.150 OS OS OS 9.2.22.40 9.2.22.36 9.2.22.125 OS OS 11 1 OS 9.2.22.63 9.2.22.121 OS OS 9.2.22.140 9.2.22.134 Accelerators 9.2.22.160 OS OS OS 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research $ ssh chassis0 $ ssh chassis1 $ ssh chassis2 chassis0 > java HelloWorld chassis1 > java HelloWorld chassis2 > java HelloWorld Virtual Chassis 0 Virtual Chassis 1 Virtual Chassis 2 Linux Linux Linux Pool of Domains Pool of Domains Pool of Domains Blades Blades Blades 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

IBM Research Java Program Legend 9.2.70.236 J9LibOS and Support DomU DomU DomU Dom0 9P Path BE BE mgt gpfs mgt gpfs 9P Component J9LibOS J9LibOS Linux 9p Linux Xen Dom0 Linux Linux SM GPFS Component gpfs JS21 XenPPC Common Linux Linux Network Interfaces 192.168.X.Y DomU DomU DomU Dom0 DomU DomU DomU Dom0 BE BE BE mgt gpfs BE BE BE mgt gpfs J9LibOS J9LibOS J9LibOS 9p Linux J9LibOS J9LibOS J9LibOS 9p Linux SM SM JS21 XenPPC JS21 XenPPC 13 June 2007 - VEE’07 Libra Monday, August 10, 2009

http://soa-biz.blogspot.com	195
http://researcher.watson.ibm.com	33
https://researcher.ibm.com	16
http://researcher.ibm.com	9
http://soa-biz.blogspot.co.uk	9
http://soa-biz.blogspot.fr	7
http://soa-biz.blogspot.ca	5
http://soa-biz.blogspot.com.au	4
http://soa-biz.blogspot.nl	3
http://soa-biz.blogspot.ch	3
http://soa-biz.blogspot.de	2
http://soa-biz.blogspot.jp	2
http://soa-biz.blogspot.ru	2
http://soa-biz.blogspot.co.nz	2
http://soa-biz.blogspot.in	2
http://soa-biz.blogspot.hk	1
http://soa-biz.blogspot.se	1
http://soa-biz.blogspot.it	1
http://soa-biz.blogspot.dk	1
http://soa-biz.blogspot.mx	1

Libra: a Library OS for a JVM

by Eric Van Hensbergen

Accessibility

Categories

Upload Details

Usage Rights

20 Embeds 299

Statistics