SlideShare a Scribd company logo
JUG Lausanne
                                  8. March 2012




 Apache Direct Memory

Reducing Heap Memory Stress
 The next battle horse for JVM
     performance tuning
JUG Lausanne
                                     8. March 2012




              About me
•   Benoit Perroud
•   Apache Direct Memory Commiter
•   bperroud@apache.org
•   @killerwhile
•   Software craftsman
•   BigData Engineer @
JUG Lausanne
                                              8. March 2012




                Today's Agenda
•   Off Heap Caching
     – Java Memory
    – Garbage Collector (GC)
    – Cache On-heap vs. Off-heap Caching
•   Apache Direct Memory
     – Design and principles
    – Uses cases
       • Multi layered cache
       • Standalone server “à la memcache”
    – Next steps
•   Questions
JUG Lausanne
                                                  8. March 2012




           Before starting
• Sorry for my bad English and my poor French
• Interrupt me anytime
• I have nothing to sell. It's just worth while sharing
• Please do ask questions
JUG Lausanne
                                 8. March 2012




           Java Memory
• Automatic memory allocation
• Garbage collector (GC)
JUG Lausanne
                                       8. March 2012




      Garbage Collector
• Several types of GC
  – Serial GC
  – Parallel GC (throughput collector)
  – Concurrent Mark & Sweep GC (concurrent
    low pause collector)
  – G1 GC (low latency concurrent M&S)
JUG Lausanne
                                           8. March 2012




       Garbage Collector
• But all GC have a stop-the-world
  behavior
• Proportional to the memory's size
• Resulting in application
  unresponsiveness
  – A pain when dealing with tight SLAs
JUG Lausanne
                                          8. March 2012




Cache On-Heap vs. Off-Heap
• On-heap
  – Objects tends to be promoted into tenured
    memory
  – GC storm effect when using refreshing
    cache
  – No overhead (for caching by reference)
JUG Lausanne
                                                 8. March 2012




    On-Heap vs. Off-Heap
• Off-heap
  – Object payload is no more affecting GC
  – Serialization/Deserialization overhead
    • Hopefully lots of work on serialization has been
      done (Protobuf, Avro, Thrift, msgpack,
      BSON, ...)
JUG Lausanne
                                   8. March 2012




   Apache Direct Memory
Apache Direct Memory is a multi
layered cache implementation featuring
off-heap memory storage to enable
caching of java objects without
degrading jvm performance.
→ Opensource implementation of
Terracotta BigMemory.
JUG Lausanne
                                                                     8. March 2012




           Apache Direct Memory
•   Apache Software Foundation Incubator project
•   Met the Incubator falls 2011
•   12 developers ATM, 10+ contributors
•   I joined 1st January 2012
    –   was the good achievement of my Hacky Christmas Holiday :)


• Disclaimer : Under heavy development
    – I rewrote most of the memory allocation service
    – APIs are subject to changes, and bugs to be found
JUG Lausanne
                                      8. March 2012




     Design & Principles
• ByteBuffer.allocateDirect is the
  foundation of the cache
• ByteBuffers are allocated in big chunk
  and then splitted for internal use
JUG Lausanne
                                           8. March 2012




      Design & Principles
• Build on layers :
  – CachingService
     • Serialize object (pluggable)
  – MemoryManagerService
     • Compute access statistics
  – ByteBufferAllocatorService
     • Eventually deal with ByteBuffers
JUG Lausanne
                                          8. March 2012




   ByteBuffers Allocation
  2 different allocation's strategies
• Merging ByteBuffers allocation
  – No memory wasted
  – Free at creation
  – Suffer from fragmentation
  – Need synchronization at allocation and
    deallocation
JUG Lausanne
                                                                         8. March 2012




      ByteBuffers Allocation
•   Fixed size ByteBuffers allocation
     – Linux kernel SLAB's style allocation
         • Select a set of fixed sizes
         • Split bigger buffers (1MB+) in that size
     – Allocation is really fast and good concurrency
        • All structures is pre-instanciated
     – Creation (or buffer's size increase) has a cost
         • 1GB split in 128 bytes slabs is 8M+ buffers created
     – Do not suffer from fragmentation
     – Waste memory if the selected size is not relevant
         • Work really well in HDFS where all blocks are of the same size
JUG Lausanne
                                          8. March 2012


 Use case 1 : Multi layers
           cache
• Idea : most used objects are cached on-heap,
  the rest off-heap, may overflown to disk.
• Sounds like BigMemory.
• See
  net.sf.ehcache.store.offheap.OffHeapStore
• Actually we inject DM in ehcache like do
  BigMemory. Ouch ;)
• Comparison needs to be done
JUG Lausanne
                                                             8. March 2012


Use case 2 : OffHeap Output
           Stream
• Idea : read Twitter firehose stream without
  filling the precious heap memory
  – OOM will lead to unpredictable behavior else where in the
    application
• From your socket directly write off-heap using
  OutputStream style
  – allocate a fixed size temporary buffer of your choice
• Read from this stream
  – InputAndOutputStream parent class that holds both
    OutputStream and InputStream instances
JUG Lausanne
                                           8. March 2012


   Use case 3 : Standalone
         cache server
• Idea : replace Memcached :)
   – But with native plain REST API
• DM has all the building blocks to implement
  such server, worth while trying
• See the server submodule
JUG Lausanne
                                                    8. March 2012




                  Next Steps
• JSR 107
• Real Benchmarks
• Builder patterns
• Integration with more libs (Spring, Guice, …)
• Implementations with DM lib (Cassandra (wip), Lucene,
  Tomcat, …)
• Cache Resizing
• Management and monitoring
• ...
• https://issues.apache.org/jira/browse/DIRECTMEMORY
JUG Lausanne
                              8. March 2012




           Questions ?
• Thanks for you attention

More Related Content

Similar to Direct memory jugl-2012.03.08

Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
Alluxio, Inc.
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)
emiltamas
 
Quick introduction to Java Garbage Collector (JVM GC)
Quick introduction to Java Garbage Collector (JVM GC)Quick introduction to Java Garbage Collector (JVM GC)
Quick introduction to Java Garbage Collector (JVM GC)
Marcos García
 
Modern Java Concurrency (OSCON 2012)
Modern Java Concurrency (OSCON 2012)Modern Java Concurrency (OSCON 2012)
Modern Java Concurrency (OSCON 2012)
Martijn Verburg
 
Container Attached Storage with OpenEBS - CNCF Paris Meetup
Container Attached Storage with OpenEBS - CNCF Paris MeetupContainer Attached Storage with OpenEBS - CNCF Paris Meetup
Container Attached Storage with OpenEBS - CNCF Paris Meetup
MayaData Inc
 
Java in High Frequency Trading
Java in High Frequency TradingJava in High Frequency Trading
Java in High Frequency Trading
Viktor Sovietov
 
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
MayaData
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
lucenerevolution
 
Heapoff memory wtf
Heapoff memory wtfHeapoff memory wtf
Heapoff memory wtf
Olivier Lamy
 
An Active and Hybrid Storage System for Data-intensive Applications
An Active and Hybrid Storage System for Data-intensive ApplicationsAn Active and Hybrid Storage System for Data-intensive Applications
An Active and Hybrid Storage System for Data-intensive Applications
Xiao Qin
 
Modern Java Concurrency (Devoxx Nov/2011)
Modern Java Concurrency (Devoxx Nov/2011)Modern Java Concurrency (Devoxx Nov/2011)
Modern Java Concurrency (Devoxx Nov/2011)
Martijn Verburg
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
Ruben Badaró
 
Solving k8s persistent workloads using k8s DevOps style
Solving k8s persistent workloads using k8s DevOps styleSolving k8s persistent workloads using k8s DevOps style
Solving k8s persistent workloads using k8s DevOps style
MayaData
 
Sheepdog: yet another all in-one storage for openstack
Sheepdog: yet another all in-one storage for openstackSheepdog: yet another all in-one storage for openstack
Sheepdog: yet another all in-one storage for openstack
Liu Yuan
 
Java on the Mainframe
Java on the MainframeJava on the Mainframe
Java on the Mainframe
Michael Erichsen
 
Alluxio - Scalable Filesystem Metadata Services
Alluxio - Scalable Filesystem Metadata ServicesAlluxio - Scalable Filesystem Metadata Services
Alluxio - Scalable Filesystem Metadata Services
Alluxio, Inc.
 
Odnoklassniki.ru Architecture
Odnoklassniki.ru ArchitectureOdnoklassniki.ru Architecture
Odnoklassniki.ru Architecture
Dmitry Buzdin
 
August 2013 HUG: Removing the NameNode's memory limitation
August 2013 HUG: Removing the NameNode's memory limitation August 2013 HUG: Removing the NameNode's memory limitation
August 2013 HUG: Removing the NameNode's memory limitation
Yahoo Developer Network
 
Machine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkMLMachine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkML
Arnab Biswas
 
Architecting for failure
Architecting for failureArchitecting for failure
Architecting for failure
Michael Brunton-Spall
 

Similar to Direct memory jugl-2012.03.08 (20)

Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
Optimizing Latency-Sensitive Queries for Presto at Facebook: A Collaboration ...
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)
 
Quick introduction to Java Garbage Collector (JVM GC)
Quick introduction to Java Garbage Collector (JVM GC)Quick introduction to Java Garbage Collector (JVM GC)
Quick introduction to Java Garbage Collector (JVM GC)
 
Modern Java Concurrency (OSCON 2012)
Modern Java Concurrency (OSCON 2012)Modern Java Concurrency (OSCON 2012)
Modern Java Concurrency (OSCON 2012)
 
Container Attached Storage with OpenEBS - CNCF Paris Meetup
Container Attached Storage with OpenEBS - CNCF Paris MeetupContainer Attached Storage with OpenEBS - CNCF Paris Meetup
Container Attached Storage with OpenEBS - CNCF Paris Meetup
 
Java in High Frequency Trading
Java in High Frequency TradingJava in High Frequency Trading
Java in High Frequency Trading
 
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
OpenEBS; asymmetrical block layer in user-space breaking the million IOPS bar...
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
 
Heapoff memory wtf
Heapoff memory wtfHeapoff memory wtf
Heapoff memory wtf
 
An Active and Hybrid Storage System for Data-intensive Applications
An Active and Hybrid Storage System for Data-intensive ApplicationsAn Active and Hybrid Storage System for Data-intensive Applications
An Active and Hybrid Storage System for Data-intensive Applications
 
Modern Java Concurrency (Devoxx Nov/2011)
Modern Java Concurrency (Devoxx Nov/2011)Modern Java Concurrency (Devoxx Nov/2011)
Modern Java Concurrency (Devoxx Nov/2011)
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Solving k8s persistent workloads using k8s DevOps style
Solving k8s persistent workloads using k8s DevOps styleSolving k8s persistent workloads using k8s DevOps style
Solving k8s persistent workloads using k8s DevOps style
 
Sheepdog: yet another all in-one storage for openstack
Sheepdog: yet another all in-one storage for openstackSheepdog: yet another all in-one storage for openstack
Sheepdog: yet another all in-one storage for openstack
 
Java on the Mainframe
Java on the MainframeJava on the Mainframe
Java on the Mainframe
 
Alluxio - Scalable Filesystem Metadata Services
Alluxio - Scalable Filesystem Metadata ServicesAlluxio - Scalable Filesystem Metadata Services
Alluxio - Scalable Filesystem Metadata Services
 
Odnoklassniki.ru Architecture
Odnoklassniki.ru ArchitectureOdnoklassniki.ru Architecture
Odnoklassniki.ru Architecture
 
August 2013 HUG: Removing the NameNode's memory limitation
August 2013 HUG: Removing the NameNode's memory limitation August 2013 HUG: Removing the NameNode's memory limitation
August 2013 HUG: Removing the NameNode's memory limitation
 
Machine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkMLMachine Learning With H2O vs SparkML
Machine Learning With H2O vs SparkML
 
Architecting for failure
Architecting for failureArchitecting for failure
Architecting for failure
 

More from Benoit Perroud

Soft-Shake 2013 : Enabling Realtime Queries to End Users
Soft-Shake 2013 : Enabling Realtime Queries to End UsersSoft-Shake 2013 : Enabling Realtime Queries to End Users
Soft-Shake 2013 : Enabling Realtime Queries to End Users
Benoit Perroud
 
Apache Cassandra @Geneva JUG 2013.02.26
Apache Cassandra @Geneva JUG 2013.02.26Apache Cassandra @Geneva JUG 2013.02.26
Apache Cassandra @Geneva JUG 2013.02.26
Benoit Perroud
 
Hadoop Successes and Failures to Drive Deployment Evolution
Hadoop Successes and Failures to Drive Deployment EvolutionHadoop Successes and Failures to Drive Deployment Evolution
Hadoop Successes and Failures to Drive Deployment Evolution
Benoit Perroud
 
Cassandra talk @JUG Lausanne, 2012.06.14
Cassandra talk @JUG Lausanne, 2012.06.14Cassandra talk @JUG Lausanne, 2012.06.14
Cassandra talk @JUG Lausanne, 2012.06.14
Benoit Perroud
 
Direct memory 3_devoxx_fr_2012-2.04.18
Direct memory 3_devoxx_fr_2012-2.04.18Direct memory 3_devoxx_fr_2012-2.04.18
Direct memory 3_devoxx_fr_2012-2.04.18
Benoit Perroud
 
NoSQL overview implementation free
NoSQL overview implementation freeNoSQL overview implementation free
NoSQL overview implementation free
Benoit Perroud
 

More from Benoit Perroud (6)

Soft-Shake 2013 : Enabling Realtime Queries to End Users
Soft-Shake 2013 : Enabling Realtime Queries to End UsersSoft-Shake 2013 : Enabling Realtime Queries to End Users
Soft-Shake 2013 : Enabling Realtime Queries to End Users
 
Apache Cassandra @Geneva JUG 2013.02.26
Apache Cassandra @Geneva JUG 2013.02.26Apache Cassandra @Geneva JUG 2013.02.26
Apache Cassandra @Geneva JUG 2013.02.26
 
Hadoop Successes and Failures to Drive Deployment Evolution
Hadoop Successes and Failures to Drive Deployment EvolutionHadoop Successes and Failures to Drive Deployment Evolution
Hadoop Successes and Failures to Drive Deployment Evolution
 
Cassandra talk @JUG Lausanne, 2012.06.14
Cassandra talk @JUG Lausanne, 2012.06.14Cassandra talk @JUG Lausanne, 2012.06.14
Cassandra talk @JUG Lausanne, 2012.06.14
 
Direct memory 3_devoxx_fr_2012-2.04.18
Direct memory 3_devoxx_fr_2012-2.04.18Direct memory 3_devoxx_fr_2012-2.04.18
Direct memory 3_devoxx_fr_2012-2.04.18
 
NoSQL overview implementation free
NoSQL overview implementation freeNoSQL overview implementation free
NoSQL overview implementation free
 

Recently uploaded

Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 

Recently uploaded (20)

Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 

Direct memory jugl-2012.03.08

  • 1. JUG Lausanne 8. March 2012 Apache Direct Memory Reducing Heap Memory Stress The next battle horse for JVM performance tuning
  • 2. JUG Lausanne 8. March 2012 About me • Benoit Perroud • Apache Direct Memory Commiter • bperroud@apache.org • @killerwhile • Software craftsman • BigData Engineer @
  • 3. JUG Lausanne 8. March 2012 Today's Agenda • Off Heap Caching – Java Memory – Garbage Collector (GC) – Cache On-heap vs. Off-heap Caching • Apache Direct Memory – Design and principles – Uses cases • Multi layered cache • Standalone server “à la memcache” – Next steps • Questions
  • 4. JUG Lausanne 8. March 2012 Before starting • Sorry for my bad English and my poor French • Interrupt me anytime • I have nothing to sell. It's just worth while sharing • Please do ask questions
  • 5. JUG Lausanne 8. March 2012 Java Memory • Automatic memory allocation • Garbage collector (GC)
  • 6. JUG Lausanne 8. March 2012 Garbage Collector • Several types of GC – Serial GC – Parallel GC (throughput collector) – Concurrent Mark & Sweep GC (concurrent low pause collector) – G1 GC (low latency concurrent M&S)
  • 7. JUG Lausanne 8. March 2012 Garbage Collector • But all GC have a stop-the-world behavior • Proportional to the memory's size • Resulting in application unresponsiveness – A pain when dealing with tight SLAs
  • 8. JUG Lausanne 8. March 2012 Cache On-Heap vs. Off-Heap • On-heap – Objects tends to be promoted into tenured memory – GC storm effect when using refreshing cache – No overhead (for caching by reference)
  • 9. JUG Lausanne 8. March 2012 On-Heap vs. Off-Heap • Off-heap – Object payload is no more affecting GC – Serialization/Deserialization overhead • Hopefully lots of work on serialization has been done (Protobuf, Avro, Thrift, msgpack, BSON, ...)
  • 10. JUG Lausanne 8. March 2012 Apache Direct Memory Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance. → Opensource implementation of Terracotta BigMemory.
  • 11. JUG Lausanne 8. March 2012 Apache Direct Memory • Apache Software Foundation Incubator project • Met the Incubator falls 2011 • 12 developers ATM, 10+ contributors • I joined 1st January 2012 – was the good achievement of my Hacky Christmas Holiday :) • Disclaimer : Under heavy development – I rewrote most of the memory allocation service – APIs are subject to changes, and bugs to be found
  • 12. JUG Lausanne 8. March 2012 Design & Principles • ByteBuffer.allocateDirect is the foundation of the cache • ByteBuffers are allocated in big chunk and then splitted for internal use
  • 13. JUG Lausanne 8. March 2012 Design & Principles • Build on layers : – CachingService • Serialize object (pluggable) – MemoryManagerService • Compute access statistics – ByteBufferAllocatorService • Eventually deal with ByteBuffers
  • 14. JUG Lausanne 8. March 2012 ByteBuffers Allocation 2 different allocation's strategies • Merging ByteBuffers allocation – No memory wasted – Free at creation – Suffer from fragmentation – Need synchronization at allocation and deallocation
  • 15. JUG Lausanne 8. March 2012 ByteBuffers Allocation • Fixed size ByteBuffers allocation – Linux kernel SLAB's style allocation • Select a set of fixed sizes • Split bigger buffers (1MB+) in that size – Allocation is really fast and good concurrency • All structures is pre-instanciated – Creation (or buffer's size increase) has a cost • 1GB split in 128 bytes slabs is 8M+ buffers created – Do not suffer from fragmentation – Waste memory if the selected size is not relevant • Work really well in HDFS where all blocks are of the same size
  • 16. JUG Lausanne 8. March 2012 Use case 1 : Multi layers cache • Idea : most used objects are cached on-heap, the rest off-heap, may overflown to disk. • Sounds like BigMemory. • See net.sf.ehcache.store.offheap.OffHeapStore • Actually we inject DM in ehcache like do BigMemory. Ouch ;) • Comparison needs to be done
  • 17. JUG Lausanne 8. March 2012 Use case 2 : OffHeap Output Stream • Idea : read Twitter firehose stream without filling the precious heap memory – OOM will lead to unpredictable behavior else where in the application • From your socket directly write off-heap using OutputStream style – allocate a fixed size temporary buffer of your choice • Read from this stream – InputAndOutputStream parent class that holds both OutputStream and InputStream instances
  • 18. JUG Lausanne 8. March 2012 Use case 3 : Standalone cache server • Idea : replace Memcached :) – But with native plain REST API • DM has all the building blocks to implement such server, worth while trying • See the server submodule
  • 19. JUG Lausanne 8. March 2012 Next Steps • JSR 107 • Real Benchmarks • Builder patterns • Integration with more libs (Spring, Guice, …) • Implementations with DM lib (Cassandra (wip), Lucene, Tomcat, …) • Cache Resizing • Management and monitoring • ... • https://issues.apache.org/jira/browse/DIRECTMEMORY
  • 20. JUG Lausanne 8. March 2012 Questions ? • Thanks for you attention