SlideShare a Scribd company logo
Selective Main Memory Compression by
identifying Program Phase Changes
Doron Nakar and Shlomo Weiss
Tel Aviv University
Reported by jiang
Background
During a program’s runtime, the stack and data segments
of the main memory often contain much redundancy.
Compress these memory can offer obvious benefits of
increasing the overall memory availability and reducing
disk accesses.
Background
•Work by this paper
Presents a new approach to compress memory
selectively by not compressing all the memory all of the
time.
Identify the working Sets which exhibit good spatial
locality, and leave it uncompressed for faster access.
4
Main Idea
The runtime program is divided into windows of fixed size.
After a window finishes executing, the segments executed
will be hashed in to a array by setting the corresponding bit
to ‘1’.
Seg 0
Seg 1
.
.
.
Seg k
Bit 0
Bit 1
.
.
.
Bit n
Instructions window Hash
A
r
r
a
y
.
Main Idea
•Then the current hash array will be compared with previous array to
get a difference factor δ .
=
•Lower δ value means less difference, as well as better locality,
which means contain much redundancy to compress.
•Pages that are in the program’s page table but not in its working set
is the candidate to be compressed.
Flowchart
Run “Window size” instructions
mark the hash array
record segment address
Calculate δ
δ small
eno
ugh
?
Replace previous table to current
Trigger compression
Clear current table
Locality algorithm evaluation
A surprise occurs when if neither bit is on or if the address
stored in the current hash table doesn’t match
the requested address. More surprises mean
less spatial locality.
Locality algorithm evaluation
“gzip” allocates and clears 60 Mbytes of space on the
heap but never uses it. And it causes abnormally
high compression ratios due to many pages that
contain only zeros.
we had to disregard pages that contain only zeros and
not take them into account in the compression
results.
9
Compression method
• Hash array can be compared to get the locality ratio, however ,to get
the addresses accessed, we need expand hash array by adding
additional information to determine, during a stable phase of the
program, which pages are unused and may be compressed.
Bit 0
Bit 1
.
.
.
Bit n
.
Expand
Bit 0
Bit 1
.
.
.
Bit n
.
Address
Address
.
.
.
Address
.
Compression method
• Compressing all the pages that are in the program’s page table but
not in its working set(hash table).
0
1
1
.
.
0
.
Null
Address
Address
.
.
Null
.
Not to be compressed!
PERFORMANCE RESULTS
1. At the end of every instruction window: the
compression ratio is calculated.
2. The percentage of memory in compressed form.
3. The number of decompressed pages per instruction
window
PERFORMANCE RESULTS
eon: Average δ Value is 0.0760
crafty: Average δ Value is 0.0200
Thanks for your appreciation!

More Related Content

Similar to Selective main memory compression by identifying program phase

Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation Hadoop
Varun Narang
 
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Yahoo Developer Network
 

Similar to Selective main memory compression by identifying program phase (20)

Consistent hashing algorithmic tradeoffs
Consistent hashing  algorithmic tradeoffsConsistent hashing  algorithmic tradeoffs
Consistent hashing algorithmic tradeoffs
 
Seminar Presentation Hadoop
Seminar Presentation HadoopSeminar Presentation Hadoop
Seminar Presentation Hadoop
 
Hadoop
HadoopHadoop
Hadoop
 
Hadoop live online training
Hadoop live online trainingHadoop live online training
Hadoop live online training
 
Introduccion a Hadoop / Introduction to Hadoop
Introduccion a Hadoop / Introduction to HadoopIntroduccion a Hadoop / Introduction to Hadoop
Introduccion a Hadoop / Introduction to Hadoop
 
Learn what is Hadoop-and-BigData
Learn  what is Hadoop-and-BigDataLearn  what is Hadoop-and-BigData
Learn what is Hadoop-and-BigData
 
Tech Talk - JPA and Query Optimization - publish
Tech Talk  -  JPA and Query Optimization - publishTech Talk  -  JPA and Query Optimization - publish
Tech Talk - JPA and Query Optimization - publish
 
GraphChi big graph processing
GraphChi big graph processingGraphChi big graph processing
GraphChi big graph processing
 
Heap Memory Management.pptx
Heap Memory Management.pptxHeap Memory Management.pptx
Heap Memory Management.pptx
 
Hadoop training-in-hyderabad
Hadoop training-in-hyderabadHadoop training-in-hyderabad
Hadoop training-in-hyderabad
 
Architecting and productionising data science applications at scale
Architecting and productionising data science applications at scaleArchitecting and productionising data science applications at scale
Architecting and productionising data science applications at scale
 
Designing data intensive applications
Designing data intensive applicationsDesigning data intensive applications
Designing data intensive applications
 
Ch22 parallel d_bs_cs561
Ch22 parallel d_bs_cs561Ch22 parallel d_bs_cs561
Ch22 parallel d_bs_cs561
 
Virtual Memory Management
Virtual Memory ManagementVirtual Memory Management
Virtual Memory Management
 
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
Apache Hadoop India Summit 2011 talk "Hadoop Map-Reduce Programming & Best Pr...
 
Memory management
Memory managementMemory management
Memory management
 
Improving application performance through swap compression
Improving application performance through swap compression Improving application performance through swap compression
Improving application performance through swap compression
 
Unix Memory Management - Operating Systems
Unix Memory Management - Operating SystemsUnix Memory Management - Operating Systems
Unix Memory Management - Operating Systems
 
Seminar_Report_hadoop
Seminar_Report_hadoopSeminar_Report_hadoop
Seminar_Report_hadoop
 
Google File System
Google File SystemGoogle File System
Google File System
 

Recently uploaded

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 

Recently uploaded (20)

IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 

Selective main memory compression by identifying program phase

  • 1. Selective Main Memory Compression by identifying Program Phase Changes Doron Nakar and Shlomo Weiss Tel Aviv University Reported by jiang
  • 2. Background During a program’s runtime, the stack and data segments of the main memory often contain much redundancy. Compress these memory can offer obvious benefits of increasing the overall memory availability and reducing disk accesses.
  • 3. Background •Work by this paper Presents a new approach to compress memory selectively by not compressing all the memory all of the time. Identify the working Sets which exhibit good spatial locality, and leave it uncompressed for faster access.
  • 4. 4 Main Idea The runtime program is divided into windows of fixed size. After a window finishes executing, the segments executed will be hashed in to a array by setting the corresponding bit to ‘1’. Seg 0 Seg 1 . . . Seg k Bit 0 Bit 1 . . . Bit n Instructions window Hash A r r a y .
  • 5. Main Idea •Then the current hash array will be compared with previous array to get a difference factor δ . = •Lower δ value means less difference, as well as better locality, which means contain much redundancy to compress. •Pages that are in the program’s page table but not in its working set is the candidate to be compressed.
  • 6. Flowchart Run “Window size” instructions mark the hash array record segment address Calculate δ δ small eno ugh ? Replace previous table to current Trigger compression Clear current table
  • 7. Locality algorithm evaluation A surprise occurs when if neither bit is on or if the address stored in the current hash table doesn’t match the requested address. More surprises mean less spatial locality.
  • 8. Locality algorithm evaluation “gzip” allocates and clears 60 Mbytes of space on the heap but never uses it. And it causes abnormally high compression ratios due to many pages that contain only zeros. we had to disregard pages that contain only zeros and not take them into account in the compression results.
  • 9. 9 Compression method • Hash array can be compared to get the locality ratio, however ,to get the addresses accessed, we need expand hash array by adding additional information to determine, during a stable phase of the program, which pages are unused and may be compressed. Bit 0 Bit 1 . . . Bit n . Expand Bit 0 Bit 1 . . . Bit n . Address Address . . . Address .
  • 10. Compression method • Compressing all the pages that are in the program’s page table but not in its working set(hash table). 0 1 1 . . 0 . Null Address Address . . Null . Not to be compressed!
  • 11. PERFORMANCE RESULTS 1. At the end of every instruction window: the compression ratio is calculated. 2. The percentage of memory in compressed form. 3. The number of decompressed pages per instruction window
  • 12. PERFORMANCE RESULTS eon: Average δ Value is 0.0760 crafty: Average δ Value is 0.0200
  • 13. Thanks for your appreciation!