Sedna XML Database: Memory Management

Ivan Shcheklein
Ivan ShchekleinEngineer at Ivan Shcheklein
Sedna: Memory Management  Nickolay Zavaritsky [email_address] Software Developer Sedna Team
Background ,[object Object],[object Object],[object Object],[object Object],[object Object]
Challenge ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 
Dereferencing DAS pointer ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
Performance Estimation ,[object Object],[object Object],[object Object],[object Object],[object Object],Following DAS pointer is nearly as fast as following a plain C pointer!    (1% overhead) 62.1 sec 63.5 sec Sedna with light DAS pointers Normal Sedna
Conclusion ,[object Object],[object Object],[object Object]
Questions?
 
1 of 10

Recommended

Sedna XML Database: Executor Internals by
Sedna XML Database: Executor InternalsSedna XML Database: Executor Internals
Sedna XML Database: Executor InternalsIvan Shcheklein
1.8K views20 slides
Sedna XML Database System: Internal Representation by
Sedna XML Database System: Internal RepresentationSedna XML Database System: Internal Representation
Sedna XML Database System: Internal RepresentationIvan Shcheklein
2.2K views19 slides
Sedna XML Database: Query Parser & Optimizing Rewriter by
Sedna XML Database: Query Parser & Optimizing RewriterSedna XML Database: Query Parser & Optimizing Rewriter
Sedna XML Database: Query Parser & Optimizing RewriterIvan Shcheklein
1.5K views21 slides
Data Retrieval over DNS in SQL Injection Attacks by
Data Retrieval over DNS in SQL Injection AttacksData Retrieval over DNS in SQL Injection Attacks
Data Retrieval over DNS in SQL Injection AttacksMiroslav Stampar
8.9K views7 slides
Parallel R in snow (english after 2nd slide) by
Parallel R in snow (english after 2nd slide)Parallel R in snow (english after 2nd slide)
Parallel R in snow (english after 2nd slide)Cdiscount
3.5K views18 slides
Percona xtra db cluster(pxc) non blocking operations, what you need to know t... by
Percona xtra db cluster(pxc) non blocking operations, what you need to know t...Percona xtra db cluster(pxc) non blocking operations, what you need to know t...
Percona xtra db cluster(pxc) non blocking operations, what you need to know t...Marco Tusa
52 views34 slides

More Related Content

What's hot

Network simulator 2 by
Network simulator 2Network simulator 2
Network simulator 2Pradeep Kumar TS
4K views20 slides
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen... by
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...BertrandDrouvot
1.1K views27 slides
Los Angeles R users group - Dec 14 2010 - Part 2 by
Los Angeles R users group - Dec 14 2010 - Part 2Los Angeles R users group - Dec 14 2010 - Part 2
Los Angeles R users group - Dec 14 2010 - Part 2rusersla
1.1K views14 slides
2013 london advanced-replication by
2013 london advanced-replication2013 london advanced-replication
2013 london advanced-replicationMarc Schwering
248 views40 slides
Session 1 introduction to ns2 by
Session 1   introduction to ns2Session 1   introduction to ns2
Session 1 introduction to ns2thenmozhi ravichandran
1.2K views24 slides
Non-Relational Postgres / Bruce Momjian (EnterpriseDB) by
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Ontico
872 views71 slides

What's hot(20)

Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen... by BertrandDrouvot
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
Reduce Resource Consumption & Clone in Seconds your Oracle Virtual Environmen...
BertrandDrouvot1.1K views
Los Angeles R users group - Dec 14 2010 - Part 2 by rusersla
Los Angeles R users group - Dec 14 2010 - Part 2Los Angeles R users group - Dec 14 2010 - Part 2
Los Angeles R users group - Dec 14 2010 - Part 2
rusersla1.1K views
2013 london advanced-replication by Marc Schwering
2013 london advanced-replication2013 london advanced-replication
2013 london advanced-replication
Marc Schwering248 views
Non-Relational Postgres / Bruce Momjian (EnterpriseDB) by Ontico
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Non-Relational Postgres / Bruce Momjian (EnterpriseDB)
Ontico872 views
Replica Sets (NYC NoSQL Meetup) by MongoDB
Replica Sets (NYC NoSQL Meetup)Replica Sets (NYC NoSQL Meetup)
Replica Sets (NYC NoSQL Meetup)
MongoDB981 views
Squeak DBX by ESUG
Squeak DBXSqueak DBX
Squeak DBX
ESUG2.2K views
How and Where in GLORP by ESUG
How and Where in GLORPHow and Where in GLORP
How and Where in GLORP
ESUG2.8K views
Gur1009 by Cdiscount
Gur1009Gur1009
Gur1009
Cdiscount5.5K views
Getting started with Clojure by John Stevenson
Getting started with ClojureGetting started with Clojure
Getting started with Clojure
John Stevenson1.7K views
Cassandra summit 2013 - DataStax Java Driver Unleashed! by Michaël Figuière
Cassandra summit 2013 - DataStax Java Driver Unleashed!Cassandra summit 2013 - DataStax Java Driver Unleashed!
Cassandra summit 2013 - DataStax Java Driver Unleashed!
Michaël Figuière3.5K views
MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering... by MongoDB
MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering...MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering...
MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering...
MongoDB831 views
Teaching PostgreSQL to new people by Tomek Borek
Teaching PostgreSQL to new peopleTeaching PostgreSQL to new people
Teaching PostgreSQL to new people
Tomek Borek829 views
data loading and unloading in IBM Netezza by www.etraining.guru by Ravikumar Nandigam
data loading and unloading in IBM Netezza by www.etraining.gurudata loading and unloading in IBM Netezza by www.etraining.guru
data loading and unloading in IBM Netezza by www.etraining.guru
Ravikumar Nandigam783 views

Similar to Sedna XML Database: Memory Management

Modern software design in Big data era by
Modern software design in Big data eraModern software design in Big data era
Modern software design in Big data eraBill GU
618 views29 slides
Cassandra for Sysadmins by
Cassandra for SysadminsCassandra for Sysadmins
Cassandra for SysadminsNathan Milford
26.6K views42 slides
Scaling with MongoDB by
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDBRick Copeland
15.2K views32 slides
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft... by
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...DataStax
3.8K views29 slides
Performance and Predictability - Richard Warburton by
Performance and Predictability - Richard WarburtonPerformance and Predictability - Richard Warburton
Performance and Predictability - Richard WarburtonJAXLondon2014
778 views70 slides
Performance and predictability (1) by
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)RichardWarburton
1.1K views70 slides

Similar to Sedna XML Database: Memory Management(20)

Modern software design in Big data era by Bill GU
Modern software design in Big data eraModern software design in Big data era
Modern software design in Big data era
Bill GU618 views
Cassandra for Sysadmins by Nathan Milford
Cassandra for SysadminsCassandra for Sysadmins
Cassandra for Sysadmins
Nathan Milford26.6K views
Scaling with MongoDB by Rick Copeland
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDB
Rick Copeland15.2K views
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft... by DataStax
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...
Running 400-node Cassandra + Spark Clusters in Azure (Anubhav Kale, Microsoft...
DataStax3.8K views
Performance and Predictability - Richard Warburton by JAXLondon2014
Performance and Predictability - Richard WarburtonPerformance and Predictability - Richard Warburton
Performance and Predictability - Richard Warburton
JAXLondon2014778 views
Performance and predictability (1) by RichardWarburton
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)
RichardWarburton1.1K views
Memory-Based Cloud Architectures by 小新 制造
Memory-Based Cloud ArchitecturesMemory-Based Cloud Architectures
Memory-Based Cloud Architectures
小新 制造502 views
A Paradigm Shift: The Increasing Dominance of Memory-Oriented Solutions for H... by Ben Stopford
A Paradigm Shift: The Increasing Dominance of Memory-Oriented Solutions for H...A Paradigm Shift: The Increasing Dominance of Memory-Oriented Solutions for H...
A Paradigm Shift: The Increasing Dominance of Memory-Oriented Solutions for H...
Ben Stopford5.5K views
Cassandra TK 2014 - Large Nodes by aaronmorton
Cassandra TK 2014 - Large NodesCassandra TK 2014 - Large Nodes
Cassandra TK 2014 - Large Nodes
aaronmorton5.2K views
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability by Ben Stopford
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBeyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Ben Stopford8.4K views
IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's C... by npinto
IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's C...IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's C...
IAP09 CUDA@MIT 6.963 - Guest Lecture: Out-of-Core Programming with NVIDIA's C...
npinto2.1K views
Sql server engine cpu cache as the new ram by Chris Adkin
Sql server engine cpu cache as the new ramSql server engine cpu cache as the new ram
Sql server engine cpu cache as the new ram
Chris Adkin2.7K views
Silicon Valley Code Camp: 2011 Introduction to MongoDB by Manish Pandit
Silicon Valley Code Camp: 2011 Introduction to MongoDBSilicon Valley Code Camp: 2011 Introduction to MongoDB
Silicon Valley Code Camp: 2011 Introduction to MongoDB
Manish Pandit1.2K views
Intuitions for scaling data centric architectures - Benjamin Stopford by JAXLondon_Conference
Intuitions for scaling data centric architectures - Benjamin StopfordIntuitions for scaling data centric architectures - Benjamin Stopford
Intuitions for scaling data centric architectures - Benjamin Stopford
Modeling data and best practices for the Azure Cosmos DB. by Mohammad Asif
Modeling data and best practices for the Azure Cosmos DB.Modeling data and best practices for the Azure Cosmos DB.
Modeling data and best practices for the Azure Cosmos DB.
Mohammad Asif1.6K views

Recently uploaded

Attacking IoT Devices from a Web Perspective - Linux Day by
Attacking IoT Devices from a Web Perspective - Linux Day Attacking IoT Devices from a Web Perspective - Linux Day
Attacking IoT Devices from a Web Perspective - Linux Day Simone Onofri
15 views68 slides
Melek BEN MAHMOUD.pdf by
Melek BEN MAHMOUD.pdfMelek BEN MAHMOUD.pdf
Melek BEN MAHMOUD.pdfMelekBenMahmoud
14 views1 slide
Spesifikasi Lengkap ASUS Vivobook Go 14 by
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14Dot Semarang
35 views1 slide
handbook for web 3 adoption.pdf by
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdfLiveplex
19 views16 slides
Tunable Laser (1).pptx by
Tunable Laser (1).pptxTunable Laser (1).pptx
Tunable Laser (1).pptxHajira Mahmood
23 views37 slides
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng Tsze by
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng TszeDigital Product-Centric Enterprise and Enterprise Architecture - Tan Eng Tsze
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng TszeNUS-ISS
19 views47 slides

Recently uploaded(20)

Attacking IoT Devices from a Web Perspective - Linux Day by Simone Onofri
Attacking IoT Devices from a Web Perspective - Linux Day Attacking IoT Devices from a Web Perspective - Linux Day
Attacking IoT Devices from a Web Perspective - Linux Day
Simone Onofri15 views
Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang35 views
handbook for web 3 adoption.pdf by Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex19 views
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng Tsze by NUS-ISS
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng TszeDigital Product-Centric Enterprise and Enterprise Architecture - Tan Eng Tsze
Digital Product-Centric Enterprise and Enterprise Architecture - Tan Eng Tsze
NUS-ISS19 views
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu... by NUS-ISS
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
NUS-ISS37 views
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software225 views
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk88 views
.conf Go 2023 - Data analysis as a routine by Splunk
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
Splunk93 views
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman27 views
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen... by NUS-ISS
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
NUS-ISS28 views
[2023] Putting the R! in R&D.pdf by Eleanor McHugh
[2023] Putting the R! in R&D.pdf[2023] Putting the R! in R&D.pdf
[2023] Putting the R! in R&D.pdf
Eleanor McHugh38 views
Black and White Modern Science Presentation.pptx by maryamkhalid2916
Black and White Modern Science Presentation.pptxBlack and White Modern Science Presentation.pptx
Black and White Modern Science Presentation.pptx
maryamkhalid291614 views
RADIUS-Omnichannel Interaction System by RADIUS
RADIUS-Omnichannel Interaction SystemRADIUS-Omnichannel Interaction System
RADIUS-Omnichannel Interaction System
RADIUS15 views
Perth MeetUp November 2023 by Michael Price
Perth MeetUp November 2023 Perth MeetUp November 2023
Perth MeetUp November 2023
Michael Price15 views

Sedna XML Database: Memory Management

Editor's Notes

  1. I am going to talk a bit about memory management in databases in general and then I will focus on Sedna-specific details. It is convenient to introduce a ‘Database Address Space’ concept when talking about memory management. Basicly all database objects – I mean low level objects like individual records – must be somehow uniquely identified. Each low level object is given a address – a unique integral number used to locate the object. The most basic design is to store DB as a single file. In this case we can use an offset of the object in the database file as it’s address. DAS is the set of all valid database objects addresses. Huge datasets are stored in databases – need large enough DAS. We can not work with data on HDD directly. We have to upload data in the main memory first. The common design is to allocate a dedicated chunk of memory for this purpose (so-called ‘buffer memory’). A data block is loaded from HDD in the buffer. DBMS works with the data. When DBMS is running low on buffer memory it starts swapping blocks back to disk. Generally buffer memory is locked so that OS can not swap it. DBMS has a better knowledge of the data-access pattern and hence can manage swap process more efficiently. (Swapping is not nessesary tied to low memory condition. For instance a background flusher can be implemented.) DAS pointer is passed to the buffer manager when asking for data. The manager either find the requested data in buffers or not. In the latter case the data location on disk is computed and IO occurs. If DAS pointers are followed infrequently the performance of the pointer dereference operation isn’t critical. However if DAS pointers are followed frequently and the referenced data is nearly always availible from buffers we have to optimize the DAS pointer dereference operation. This is the Sedna case – we are using internal XML representation with a large amount of interconnecting pointers.