SlideShare a Scribd company logo
An illustrated guide to
  the buffer cache
               Selena Deckelmann
  Based on Greg Smith’s “Inside the Buffer Cache”
http://www.westnet.com/∼gsmith/content/postgresql
Table

Databases                        Table


                                 Table



     Databases are made of tables...
$PGDATA/base/                 Table




    A table is just a directory...
$PGDATA/base/               Table


                            Table

                            Table


  You can have more than one table.
$PGDATA/base/   Table

                Table

                Table   Made of           Made o
                                                   f          8K
                                  File                        8K
                                                              8K
                                  File
                                                              8K
                                                              8K
                                  File
                                                              8K
                                  File                        8K
                                                              ...
                                   ...                 Up to 1 GB

                Files are made up of 8K blocks.*

                                                         * you can change that at compile time.
shared_buffers
shared_buffers

       ptr   ptr   ptr   ptr   ptr   ...

      An array of pointers to 8K blocks
           (and some other stuff)

shared_buffers is configured at database start.
  You have to restart Postgres to change it.
shared_buffers:
 the other stuff
FREE FREE FREE FREE FREE FREE



       In the beginning...
shared_buffers:
 the other stuff
FREE FREE FREE FREE FREE FREE
 456  12   100 612   32   409
shared_buffers:
the other stuff
 X    X     X     X    X     X
456   12   100   612   32   409
shared_buffers:
         the other stuff
         X         X         X         X         X         X
Tag   base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

       456        12       100       612        32       409
shared_buffers:
            the other stuff
            X         X         X         X         X         X
  Tag    base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

          456        12       100       612        32       409
Status   FLAGS FLAGS FLAGS FLAGS FLAGS FLAGS
shared_buffers:
            the other stuff
            X         X         X         X         X         X
  Tag    base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

          456        12       100       612        32       409
Status   PINNED     flags      flags      flags      flags      flags




                    {                                                  }
                           UPDATE mytable (status)
                           VALUES(‘my cat is the best ever’)
                           where user = ‘selenamarie’;
shared_buffers:
            the other stuff
            X         X         X         X         X         X
  Tag    base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

          456        12       100       612        32       409
Status   PINNED    DIRTY flags           flags      flags      flags




                                  {     COMMIT;
                                                                       }
shared_buffers:
            the other stuff
            X         X         X         X         X         X
  Tag    base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

          456        12       100       612        32       409
Status   PINNED    DIRTY OTHER OTHER OTHER OTHER




                                           {     BLAH BLAH BLAH;
                                                                       }
shared_buffers:
            the other stuff
            X         X         X         X         X         X
  Tag    base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

          456        12       100       612        32       409
Status   PINNED    DIRTY OTHER OTHER OTHER OTHER
Usage       1        5     5     4     3     0
shared_buffers:
     the lifecycle
   X         X         X         X         X         X
base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

 456        12       100       612        32       409
PINNED    DIRTY OTHER OTHER OTHER OTHER
   1        5     5     4     3     0



           {      myPgProcess calls:
                  BufferAlloc(‘base/file1’, 100);              }
shared_buffers:
     the lifecycle
   X         X         X         X         X         X
base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

 456        12       100       612        32       409
PINNED    DIRTY     PINNED    OTHER OTHER OTHER
   1        5          5         4         3         0



           {      myPgProcess calls:
                  BufferAlloc(‘base/file1’, 100);              }
shared_buffers:
     the lifecycle
   X         X         X         X         X         X
base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

 456        12       100       612        32       409
PINNED    DIRTY OTHER OTHER OTHER OTHER
   1        5     5     4     3     0



           {      myPgProcess calls:
                  BufferAlloc(‘base/file1’, 101);              }
shared_buffers:
                 the lifecycle
                      Who’s it gonna be?
               X         X         X         X         X         X
            base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

             456        12       100       612        32       409
            PINNED    DIRTY OTHER OTHER OTHER OTHER
               1        5     5     4     3     0


Eviction!              {      myPgProcess calls:
                              BufferAlloc(‘base/file1’, 101);              }
shared_buffers:
                the lifecycle
                     Who’s it gonna be?
              X         X         X         X         X         X
           base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

            456        12       100       612        32       409
           PINNED    DIRTY OTHER OTHER OTHER OTHER
              0        5     5     4     3     0

Scan the cache!
  Usage != 0,
so decrement.
                      {      myPgProcess calls:
                             BufferAlloc(‘base/file1’, 101);              }
shared_buffers:
     the lifecycle
          Who’s it gonna be?
   X         X         X         X         X         X
base/file1 base/file2 base/file1 base/file3 base/file1 base/file1   Usage == 0,
 456        12       100       612        32       409         Goodbye!
PINNED    DIRTY OTHER OTHER OTHER OTHER
   0        4     4     3     2     0



           {      myPgProcess calls:
                  BufferAlloc(‘base/file1’, 101);                      }
shared_buffers:
     the lifecycle
          Who’s it gonna be?
   X         X         X         X         X         X
base/file1 base/file2 base/file1 base/file3 base/file1 base/file1

 456        12       100       612        32       101
PINNED    DIRTY OTHER OTHER OTHER                 PINNED

   0        4     4     3     2                      1



           {      myPgProcess calls:
                  BufferAlloc(‘base/file1’, 101);              }
shared_buffers:
                   optimizations*
         X         X         X         X         X         X         X         ...
      base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2    ...

       456        12       100       612        32       101       891         ...
      PINNED    DIRTY OTHER OTHER OTHER PINNED OTHER                           ...
         0        4     4     3     2      0     1                             ...



                           {                                                              }
Result is greater
      than:                       SELECT pg_relation_size(‘mytable’);
shared_buffers/4                  VACUUM mytable;
                                                                       * Version 8.3 and later
shared_buffers:
                    optimizations
         X         X         X         X         X         X         X         ...
      base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2    ...

        456       12       100       612        32       101       891         ...
      PINNED    DIRTY OTHER OTHER OTHER PINNED OTHER                           ...
         0        4     4     3     2      0     1                             ...



   256K ring
buffer allocated!
                           {      SELECT pg_relation_size(‘mytable’);
                                  VACUUM mytable;                                         }
                                                                       * Version 8.3 and later
shared_buffers:
                    optimizations VACUUM
                                  Safe during

        OVERSIMPLIFICATION WARNING!
         X         X         X         X         X         X         X         ...
      base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2    ...

        456       12       100       612        32       101       891         ...
      PINNED    DIRTY OTHER OTHER OTHER PINNED OTHER                           ...
         0        4     4     3     2      0     1                             ...



   256K ring
buffer allocated!
                           {      SELECT pg_relation_size(‘mytable’);
                                  VACUUM mytable;                                         }
                                                                       * Version 8.3 and later

More Related Content

Viewers also liked

Martin karlssons vykortssamling sigtunastiftelsen
Martin karlssons vykortssamling   sigtunastiftelsenMartin karlssons vykortssamling   sigtunastiftelsen
Martin karlssons vykortssamling sigtunastiftelsenhembygdsigtuna
 
Measuring Social Media For B2B
Measuring Social Media For B2BMeasuring Social Media For B2B
Measuring Social Media For B2B
Jon Gatrell
 
Martin karlssons vykortssamling flygfoton
Martin karlssons vykortssamling   flygfotonMartin karlssons vykortssamling   flygfoton
Martin karlssons vykortssamling flygfotonhembygdsigtuna
 
Arduino yun × apiで遊んでみる
Arduino yun × apiで遊んでみるArduino yun × apiで遊んでみる
Arduino yun × apiで遊んでみる
Atsushi Nakatsugawa
 
Christmas is for Cookies
Christmas is for CookiesChristmas is for Cookies
Christmas is for Cookies
Jon Gatrell
 
Securing Business: Strategic Enablement of Users
Securing Business: Strategic Enablement of UsersSecuring Business: Strategic Enablement of Users
Securing Business: Strategic Enablement of Users
Jon Gatrell
 
Chrome拡張開発者のためのFirefox拡張開発
Chrome拡張開発者のためのFirefox拡張開発Chrome拡張開発者のためのFirefox拡張開発
Chrome拡張開発者のためのFirefox拡張開発
swdyh
 
Product Management: Site & Situation
Product Management: Site & SituationProduct Management: Site & Situation
Product Management: Site & Situation
Jon Gatrell
 
2009 Meet The Principal
2009 Meet The Principal2009 Meet The Principal
2009 Meet The Principal
ph0enix74
 
The teacher librarian as resource creator: Developing learning websites
The teacher librarian as resource creator: Developing learning websitesThe teacher librarian as resource creator: Developing learning websites
The teacher librarian as resource creator: Developing learning websites
jherring
 
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えしますプッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
Atsushi Nakatsugawa
 

Viewers also liked (12)

Martin karlssons vykortssamling sigtunastiftelsen
Martin karlssons vykortssamling   sigtunastiftelsenMartin karlssons vykortssamling   sigtunastiftelsen
Martin karlssons vykortssamling sigtunastiftelsen
 
Measuring Social Media For B2B
Measuring Social Media For B2BMeasuring Social Media For B2B
Measuring Social Media For B2B
 
Martin karlssons vykortssamling flygfoton
Martin karlssons vykortssamling   flygfotonMartin karlssons vykortssamling   flygfoton
Martin karlssons vykortssamling flygfoton
 
Arduino yun × apiで遊んでみる
Arduino yun × apiで遊んでみるArduino yun × apiで遊んでみる
Arduino yun × apiで遊んでみる
 
Christmas is for Cookies
Christmas is for CookiesChristmas is for Cookies
Christmas is for Cookies
 
Securing Business: Strategic Enablement of Users
Securing Business: Strategic Enablement of UsersSecuring Business: Strategic Enablement of Users
Securing Business: Strategic Enablement of Users
 
Chrome拡張開発者のためのFirefox拡張開発
Chrome拡張開発者のためのFirefox拡張開発Chrome拡張開発者のためのFirefox拡張開発
Chrome拡張開発者のためのFirefox拡張開発
 
Product Management: Site & Situation
Product Management: Site & SituationProduct Management: Site & Situation
Product Management: Site & Situation
 
2009 Meet The Principal
2009 Meet The Principal2009 Meet The Principal
2009 Meet The Principal
 
The teacher librarian as resource creator: Developing learning websites
The teacher librarian as resource creator: Developing learning websitesThe teacher librarian as resource creator: Developing learning websites
The teacher librarian as resource creator: Developing learning websites
 
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えしますプッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
プッシュからデータ保存まで。アプリ開発でニフティクラウド mobile backendを使う上での良くある質問、疑問にお答えします
 
Prietenior Mei...
Prietenior Mei...Prietenior Mei...
Prietenior Mei...
 

More from Selena Deckelmann

While we're here, let's fix computer science education
While we're here, let's fix computer science educationWhile we're here, let's fix computer science education
While we're here, let's fix computer science education
Selena Deckelmann
 
Algorithms are Recipes
Algorithms are RecipesAlgorithms are Recipes
Algorithms are Recipes
Selena Deckelmann
 
Hire the right way
Hire the right wayHire the right way
Hire the right way
Selena Deckelmann
 
Mistakes were made - LCA 2012
Mistakes were made - LCA 2012Mistakes were made - LCA 2012
Mistakes were made - LCA 2012
Selena Deckelmann
 
Pg92 HA, LCA 2012, Ballarat
Pg92 HA, LCA 2012, BallaratPg92 HA, LCA 2012, Ballarat
Pg92 HA, LCA 2012, Ballarat
Selena Deckelmann
 
Managing terabytes
Managing terabytesManaging terabytes
Managing terabytes
Selena Deckelmann
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
Selena Deckelmann
 
Postgres needs an aircraft carrier
Postgres needs an aircraft carrierPostgres needs an aircraft carrier
Postgres needs an aircraft carrier
Selena Deckelmann
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
Selena Deckelmann
 
Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1
Selena Deckelmann
 
How to ask for money
How to ask for moneyHow to ask for money
How to ask for money
Selena Deckelmann
 
Letters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres communityLetters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres community
Selena Deckelmann
 
Own it: working with a changing open source community
Own it: working with a changing open source communityOwn it: working with a changing open source community
Own it: working with a changing open source community
Selena Deckelmann
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
Selena Deckelmann
 
Managing terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets bigManaging terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets big
Selena Deckelmann
 
Pdxpugday2010 pg90
Pdxpugday2010 pg90Pdxpugday2010 pg90
Pdxpugday2010 pg90
Selena Deckelmann
 
Making Software Communities
Making Software CommunitiesMaking Software Communities
Making Software Communities
Selena Deckelmann
 
Bucardo
BucardoBucardo
How a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged ElectionHow a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged Election
Selena Deckelmann
 
Open Source Bridge Opening Day
Open Source Bridge Opening DayOpen Source Bridge Opening Day
Open Source Bridge Opening Day
Selena Deckelmann
 

More from Selena Deckelmann (20)

While we're here, let's fix computer science education
While we're here, let's fix computer science educationWhile we're here, let's fix computer science education
While we're here, let's fix computer science education
 
Algorithms are Recipes
Algorithms are RecipesAlgorithms are Recipes
Algorithms are Recipes
 
Hire the right way
Hire the right wayHire the right way
Hire the right way
 
Mistakes were made - LCA 2012
Mistakes were made - LCA 2012Mistakes were made - LCA 2012
Mistakes were made - LCA 2012
 
Pg92 HA, LCA 2012, Ballarat
Pg92 HA, LCA 2012, BallaratPg92 HA, LCA 2012, Ballarat
Pg92 HA, LCA 2012, Ballarat
 
Managing terabytes
Managing terabytesManaging terabytes
Managing terabytes
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
 
Postgres needs an aircraft carrier
Postgres needs an aircraft carrierPostgres needs an aircraft carrier
Postgres needs an aircraft carrier
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
 
Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1
 
How to ask for money
How to ask for moneyHow to ask for money
How to ask for money
 
Letters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres communityLetters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres community
 
Own it: working with a changing open source community
Own it: working with a changing open source communityOwn it: working with a changing open source community
Own it: working with a changing open source community
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
 
Managing terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets bigManaging terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets big
 
Pdxpugday2010 pg90
Pdxpugday2010 pg90Pdxpugday2010 pg90
Pdxpugday2010 pg90
 
Making Software Communities
Making Software CommunitiesMaking Software Communities
Making Software Communities
 
Bucardo
BucardoBucardo
Bucardo
 
How a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged ElectionHow a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged Election
 
Open Source Bridge Opening Day
Open Source Bridge Opening DayOpen Source Bridge Opening Day
Open Source Bridge Opening Day
 

Recently uploaded

"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Sease
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 

Recently uploaded (20)

"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMsFrom Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 

Illustrated buffer cache

  • 1. An illustrated guide to the buffer cache Selena Deckelmann Based on Greg Smith’s “Inside the Buffer Cache” http://www.westnet.com/∼gsmith/content/postgresql
  • 2. Table Databases Table Table Databases are made of tables...
  • 3. $PGDATA/base/ Table A table is just a directory...
  • 4. $PGDATA/base/ Table Table Table You can have more than one table.
  • 5. $PGDATA/base/ Table Table Table Made of Made o f 8K File 8K 8K File 8K 8K File 8K File 8K ... ... Up to 1 GB Files are made up of 8K blocks.* * you can change that at compile time.
  • 7. shared_buffers ptr ptr ptr ptr ptr ... An array of pointers to 8K blocks (and some other stuff) shared_buffers is configured at database start. You have to restart Postgres to change it.
  • 8. shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE In the beginning...
  • 9. shared_buffers: the other stuff FREE FREE FREE FREE FREE FREE 456 12 100 612 32 409
  • 10. shared_buffers: the other stuff X X X X X X 456 12 100 612 32 409
  • 11. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409
  • 12. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status FLAGS FLAGS FLAGS FLAGS FLAGS FLAGS
  • 13. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED flags flags flags flags flags { } UPDATE mytable (status) VALUES(‘my cat is the best ever’) where user = ‘selenamarie’;
  • 14. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY flags flags flags flags { COMMIT; }
  • 15. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER { BLAH BLAH BLAH; }
  • 16. shared_buffers: the other stuff X X X X X X Tag base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 Status PINNED DIRTY OTHER OTHER OTHER OTHER Usage 1 5 5 4 3 0
  • 17. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
  • 18. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY PINNED OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 100); }
  • 19. shared_buffers: the lifecycle X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 20. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 1 5 5 4 3 0 Eviction! { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 21. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 409 PINNED DIRTY OTHER OTHER OTHER OTHER 0 5 5 4 3 0 Scan the cache! Usage != 0, so decrement. { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 22. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 Usage == 0, 456 12 100 612 32 409 Goodbye! PINNED DIRTY OTHER OTHER OTHER OTHER 0 4 4 3 2 0 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 23. shared_buffers: the lifecycle Who’s it gonna be? X X X X X X base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 456 12 100 612 32 101 PINNED DIRTY OTHER OTHER OTHER PINNED 0 4 4 3 2 1 { myPgProcess calls: BufferAlloc(‘base/file1’, 101); }
  • 24. shared_buffers: optimizations* X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... { } Result is greater than: SELECT pg_relation_size(‘mytable’); shared_buffers/4 VACUUM mytable; * Version 8.3 and later
  • 25. shared_buffers: optimizations X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later
  • 26. shared_buffers: optimizations VACUUM Safe during OVERSIMPLIFICATION WARNING! X X X X X X X ... base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ... 456 12 100 612 32 101 891 ... PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ... 0 4 4 3 2 0 1 ... 256K ring buffer allocated! { SELECT pg_relation_size(‘mytable’); VACUUM mytable; } * Version 8.3 and later