SlideShare a Scribd company logo
META FIELDSby Bo Andersen - codingexplained.com
OUTLINE
➤ Introduction to meta fields
➤ Meta fields categories
➤ Identity
➤ Document source
➤ Indexing
➤ Routing
➤ Other
INTRODUCTION TO META FIELDS
➤ Each document has metadata associated with it
➤ E.g. the _index, _type, and _id meta fields.
➤ The behavior of some of these meta-fields can be customized
➤ This is done when creating a mapping type
IDENTITY
META FIELDS
IDENTITY META FIELDS (PART 1)
➤ _index
➤ Allows matching documents based on their indexes
➤ _index is exposed as a virtual field and is not added to the Lucene index as a real
field
➤ _type
➤ The type of the document
➤ Indexed in order to make searching by type name fast
IDENTITY META FIELDS (PART 2)
➤ _id
➤ The ID of the document
➤ Not indexed, because its value can be automatically derived from the _uid field
➤ _uid
➤ The values of _type and _id are combined as {type}#{id} and indexed
DOCUMENT
SOURCE
META
FIELDS
DOCUMENT SOURCE META FIELDS
➤ _source
➤ Contains the JSON that was passed at index time (unless updated)
➤ The field is not indexed and is therefore not searchable
➤ It is stored so that it can be returned when executing fetch requests
➤ Can be disabled to save storage space
➤ _size
➤ Indexes the size of the _source field in bytes
➤ Available as a plugin that must be installed before the field is available
➤ sudo bin/plugin install mapper-size
INDEXING
META
FIELDS
_ALL
➤ A special field that concatenates the values of all of the other fields into one string
➤ Uses space as a delimiter
➤ The field is analyzed and indexed, but not stored
➤ This means that it can be searched, but not retrieved
➤ The field allows you to search for values in documents without knowing which field
contains the value
_FIELD_NAMES
➤ Indexes the names of every field in a document that contains a value other than
NULL
➤ Used by the exists and missing queries
➤ To find documents that either have or do not have a non-null value for a particular
field
ROUTING
META
FIELDS
_PARENT
➤ Can be used to establish a parent-child relationship between documents in an index
➤ This is done by making one mapping type the parent of another
"mappings": {
"employee": {
"_parent": {
"type": "person"
}
}
}
When adding an index:
_ROUTING
➤ Used to route a document to a particular shard within an index
➤ Enables you to define rules that control in which shards documents are stored
➤ For advanced use cases!
OTHER
META
FIELDS
_META
➤ Each mapping type can have custom meta data associated with it
➤ Elasticsearch does not use this meta data at all
➤ Can be used to store application-specific meta data
➤ This can be anything you want
THANK YOU FOR
WATCHING!

More Related Content

What's hot

Indexing techniques
Indexing techniquesIndexing techniques
Indexing techniques
Huda Alameen
 
uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2
Abhishek Kesharwani
 
Data indexing presentation
Data indexing presentationData indexing presentation
Data indexing presentation
gmbmanikandan
 
XML and Databases
XML and DatabasesXML and Databases
XML and Databases
Cittrex
 
03 namespace
03 namespace03 namespace
03 namespace
Baskarkncet
 
Pattern matching & file input and output
Pattern matching & file input and outputPattern matching & file input and output
Pattern matching & file input and output
Mehul Jariwala
 
Data exchange over internet (XML vs JSON)
Data exchange over internet (XML vs JSON)Data exchange over internet (XML vs JSON)
Data exchange over internet (XML vs JSON)
Wajahat Shahid
 
uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2
Abhishek Kesharwani
 
Introduction to XML
Introduction to XMLIntroduction to XML
Introduction to XML
Abhra Basak
 
11 wordprocessing ml subject - glossary document
11   wordprocessing ml subject - glossary document11   wordprocessing ml subject - glossary document
11 wordprocessing ml subject - glossary document
Shawn Villaron
 
Zhishi.me - Weaving Chinese Linking Open Data
Zhishi.me - Weaving Chinese Linking Open DataZhishi.me - Weaving Chinese Linking Open Data
Zhishi.me - Weaving Chinese Linking Open Data
Xing Niu
 
Sql introduction
Sql introductionSql introduction
Sql introduction
Bhavya Chawla
 
Introduction to XML
Introduction to XMLIntroduction to XML
Introduction to XML
Kumar
 
Introduction to XML and Databases
Introduction to XML and DatabasesIntroduction to XML and Databases
Introduction to XML and Databases
torp42
 
File organization 1
File organization 1File organization 1
File organization 1
Rupali Rana
 
Dutch Government Business Case
Dutch Government Business CaseDutch Government Business Case
Dutch Government Business Case
Hans Overbeek
 
DBMS-Quick Reference
DBMS-Quick ReferenceDBMS-Quick Reference
DBMS-Quick Reference
Shrija Madhu
 

What's hot (20)

Indexing
IndexingIndexing
Indexing
 
Indexing techniques
Indexing techniquesIndexing techniques
Indexing techniques
 
uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2
 
Data indexing presentation
Data indexing presentationData indexing presentation
Data indexing presentation
 
XML and Databases
XML and DatabasesXML and Databases
XML and Databases
 
03 namespace
03 namespace03 namespace
03 namespace
 
Pattern matching & file input and output
Pattern matching & file input and outputPattern matching & file input and output
Pattern matching & file input and output
 
Data exchange over internet (XML vs JSON)
Data exchange over internet (XML vs JSON)Data exchange over internet (XML vs JSON)
Data exchange over internet (XML vs JSON)
 
uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2uptu web technology unit 2 Xml2
uptu web technology unit 2 Xml2
 
Introduction to XML
Introduction to XMLIntroduction to XML
Introduction to XML
 
11 wordprocessing ml subject - glossary document
11   wordprocessing ml subject - glossary document11   wordprocessing ml subject - glossary document
11 wordprocessing ml subject - glossary document
 
Zhishi.me - Weaving Chinese Linking Open Data
Zhishi.me - Weaving Chinese Linking Open DataZhishi.me - Weaving Chinese Linking Open Data
Zhishi.me - Weaving Chinese Linking Open Data
 
Sql introduction
Sql introductionSql introduction
Sql introduction
 
Xml
XmlXml
Xml
 
Introduction to XML
Introduction to XMLIntroduction to XML
Introduction to XML
 
Introduction to XML and Databases
Introduction to XML and DatabasesIntroduction to XML and Databases
Introduction to XML and Databases
 
File organization 1
File organization 1File organization 1
File organization 1
 
Dutch Government Business Case
Dutch Government Business CaseDutch Government Business Case
Dutch Government Business Case
 
DATA BASE MODEL Rohini
DATA BASE MODEL RohiniDATA BASE MODEL Rohini
DATA BASE MODEL Rohini
 
DBMS-Quick Reference
DBMS-Quick ReferenceDBMS-Quick Reference
DBMS-Quick Reference
 

Recently uploaded

Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Enhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZEnhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZ
Globus
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
Jen Stirrup
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptxSecstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
nkrafacyberclub
 
Quantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIsQuantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIs
Vlad Stirbu
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 

Recently uploaded (20)

Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Enhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZEnhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZ
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptxSecstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
 
Quantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIsQuantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIs
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 

Elasticsearch Meta Fields

  • 1. META FIELDSby Bo Andersen - codingexplained.com
  • 2. OUTLINE ➤ Introduction to meta fields ➤ Meta fields categories ➤ Identity ➤ Document source ➤ Indexing ➤ Routing ➤ Other
  • 3. INTRODUCTION TO META FIELDS ➤ Each document has metadata associated with it ➤ E.g. the _index, _type, and _id meta fields. ➤ The behavior of some of these meta-fields can be customized ➤ This is done when creating a mapping type
  • 5. IDENTITY META FIELDS (PART 1) ➤ _index ➤ Allows matching documents based on their indexes ➤ _index is exposed as a virtual field and is not added to the Lucene index as a real field ➤ _type ➤ The type of the document ➤ Indexed in order to make searching by type name fast
  • 6. IDENTITY META FIELDS (PART 2) ➤ _id ➤ The ID of the document ➤ Not indexed, because its value can be automatically derived from the _uid field ➤ _uid ➤ The values of _type and _id are combined as {type}#{id} and indexed
  • 8. DOCUMENT SOURCE META FIELDS ➤ _source ➤ Contains the JSON that was passed at index time (unless updated) ➤ The field is not indexed and is therefore not searchable ➤ It is stored so that it can be returned when executing fetch requests ➤ Can be disabled to save storage space ➤ _size ➤ Indexes the size of the _source field in bytes ➤ Available as a plugin that must be installed before the field is available ➤ sudo bin/plugin install mapper-size
  • 10. _ALL ➤ A special field that concatenates the values of all of the other fields into one string ➤ Uses space as a delimiter ➤ The field is analyzed and indexed, but not stored ➤ This means that it can be searched, but not retrieved ➤ The field allows you to search for values in documents without knowing which field contains the value
  • 11. _FIELD_NAMES ➤ Indexes the names of every field in a document that contains a value other than NULL ➤ Used by the exists and missing queries ➤ To find documents that either have or do not have a non-null value for a particular field
  • 13. _PARENT ➤ Can be used to establish a parent-child relationship between documents in an index ➤ This is done by making one mapping type the parent of another "mappings": { "employee": { "_parent": { "type": "person" } } } When adding an index:
  • 14. _ROUTING ➤ Used to route a document to a particular shard within an index ➤ Enables you to define rules that control in which shards documents are stored ➤ For advanced use cases!
  • 16. _META ➤ Each mapping type can have custom meta data associated with it ➤ Elasticsearch does not use this meta data at all ➤ Can be used to store application-specific meta data ➤ This can be anything you want