This document discusses using Elasticsearch as an external data provider (EDP) to access catalog and product data from an e-commerce platform in DX. Elasticsearch allows indexing the data in a way that supports desired search capabilities. Challenges include denormalizing data and handling a large number of flat product records. The implementation indexes the catalog and products from the e-commerce system into Elasticsearch. It provides a catalog browsing experience and product search, and enriches product data using services like IBM Watson. GraphQL endpoints allow direct querying of Elasticsearch to power commerce experiences.

1. © 2002-2018 Jahia© 2002-2018 Jahia
Elasticsearch
powered EDP
By Cedric Mailleux

2. © 2002-2018 Jahia© 2002-2018 Jahia
1.
Introduction

3. © 2002-2018 Jahia© 2002-2018 Jahia
External Data Provider
EDPs are a way of accessing content from outside DX as local content
With the External Data Provider, external data can be accessed,
manipulated and searched as if they were directly stored in the DX
repository

4. © 2002-2018 Jahia© 2002-2018 Jahia
Commerce Platform
Many datasets (Products/Catalogs/Customers/Orders/…)
Sometimes many entry points of data need to be accessed to make sense of
it
For examples :
● Get the catalog structure
● Get the products

5. © 2002-2018 Jahia© 2002-2018 Jahia
Elasticsearch
Elasticsearch is a distributed, RESTful search and analytics engine capable of
solving a growing number of use cases. As the heart of the Elastic Stack, it
centrally stores your data so you can discover the expected and uncover the
unexpected.

6. © 2002-2018 Jahia© 2002-2018 Jahia
2.
Elasticsearch/EDP

7. © 2002-2018 Jahia© 2002-2018 Jahia
Why use Elasticsearch
Most data of the catalog/product are static, it did not make sense to query
them everytime, a cache will help but won’t give us extra capabilities
Whereas Elasticsearch allows to index data the way we want to find them.
It also offers the possibility to query multiple indices at the same time

8. © 2002-2018 Jahia© 2002-2018 Jahia
Challenges
Elasticsearch indices must be as flat as possible so you need to denormalize
your objects.
In our case for each products category we expands the whole categories
information instead of having only the identifier
{ "categoryType": "categories",
"shortIdUrl": "/categories/330000/330100",
"code": "330100",
"shortUrl": "/Categories/Snowwear-youth/Snow-Jackets-youth",
"name": "Snow Jackets youth",
"categoryIdType": "categories"
}

9. © 2002-2018 Jahia© 2002-2018 Jahia
Another challenge was that JCR is a tree structure, not meant to handle
100K or more of flat products
Chosen solution is to generate a system path for each product based on hash
of their sku
"mountedPath": "/p/cd/jb/ie/95366_honeydew",
"vanityUrl": "/product/en/95366_honeydew.html"
And also associate a vanity URL for SEO purposes
Challenges

10. © 2002-2018 Jahia© 2002-2018 Jahia
Catalogue browsing
● Only the catalog is browsable,
mounted in the site

11. © 2002-2018 Jahia© 2002-2018 Jahia
Product browsing
● Products are only searchable
○ New product picker

12. © 2002-2018 Jahia© 2002-2018 Jahia
Enrich data
Getting the data out of the ecommerce platform for the products and
catalog allows also to enrich them, like tagging the photo automatically with
IBM Watson, this can be done in a 3rd index to avoid going back to IBM
Watson unless the product information changed

13. © 2002-2018 Jahia© 2002-2018 Jahia
3.
Implementation

14. © 2002-2018 Jahia© 2002-2018 Jahia
EDP
Mapping data from indices to
JCR nodes
Catalog Tree browsing
Products/Medias nodes
Uses Elasticsearch connector
module for access to
Elasticsearch cluster

15. © 2002-2018 Jahia© 2002-2018 Jahia
Indexation process
First indexation happens upon saving the settings
DX request the catalog from the ecommerce system
Products are loaded in parallel with paginated queries
30s for 2K SKUs
After first indexation, a background job is checking every 5 minutes if there
is changes if yes we reindex in new indices and switch the active indices
when indexing is over

16. © 2002-2018 Jahia© 2002-2018 Jahia
EDP - Vanity/I18N
DX mixin for vanity URLS
Lazy properties for I18N
Ensure products have human
readable URLs in different
languages

17. © 2002-2018 Jahia© 2002-2018 Jahia
EDP - Federated search
Send events to DX to index the
products as contents
Allowing to search products and
content using the full text
search API and the DX
Elasticsearch provider

18. © 2002-2018 Jahia© 2002-2018 Jahia
Experience components
Provide macros to include
products everywhere as
references
Exposing GraphQL endpoints to
have dedicated Commerce API
bypassing JCR and requesting
Elasticsearch directly
Find products, facets
Get up to date info for price and
stock by requesting Ecommerce
backend if needed

19. © 2002-2018 Jahia© 2002-2018 Jahia
GraphQL example
query products {
cioProducts(connection:"commio01",index:"apparelukfull_alias_en",ca
tegory:"/brands",limit:100,offset:0){
sku
name
vanityUrl
mountedPath
}
}
query facets {
cioShopByCategory(connection:"commio01",index:"apparelukfull_alias_
en",category:"/categories",type:"categories") {
docCount
key
category {
name
children {
name
}
}
}
}

20. © 2002-2018 Jahia© 2002-2018 Jahia
Questions
Answers