Download as PDF, PPTX
![Indexing Data
● Iterate over all creatives
● Call our GraphQL server
with a query
● Put the result into
Elasticsearch
query getCreativesById($creativeIds: [UUID]) {
creative(creativeIds: $creativeIds) {
id
...AllCreativeFields
show {
...AllShowFields
titleRanking {
...AllTitleRankingFields
}
}
comments {
...AllCommentFields
}
}
}](https://image.slidesharecdn.com/graphqlsummitpresentation-191101011730/85/GraphQL-Search-9-320.jpg)
GraphQL Search
- 1.
- 2.
- 3. How can wequickly search and aggregate data spread across multiple sources?
- 4. Roadmap Indexing the Data Keeping It Updated DataFlow Learnings
- 5. Marketing Tech atNetflix Creatives
- 6. Our Data query getAllCreatives(){ creative { id ...AllCreativeFields show { ...AllShowFields titleRanking { ...AllTitleRankingFields } } comments { ...AllCommentFields } } } Creative Service Show Service Title Ranking Algo Comment Service
- 7.
- 8. Roadmap Indexing the Data Keeping It Updated DataFlow Learnings
- 9. Indexing Data ● Iterateover all creatives ● Call our GraphQL server with a query ● Put the result into Elasticsearch query getCreativesById($creativeIds: [UUID]) { creative(creativeIds: $creativeIds) { id ...AllCreativeFields show { ...AllShowFields titleRanking { ...AllTitleRankingFields } } comments { ...AllCommentFields } } }
- 10. After Our Datais Indexed ● Sort, group, and filter the data on arbitrary fields ● Provide type-ahead suggestions ● Implement faceted search ● Display results with infinite scroll ● Our page loads much faster
- 11. Roadmap Indexing the Data Keeping It Updated DataFlow Learnings
- 12. Keeping the IndexUp To Date ● If the creative changes, then reindex that creative ● if the show changes, then reindex the creative with that show id ● If the title ranking changes, then find the related show, use the show to find the creative, and reindex it ● if a comment changes, then reindex the creative with that comment id If our data changes, we need to reindex
- 13. GraphQL Defines Relationshipsin Our Data Title Ranking Show Creative Comments
- 14. Auto Search Indexer TitleRanking Show Creative Comments 1. Change event
- 15. Auto Search Indexer 1.Change event 2. Fan out one level Title Ranking Show Creative Comments
- 16. Auto Search Indexer 1.Change event 2. Fan out one level 3. Search all indexed documents Title Ranking Show Creative Comments Elasticsearch
- 17. Auto Search Indexer 1.Change event 2. Fan out one level 3. Search all indexed documents 4. Rebuild and reindex documents Title Ranking Show Creative Comments
- 18. Auto Search Indexer 1.Change event 2. Fan out one level 3. Search all indexed documents 4. Rebuild and reindex documents rebuild search fan out change
- 19. Roadmap Indexing the Data Keeping It Updated DataFlow Learnings
- 20. Data Flow: BeforeIndexing BrowserGraphQL aggregation on page load (read) Creative Service Show Service Title Ranking Algo Commenting Service
- 21. Data Flow: AfterIndexing BrowserGraphQL Creative Service Show Service Title Ranking Algo Commenting Service Elasticsearch aggregation on data change (write)
- 22. Data Flow: AbstractElasticsearch BrowserGraphQL Creative Service Show Service Title Ranking Algo Commenting Service Elasticsearch GraphQL can choose its data source
- 23. Search becomes aplugin that you can enable on a GraphQL server with minimal configuration
- 24. Roadmap Indexing the Data Keeping It Updated DataFlow Learnings
- 25. Initial Setup ofthe Search Indexer Search Indexer GraphQL Kafka Elasticsearch 1. Use GraphQL - graph edges must be bidirectional 2. Listen to change events 3. Index to a search database
- 26. 1. Define aGraphQL query 2. Done How to Build a New Index
- 27. Search Indexer Performance Fanout Search Rebuild+ +Change + < 500ms =
- 28. More Information onMonday medium.com/netflix-techblog
- 29. GraphQL Search Indexing ●Robust search and instant page loads ● Minimal refactoring on the client side ● Configuration driven and automated +
- 30.