Downloaded 11 times
Quix presto ide, presto summit IL
- 1. Valery Frolov, Sr.Data Engineer valeryf@wix.com github.com/frolovv Quix - Presto IDE
- 2. Wix Data at Wix Prestoat Wix Quix IDE AGENDA
- 3. Facts about Wix >150M Wix Freeusers >4MPremiums >2600Employee s >1000Micro- services >4KExperiments >12KTypes of BI-events > 50%Employee s used Quix >3BDaily BI- events
- 4.
- 5. Data Engineering ▪ Infrastructure ○Hadoop, Storm, Presto, Spark ▪ Get data from and back to production ○ Logs collection, DB replications ○ Analytics, Personalization, ML ▪ Self-service tools ○ User Explorer, Quix, Job Scheduler, Events Catalog, Feature Store, Profile Catalog, AB-test analytics, Reporting tools Wix
- 11. Wix Data at Wix Prestoat Wix Quix IDE AGENDA
- 12.
- 13.
- 14.
- 15. Wix Data at Wix Prestoat Wix Quix IDE AGENDA
- 16. Presto at Wix >400K Prestoat Wix Weekly queries >34KTables >11Catalogs 10Clients
- 17. Presto at Wix ▪Mix of on-demand and a fleet of spot instances in AWS ▪ Complex ecosystem of 9 in-house plugins ○ Hbase + parquet ○ Native sql syntax ○ Secure execution ○ GDrive Presto at Wix
- 18. Hbase + Parquetinterleaving Presto at Wix -- Query 1 select count(1) from users where date_created between date ‘2019-04-14’ and current_date -- Query 2 select count(1) from users where date_created between date ‘2019-04-14’ and localtimestamp
- 19. Native sql syntax(BQ & TSQL) Presto at Wix
- 20.
- 21.
- 22. Presto at Wix- Clients ▪ Dozen of internal clients sending ~100K queries daily ○ Quix ○ Reporting tools (tableau, power-bi) ○ Scheduled jobs (oozie, airflow, cron) ○ Production integrations (jdbc, rpc, results on kafka) Presto at Wix
- 23. Wix Data at Wix Prestoat Wix Quix IDE AGENDA
- 25. Facts about Quix >1300 Quix Employee s usedQuix >105KNotes >20KNotebook s 3KBy most active user >8KDaily queries >400Weekly users >34KTables > 2KScheduled jobs
- 32. ▪ Notes /Notebooks / Folders ▪ Different note types ▪ Smart editor ▪ Dataset explorer ▪ Job Scheduler ▪ Public notes Quix : recap Quix IDE
- 33. ▪ github.com/wix-incubator/quix ▪ Firstphase : ○ Presto notes, notebooks, folders, dataset explorer, smart editor ▪ Second phase ○ Big query presto plugin with native execution ○ Native notes in quix Quix is going open source Quix IDE
- 34.