Successfully reported this slideshow.
Your SlideShare is downloading. ×

Data Engineer's Lunch #66: Airflow and Presto

Data Engineer's Lunch #66: Airflow and Presto

Download to read offline

In Data Engineer's Lunch #66, Arpan Patel will discuss how to connect Airflow and Presto

Accompanying Blog: Coming Soon!

Accompanying YouTube: https://youtu.be/HcxBg6MkBv4

Sign Up For Our Newsletter: http://eepurl.com/grdMkn

Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers-DC/events/

Cassandra.Link:
https://cassandra.link/

Follow Us and Reach Us At:

Anant:
https://www.anant.us/

Awesome Cassandra:
https://github.com/Anant/awesome-cassandra

Email:
solutions@anant.us

LinkedIn:
https://www.linkedin.com/company/anant/

Twitter:
https://twitter.com/anantcorp

Eventbrite:
https://www.eventbrite.com/o/anant-1072927283

Facebook:
https://www.facebook.com/AnantCorp/

Join The Anant Team:
https://www.careers.anant.us

In Data Engineer's Lunch #66, Arpan Patel will discuss how to connect Airflow and Presto

Accompanying Blog: Coming Soon!

Accompanying YouTube: https://youtu.be/HcxBg6MkBv4

Sign Up For Our Newsletter: http://eepurl.com/grdMkn

Join Data Engineer’s Lunch Weekly at 12 PM EST Every Monday:
https://www.meetup.com/Data-Wranglers-DC/events/

Cassandra.Link:
https://cassandra.link/

Follow Us and Reach Us At:

Anant:
https://www.anant.us/

Awesome Cassandra:
https://github.com/Anant/awesome-cassandra

Email:
solutions@anant.us

LinkedIn:
https://www.linkedin.com/company/anant/

Twitter:
https://twitter.com/anantcorp

Eventbrite:
https://www.eventbrite.com/o/anant-1072927283

Facebook:
https://www.facebook.com/AnantCorp/

Join The Anant Team:
https://www.careers.anant.us

Advertisement
Advertisement

More Related Content

More from Anant Corporation

Advertisement

Data Engineer's Lunch #66: Airflow and Presto

  1. 1. Version 1.0 Airflow and Presto In Data Engineer's Lunch #66, Arpan Patel will discuss how to connect Airflow and Presto Arpan Patel Engineer @ Anant
  2. 2. Introduction ● Open Source Distributed SQL Query Engine for Big Data ● Originally created at Facebook to solve for slow queries on a 300 PB Hive Data Warehouse. This original version of Presto is called PrestoDB. A few of the founders of PrestoDB left Facebook in 2018 and created PrestoSQL (now Trino) ● Designed and written from the ground up for interactive analytics and approaches the speed of commercial data warehouses while scaling to the size of organizations like Facebook. ● A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization. ○ Allows querying data where it lives ○ Comcast: Apache Cassandra, Microsoft SQL Server, MongoDB, Oracle, Teradata ○ Airbnb and Dropbox
  3. 3. Presto CLI
  4. 4. Presto UI
  5. 5. Demo ● Spin up Docker Containers on Gitpod ● Connect Presto and Airflow ● Take a look at the Presto UI and CLI ● Run Airflow DAG that runs a Presto Query
  6. 6. Interesting Read / Watch ● Presto-on-Spark Runs Presto code as a library within Spark executor. Design Docs ● https://prestodb.io/blog/2021/10/26/Scaling-with- Presto-on-Spark ● https://databricks.com/session_na20/presto-on- apache-spark-a-tale-of-two-computation-engines
  7. 7. Strategy: Scalable Fast Data Architecture: Cassandra, Spark, Kafka Engineering: Node, Python, JVM,CLR Operations: Cloud, Container Rescue: Downtime!! I need help. www.anant.us | solutions@anant.us | (855) 262-6826 3 Washington Circle, NW | Suite 301 | Washington, DC 20037

×