Data Lakes on Public Cloud: Breaking Data Management Monoliths
•
0 likes•289 views
Sharon Dashet (Sr. Data Analytics Solution Lead) @ Google Cloud: The worlds of traditional RDBMS and Data Lake Hadoop systems are converging and moving to public cloud and SaaS offerings. In this session, Sharon will share her personal journey as a data professional since the 90s weaved into the history of data management systems. The session will also cover the differences between on-premise and cloud Data Lakes.
Report
Share
Report
Share
Download to read offline
Recommended
Why do the majority of Data Science projects never make it to production?
María de la Fuente (Solutions Architect Manager for IMEA) @ Databricks
While most companies understand the value creation of leveraging data and are taking on board an AI strategy, only 13% of the data science projects make it to production successfully.
Besides the well-known skills gap in the market, we need to level up our end-to-end approach and cover all aspects involved when working with AI.
In this session, we will discuss the main obstacles to overcome and how we can avoid the major pitfalls to ensure our data science journey becomes successful.
Mohamed Sabri: Operationalize machine learning with Kubeflow
This document summarizes a hands-on workshop on Kubeflow Pipeline. The workshop will cover requirements, an introduction to the presenter Mohamed Sabri, and their approach of strategizing, shaping, and spreading knowledge. It then discusses operationalizing machine learning (MLOps) and provides an analysis, design, coaching, and implementation framework. Deliverables include an implemented MLOps environment, training sessions, design documents, and a recommendations roadmap. The rest of the document discusses MLOps architectures, challenges, example technologies and tools, a use case, and deployment workflows from notebooks to production.
Azure databricks by usama whaba khan
Usama Wahab Khan is a data scientist, developer, and CTO at Evolution Technologies. He is responsible for more data types and technologies in his role as a data engineer. Microsoft Azure provides a wide set of tools that help with data storage, processing, and analytics including Azure Data Lake Store, Azure Storage, Azure HDInsight, Azure Data Lake Analytics, and Azure Databricks.
Future of Data Platform in Cloud Native world
Presentation of my talk given in Phoenix Data Conference 2019. In this we will look at challenges with current Apache Hadoop ecosystem
Apache Hadoop is still relevant but way of doing Hadoop and enterprise data architecture has to be re-looked as we enter Cognitive and Cloud Native Era
We need
Architecture that is enabled by common run time layer across on premise and cloud
Architecture that can abstract away dependency and version conflicts with tons of open source machine learning out there. Yarn did not scale up in that aspect until one want to deal with multiple conda environment
Architecture that can enable real Hybrid Cloud and Multi Cloud portability
And many more challenges that one has to overcome to keep architecture simple, infrastructure agile and better utilized
Introducing MLflow for End-to-End Machine Learning on Databricks
Solving a data science problem is about more than making a model. It entails data cleaning, exploration, modeling and tuning, production deployment, and workflows governing each of these steps. In this simple example, we’ll take a look at how health data can be used to predict life expectancy. It will start with data engineering in Apache Spark, data exploration, model tuning and autologging with hyperopt and MLflow. It will continue with examples of how the model registry governs model promotion, and simple deployment to production with MLflow as a job or REST endpoint. This tutorial will cover the latest innovations from MLflow 1.12.
2016 Spark Summit East Keynote: Ali Ghodsi and Databricks Community Edition demo
This document discusses Databricks' goal of democratizing access to Spark. It introduces the Databricks cloud platform, which provides a hosted model for Spark with rapid releases, dynamic scaling, and security controls. The platform is used for just-in-time data warehousing, advanced analytics, and real-time use cases. Many companies struggle with the steep learning curve and costs of big data projects. To empower more developers, Databricks trained thousands on Spark and launched online courses with over 100,000 students. They are announcing the Databricks Community Edition, a free version of their platform, to further democratize access to Spark through mini clusters, notebooks, APIs, and continuous delivery of learning content.
Intro to Delta Lake
Delta Lake brings reliability, performance, and security to data lakes. It provides ACID transactions, schema enforcement, and unified handling of batch and streaming data to make data lakes more reliable. Delta Lake also features lightning fast query performance through its optimized Delta Engine. It enables security and compliance at scale through access controls and versioning of data. Delta Lake further offers an open approach and avoids vendor lock-in by using open formats like Parquet that can integrate with various ecosystems.
MLOps with Kubeflow
This document discusses MLOps and Kubeflow. It begins with an introduction to the speaker and defines MLOps as addressing the challenges of independently autoscaling machine learning pipeline stages, choosing different tools for each stage, and seamlessly deploying models across environments. It then introduces Kubeflow as an open source project that uses Kubernetes to minimize MLOps efforts by enabling composability, scalability, and portability of machine learning workloads. The document outlines key MLOps capabilities in Kubeflow like Jupyter notebooks, hyperparameter tuning with Katib, and model serving with KFServing and Seldon Core. It describes the typical machine learning process and how Kubeflow supports experimental and production phases.
Recommended
Why do the majority of Data Science projects never make it to production?
María de la Fuente (Solutions Architect Manager for IMEA) @ Databricks
While most companies understand the value creation of leveraging data and are taking on board an AI strategy, only 13% of the data science projects make it to production successfully.
Besides the well-known skills gap in the market, we need to level up our end-to-end approach and cover all aspects involved when working with AI.
In this session, we will discuss the main obstacles to overcome and how we can avoid the major pitfalls to ensure our data science journey becomes successful.
Mohamed Sabri: Operationalize machine learning with Kubeflow
This document summarizes a hands-on workshop on Kubeflow Pipeline. The workshop will cover requirements, an introduction to the presenter Mohamed Sabri, and their approach of strategizing, shaping, and spreading knowledge. It then discusses operationalizing machine learning (MLOps) and provides an analysis, design, coaching, and implementation framework. Deliverables include an implemented MLOps environment, training sessions, design documents, and a recommendations roadmap. The rest of the document discusses MLOps architectures, challenges, example technologies and tools, a use case, and deployment workflows from notebooks to production.
Azure databricks by usama whaba khan
Usama Wahab Khan is a data scientist, developer, and CTO at Evolution Technologies. He is responsible for more data types and technologies in his role as a data engineer. Microsoft Azure provides a wide set of tools that help with data storage, processing, and analytics including Azure Data Lake Store, Azure Storage, Azure HDInsight, Azure Data Lake Analytics, and Azure Databricks.
Future of Data Platform in Cloud Native world
Presentation of my talk given in Phoenix Data Conference 2019. In this we will look at challenges with current Apache Hadoop ecosystem
Apache Hadoop is still relevant but way of doing Hadoop and enterprise data architecture has to be re-looked as we enter Cognitive and Cloud Native Era
We need
Architecture that is enabled by common run time layer across on premise and cloud
Architecture that can abstract away dependency and version conflicts with tons of open source machine learning out there. Yarn did not scale up in that aspect until one want to deal with multiple conda environment
Architecture that can enable real Hybrid Cloud and Multi Cloud portability
And many more challenges that one has to overcome to keep architecture simple, infrastructure agile and better utilized
Introducing MLflow for End-to-End Machine Learning on Databricks
Solving a data science problem is about more than making a model. It entails data cleaning, exploration, modeling and tuning, production deployment, and workflows governing each of these steps. In this simple example, we’ll take a look at how health data can be used to predict life expectancy. It will start with data engineering in Apache Spark, data exploration, model tuning and autologging with hyperopt and MLflow. It will continue with examples of how the model registry governs model promotion, and simple deployment to production with MLflow as a job or REST endpoint. This tutorial will cover the latest innovations from MLflow 1.12.
2016 Spark Summit East Keynote: Ali Ghodsi and Databricks Community Edition demo
This document discusses Databricks' goal of democratizing access to Spark. It introduces the Databricks cloud platform, which provides a hosted model for Spark with rapid releases, dynamic scaling, and security controls. The platform is used for just-in-time data warehousing, advanced analytics, and real-time use cases. Many companies struggle with the steep learning curve and costs of big data projects. To empower more developers, Databricks trained thousands on Spark and launched online courses with over 100,000 students. They are announcing the Databricks Community Edition, a free version of their platform, to further democratize access to Spark through mini clusters, notebooks, APIs, and continuous delivery of learning content.
Intro to Delta Lake
Delta Lake brings reliability, performance, and security to data lakes. It provides ACID transactions, schema enforcement, and unified handling of batch and streaming data to make data lakes more reliable. Delta Lake also features lightning fast query performance through its optimized Delta Engine. It enables security and compliance at scale through access controls and versioning of data. Delta Lake further offers an open approach and avoids vendor lock-in by using open formats like Parquet that can integrate with various ecosystems.
MLOps with Kubeflow
This document discusses MLOps and Kubeflow. It begins with an introduction to the speaker and defines MLOps as addressing the challenges of independently autoscaling machine learning pipeline stages, choosing different tools for each stage, and seamlessly deploying models across environments. It then introduces Kubeflow as an open source project that uses Kubernetes to minimize MLOps efforts by enabling composability, scalability, and portability of machine learning workloads. The document outlines key MLOps capabilities in Kubeflow like Jupyter notebooks, hyperparameter tuning with Katib, and model serving with KFServing and Seldon Core. It describes the typical machine learning process and how Kubeflow supports experimental and production phases.
platform for Machine Learning
The document discusses several machine learning platforms for 2020 including Altair Knowledge Studio, Anaconda Enterprise, Databricks Unified Analytics Platform, Dataiku Data Science Studio, DataRobot, Domino Data Science Platform, H2O, KNIME Analytics Platform, MATLAB, and TIBCO Data Science. Each platform is briefly described and a link to a YouTube video about the platform is provided.
ML-Ops: From Proof-of-Concept to Production Application
Successfully deploying a working machine learning prototype to a production application is a challenging task, frought with difficulties not experienced in traditional software deployments.
In this talk, you will learn techniques to successfully deploy ML applications in a scalable, maintainable, and automated way.
What’s New with Databricks Machine Learning
In this session, the Databricks product team provides a deeper dive into the machine learning announcements. Join us for a detailed demo that gives you insights into the latest innovations that simplify the ML lifecycle — from preparing data, discovering features, and training and managing models in production.
Managing R&D Data on Parallel Compute Infrastructure
Clinical genomic analytics pipelines using Databricks and the Delta Lake for the benefit of loading individual reads from raw sequencing or base-call files have significant advantages over more traditional methods. Analysis pipelines that perform genomic mapping to purpose-built reference data artifacts persisted to tables allows for enhanced performance that is magnitudes greater than previous mapping methods. These scalable, reproducible, and potentially open sourced methods have the ability to transform bioinformatics and R&D data management / governance.
Deep Learning for Recommender Systems with Nick pentreath
In the last few years, deep learning has achieved significant success in a wide range of domains, including computer vision, artificial intelligence, speech, NLP, and reinforcement learning. However, deep learning in recommender systems has, until recently, received relatively little attention. This talks explores recent advances in this area in both research and practice. I will explain how deep learning can be applied to recommendation settings, architectures for handling contextual data, side information, and time-based models, and compare deep learning approaches to other cutting-edge contextual recommendation models, and finally explore scalability issues and model serving challenges.
Real World End to End machine Learning Pipeline
Purpose of this presentation is to highlight how end to end machine learning looks like in real world enterprise. This is to provide insight to aspiring data scientist who have been through courses or education in ML that mostly focus on ML algorithms and not end to end pipeline.
Architecture and components mentioned in Slide 11 will be discussed in detailed in series of post on LinkedIn over the course of next few month
To get updates on this follow me on LinkedIn or search/follow hashtag #end2endDS. Post will be active in August 2019 and will be posted till September 2019
Accelerating Innovation with Unified Analytics with Ali Ghodsi
Today at the 10th Spark Summit, Databricks CEO & Co-founder revealed Databricks Serverless, a new initiative to offer serverless computing for complex data science and Apache Spark workloads. Databricks Serverless is the first product to offer a serverless API for Apache Spark, greatly simplifying and unifying data science and big data workloads for both end-users and DevOps.
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligence, "Making Big Data easier on Azure"
AI meets Big Data
This document discusses artificial intelligence and machine learning. It provides a brief history of AI from the Perceptron model in 1958 to modern deep learning approaches. It then discusses several applications of machine learning like image classification, medical diagnosis, and autonomous vehicles. It also discusses challenges like distributed machine learning and hidden technical debt. Finally, it provides examples of how AI can be applied to commerce and automotive use cases.
Building predictive models in Azure Machine Learning
This presentation covers how to build and drive insights from data by building machine learning models. The session covers how to develop and train models in Python/R using Azure Machine Learning. The session covers how to explore key concepts in data acquisition, preparation, exploration, and visualization, and take a look at how to build a predictive solution using Azure Machine Learning, R, and Python. The session covers tips and tricks on selecting the right algorithm for your data science problem and how to utilize Machine Learning to solve it.
Summary introduction to data engineering
Data engineering involves designing, building, and maintaining data warehouses to transform raw data into queryable forms that enable analytics. A core task of data engineers is Extract, Transform, and Load (ETL) processes - extracting data from sources, transforming it through processes like filtering and aggregation, and loading it into destinations. Data engineers help divide systems into transactional (OLTP) and analytical (OLAP) databases, with OLTP providing source data to data warehouses analyzed through OLAP systems. While similar, data engineers focus more on infrastructure and ETL processes, while data scientists focus more on analysis, modeling, and insights.
Big Data- Automotive Industry Use Case
Big Data has emerged as a powerful new technology paradigm. To manage the massive data generated by social media, online transactions, Weblogs, or sensors, Big Data incorporates innovative technologies in data management (unstructured, semi-structured and structured), processing, real-time analytics, and visualization. It is also useful for reporting in circumstances where a relational database approach is not effective or too costly. This Big Data project is to be primarily exposed to utilize the tools. Tool usage, programming, algorithms and application development are covered in relevant courses.
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Zalando transitioned from a centralized data platform to a data mesh architecture. This decentralized their data infrastructure by having individual domains own datasets and pipelines rather than a central team. It provided self-service data infrastructure tools and governance to enable domains to operate independently while maintaining global interoperability. This improved data quality by making domains responsible for their data and empowering them through the data mesh approach.
Hadoop for Humans: Introducing SnapReduce 2.0
In this webinar, we talk about Hadoop, big data and SnapReduce 2.0 with SnapLogic Chief Scientist Greg Benson, Professor of Computer Science at the University of San Francisco. This webinar features a dive into SnapReduce, and a discussion about how SnapLogic delivers big data acquisition, better big data preparation and universal big data delivery.
To learn more, visit: http://www.snaplogic.com/snapreduce
Data & AI Platform Concepts
This is the pilot episode of Data & AI Platform Concepts, where I intend to cover all the aspects of Data & AI Platform like Data Systems, Analytics/ML/AI, Cloud Services, DevOps, Governance.
https://www.ankitrathi.com
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...Amazon Web Services Korea
This document discusses the democratization of data science and machine learning using automated machine learning tools. It provides examples of how DataRobot has helped customers in various industries build predictive models faster and with less coding than traditional approaches. Specifically, it summarizes how DataRobot has helped customers in banking, insurance, retail, and other industries with use cases like predictive maintenance, sales forecasting, fraud detection, customer churn prediction, and insurance underwriting.Snaplogic Live: Big Data in Motion
Watch this recorded demonstration of SnapLogic from our team of experts who answer your hybrid cloud and big data integration questions.
demo, ipaas, elastic integration, cloud data, app integration, data integration, hybrid could integration, big data, big data integration
How Cloud is Affecting Data Scientists
This document discusses how data science models have transitioned to the cloud to take advantage of greater computing resources. It notes that data science models are resource-intensive and traditionally required powerful local machines. The cloud allows data scientists to run models on cloud infrastructure for lower costs than high-end laptops and with access to many GPUs. Several major cloud platforms - Azure, AWS, and Google Cloud - are discussed and compared in terms of their machine learning offerings. The document also introduces Microsoft's Team Data Science Process, which aims to help data science teams collaborate more effectively on projects in the cloud.
Platform for Data Scientists
This document discusses a platform for data scientists that aims to automate routine jobs, maximize resource utilization, and allow data scientists to focus more on business solutions. The platform provides capabilities for data capture, analysis, modeling, and output of analytics. It seeks to reduce the time taken to turn data into insights from months to weeks. Key elements of the platform include tools for exploratory data analysis, advanced modeling, distributed architecture, bespoke algorithms, and packaged analytics solutions.
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
This document discusses the evolution from traditional machine learning to learning machines. It outlines the machine learning process and highlights how learning machines enable continuous feedback and retraining through automated modeling. The key design principles of learning machines are presented as collaboration across roles, convergence of technologies, and simplicity through automation and intuitiveness. Examples are provided of how learning machines can power experiences and services.
Building a modern data warehouse
Embarking on building a modern data warehouse in the cloud can be an overwhelming experience due to the sheer number of products that can be used, especially when the use cases for many products overlap others. In this talk I will cover the use cases of many of the Microsoft products that you can use when building a modern data warehouse, broken down into four areas: ingest, store, prep, and model & serve. It’s a complicated story that I will try to simplify, giving blunt opinions of when to use what products and the pros/cons of each.
What is hadoop
The document discusses big data and Hadoop. It describes the three V's of big data - variety, volume, and velocity. It also discusses Hadoop components like HDFS, MapReduce, Pig, Hive, and YARN. Hadoop is a framework for storing and processing large datasets in a distributed computing environment. It allows for the ability to store and use all types of data at scale using commodity hardware.
More Related Content
What's hot
platform for Machine Learning
The document discusses several machine learning platforms for 2020 including Altair Knowledge Studio, Anaconda Enterprise, Databricks Unified Analytics Platform, Dataiku Data Science Studio, DataRobot, Domino Data Science Platform, H2O, KNIME Analytics Platform, MATLAB, and TIBCO Data Science. Each platform is briefly described and a link to a YouTube video about the platform is provided.
ML-Ops: From Proof-of-Concept to Production Application
Successfully deploying a working machine learning prototype to a production application is a challenging task, frought with difficulties not experienced in traditional software deployments.
In this talk, you will learn techniques to successfully deploy ML applications in a scalable, maintainable, and automated way.
What’s New with Databricks Machine Learning
In this session, the Databricks product team provides a deeper dive into the machine learning announcements. Join us for a detailed demo that gives you insights into the latest innovations that simplify the ML lifecycle — from preparing data, discovering features, and training and managing models in production.
Managing R&D Data on Parallel Compute Infrastructure
Clinical genomic analytics pipelines using Databricks and the Delta Lake for the benefit of loading individual reads from raw sequencing or base-call files have significant advantages over more traditional methods. Analysis pipelines that perform genomic mapping to purpose-built reference data artifacts persisted to tables allows for enhanced performance that is magnitudes greater than previous mapping methods. These scalable, reproducible, and potentially open sourced methods have the ability to transform bioinformatics and R&D data management / governance.
Deep Learning for Recommender Systems with Nick pentreath
In the last few years, deep learning has achieved significant success in a wide range of domains, including computer vision, artificial intelligence, speech, NLP, and reinforcement learning. However, deep learning in recommender systems has, until recently, received relatively little attention. This talks explores recent advances in this area in both research and practice. I will explain how deep learning can be applied to recommendation settings, architectures for handling contextual data, side information, and time-based models, and compare deep learning approaches to other cutting-edge contextual recommendation models, and finally explore scalability issues and model serving challenges.
Real World End to End machine Learning Pipeline
Purpose of this presentation is to highlight how end to end machine learning looks like in real world enterprise. This is to provide insight to aspiring data scientist who have been through courses or education in ML that mostly focus on ML algorithms and not end to end pipeline.
Architecture and components mentioned in Slide 11 will be discussed in detailed in series of post on LinkedIn over the course of next few month
To get updates on this follow me on LinkedIn or search/follow hashtag #end2endDS. Post will be active in August 2019 and will be posted till September 2019
Accelerating Innovation with Unified Analytics with Ali Ghodsi
Today at the 10th Spark Summit, Databricks CEO & Co-founder revealed Databricks Serverless, a new initiative to offer serverless computing for complex data science and Apache Spark workloads. Databricks Serverless is the first product to offer a serverless API for Apache Spark, greatly simplifying and unifying data science and big data workloads for both end-users and DevOps.
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligence, "Making Big Data easier on Azure"
AI meets Big Data
This document discusses artificial intelligence and machine learning. It provides a brief history of AI from the Perceptron model in 1958 to modern deep learning approaches. It then discusses several applications of machine learning like image classification, medical diagnosis, and autonomous vehicles. It also discusses challenges like distributed machine learning and hidden technical debt. Finally, it provides examples of how AI can be applied to commerce and automotive use cases.
Building predictive models in Azure Machine Learning
This presentation covers how to build and drive insights from data by building machine learning models. The session covers how to develop and train models in Python/R using Azure Machine Learning. The session covers how to explore key concepts in data acquisition, preparation, exploration, and visualization, and take a look at how to build a predictive solution using Azure Machine Learning, R, and Python. The session covers tips and tricks on selecting the right algorithm for your data science problem and how to utilize Machine Learning to solve it.
Summary introduction to data engineering
Data engineering involves designing, building, and maintaining data warehouses to transform raw data into queryable forms that enable analytics. A core task of data engineers is Extract, Transform, and Load (ETL) processes - extracting data from sources, transforming it through processes like filtering and aggregation, and loading it into destinations. Data engineers help divide systems into transactional (OLTP) and analytical (OLAP) databases, with OLTP providing source data to data warehouses analyzed through OLAP systems. While similar, data engineers focus more on infrastructure and ETL processes, while data scientists focus more on analysis, modeling, and insights.
Big Data- Automotive Industry Use Case
Big Data has emerged as a powerful new technology paradigm. To manage the massive data generated by social media, online transactions, Weblogs, or sensors, Big Data incorporates innovative technologies in data management (unstructured, semi-structured and structured), processing, real-time analytics, and visualization. It is also useful for reporting in circumstances where a relational database approach is not effective or too costly. This Big Data project is to be primarily exposed to utilize the tools. Tool usage, programming, algorithms and application development are covered in relevant courses.
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Zalando transitioned from a centralized data platform to a data mesh architecture. This decentralized their data infrastructure by having individual domains own datasets and pipelines rather than a central team. It provided self-service data infrastructure tools and governance to enable domains to operate independently while maintaining global interoperability. This improved data quality by making domains responsible for their data and empowering them through the data mesh approach.
Hadoop for Humans: Introducing SnapReduce 2.0
In this webinar, we talk about Hadoop, big data and SnapReduce 2.0 with SnapLogic Chief Scientist Greg Benson, Professor of Computer Science at the University of San Francisco. This webinar features a dive into SnapReduce, and a discussion about how SnapLogic delivers big data acquisition, better big data preparation and universal big data delivery.
To learn more, visit: http://www.snaplogic.com/snapreduce
Data & AI Platform Concepts
This is the pilot episode of Data & AI Platform Concepts, where I intend to cover all the aspects of Data & AI Platform like Data Systems, Analytics/ML/AI, Cloud Services, DevOps, Governance.
https://www.ankitrathi.com
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...Amazon Web Services Korea
This document discusses the democratization of data science and machine learning using automated machine learning tools. It provides examples of how DataRobot has helped customers in various industries build predictive models faster and with less coding than traditional approaches. Specifically, it summarizes how DataRobot has helped customers in banking, insurance, retail, and other industries with use cases like predictive maintenance, sales forecasting, fraud detection, customer churn prediction, and insurance underwriting.Snaplogic Live: Big Data in Motion
Watch this recorded demonstration of SnapLogic from our team of experts who answer your hybrid cloud and big data integration questions.
demo, ipaas, elastic integration, cloud data, app integration, data integration, hybrid could integration, big data, big data integration
How Cloud is Affecting Data Scientists
This document discusses how data science models have transitioned to the cloud to take advantage of greater computing resources. It notes that data science models are resource-intensive and traditionally required powerful local machines. The cloud allows data scientists to run models on cloud infrastructure for lower costs than high-end laptops and with access to many GPUs. Several major cloud platforms - Azure, AWS, and Google Cloud - are discussed and compared in terms of their machine learning offerings. The document also introduces Microsoft's Team Data Science Process, which aims to help data science teams collaborate more effectively on projects in the cloud.
Platform for Data Scientists
This document discusses a platform for data scientists that aims to automate routine jobs, maximize resource utilization, and allow data scientists to focus more on business solutions. The platform provides capabilities for data capture, analysis, modeling, and output of analytics. It seeks to reduce the time taken to turn data into insights from months to weeks. Key elements of the platform include tools for exploratory data analysis, advanced modeling, distributed architecture, bespoke algorithms, and packaged analytics solutions.
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
This document discusses the evolution from traditional machine learning to learning machines. It outlines the machine learning process and highlights how learning machines enable continuous feedback and retraining through automated modeling. The key design principles of learning machines are presented as collaboration across roles, convergence of technologies, and simplicity through automation and intuitiveness. Examples are provided of how learning machines can power experiences and services.
What's hot (20)
ML-Ops: From Proof-of-Concept to Production Application
ML-Ops: From Proof-of-Concept to Production Application
Managing R&D Data on Parallel Compute Infrastructure
Managing R&D Data on Parallel Compute Infrastructure
Deep Learning for Recommender Systems with Nick pentreath
Deep Learning for Recommender Systems with Nick pentreath
Accelerating Innovation with Unified Analytics with Ali Ghodsi
Accelerating Innovation with Unified Analytics with Ali Ghodsi
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Building predictive models in Azure Machine Learning
Building predictive models in Azure Machine Learning
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...
Democratization - New Wave of Data Science (홍운표 상무, DataRobot) :: AWS Techfor...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
Similar to Data Lakes on Public Cloud: Breaking Data Management Monoliths
Building a modern data warehouse
Embarking on building a modern data warehouse in the cloud can be an overwhelming experience due to the sheer number of products that can be used, especially when the use cases for many products overlap others. In this talk I will cover the use cases of many of the Microsoft products that you can use when building a modern data warehouse, broken down into four areas: ingest, store, prep, and model & serve. It’s a complicated story that I will try to simplify, giving blunt opinions of when to use what products and the pros/cons of each.
What is hadoop
The document discusses big data and Hadoop. It describes the three V's of big data - variety, volume, and velocity. It also discusses Hadoop components like HDFS, MapReduce, Pig, Hive, and YARN. Hadoop is a framework for storing and processing large datasets in a distributed computing environment. It allows for the ability to store and use all types of data at scale using commodity hardware.
Big Data Analytics with Hadoop, MongoDB and SQL Server
This document discusses SQL Server and big data analytics projects in the real world. It covers the big data technology landscape, big data analytics, and three big data analytics scenarios using different technologies like Hadoop, MongoDB, and SQL Server. It also discusses SQL Server's role in the big data world and how to get data into Hadoop for analysis.
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
In today’s world of exponentially growing big data, enterprises are becoming increasingly more aware of the business utility and necessity of harnessing, storing and analyzing this information. Apache Hadoop has rapidly evolved to become a leading platform for managing and processing big data, with the vital management, monitoring, metadata and integration services required by organizations to glean maximum business value and intelligence from their burgeoning amounts of information on customers, web trends, products and competitive markets. In this session, Hortonworks' Himanshu Bari will discuss the opportunities for deriving business value from big data by looking at how organizations utilize Hadoop to store, transform and refine large volumes of this multi-structured information. Connolly will also discuss the evolution of Apache Hadoop and where it is headed, the component requirements of a Hadoop-powered platform, as well as solution architectures that allow for Hadoop integration with existing data discovery and data warehouse platforms. In addition, he will look at real-world use cases where Hadoop has helped to produce more business value, augment productivity or identify new and potentially lucrative opportunities.
Logical Data Warehouse: How to Build a Virtualized Data Services Layer
The document discusses the emergence of logical data warehouses in response to big data. It describes how a logical data warehouse uses virtualization, distributed processing, and other techniques to provide a unified view of data across different repositories like Hadoop, relational databases and NoSQL stores. It also discusses how organizations can optimize resources by offloading analytical workloads from their enterprise data warehouse to Hadoop clusters to reduce costs while still using existing code and applications.
Securing your Big Data Environments in the Cloud
Big Data tools are becoming a critical part of enterprise architectures and as such securing the data, at rest, and in motion is a necessity. More so, when you’re implementing these solutions in the cloud and the data doesn't reside within the confines of your trusted data center. Also, there is a fine balance between implementing enterprise-grade security and negotiating utmost performance given the overheads of encryption and/or identity management.
This session is designed to tackle these challenges head on and explain the various options available in the cloud. The focal points are the implementation of tools like Ranger and Knox for cloud deployments, but we also pay attention to the security features offered in the cloud that complement this process and secure the data in unprecedented ways.
Cloud Security + OSS Security tools are a deadly combination, when it comes to securing your Data Lake.
عصر کلان داده، چرا و چگونه؟
ارائه در زمینه کلان داده،
کارگاه آموزشی "عصر کلان داده، چرا و چگونه؟" در بیست و دومین کنفرانس انجمن کامپیوتر ایران csicc2017.ir
وحید امیری
vahidamiry.ir
datastack.ir
Hadoop workshop
This document outlines an agenda for a Hadoop workshop covering Cisco's use of Hadoop. The agenda includes introductions, presentations on Hadoop concepts and Cisco's Hadoop architecture, and two hands-on exercises configuring Hadoop and using Hive and Impala for analytics. Key topics to be covered are Hadoop and big data concepts, Cisco's Webex Hadoop architecture using Cisco UCS, and how Hadoop addresses the challenges of large volumes of structured and unstructured data across global data centers.
EMC Isilon Database Converged deck
The document discusses trends in data and analytics, including the growth of digital data and devices. It summarizes predictions that by 2020 there will be over 30 billion connected devices, 7 billion people, and over 1 million new businesses. The document also discusses how analytics is converging databases and Hadoop to enable querying both structured and unstructured data, and how this will impact industries and skills. It focuses on trends like machine learning and the increasing importance of outcomes over specific technologies like Hadoop.
A Reference Architecture for ETL 2.0
More and more organizations are moving their ETL workloads to a Hadoop based ELT grid architecture. Hadoop`s inherit capabilities, especially it`s ability to do late binding addresses some of the key challenges with traditional ETL platforms. In this presentation, attendees will learn the key factors, considerations and lessons around ETL for Hadoop. Areas such as pros and cons for different extract and load strategies, best ways to batch data, buffering and compression considerations, leveraging HCatalog, data transformation, integration with existing data transformations, advantages of different ways of exchanging data and leveraging Hadoop as a data integration layer. This is an extremely popular presentation around ETL and Hadoop.
Migrating legacy ERP data into Hadoop
Legacy ERP architecture offers an incredibly, efficient means of operational resource management, but a real challenge comes from extracting business insights from them. Over the past 30 years, ERP data system, such as SAP, can be hard to interact with especially at the source database level. Whether initial translation of business logic and hierarchies create significant customizations, as well as, merging those changes into analytical applications. Overall, the entire process of designing self-service reporting with business level context can be quite cumbersome, looking at an example platform like, SAP, which contain pre-packaged modules (MM, SD, PP, etc), integrating these systems into a series of pre-built analytics.
The orchestration and integration over a wide range of open source technology solutions with some commercial CDC and reporting solutions into a reference solution that mimics several real customer scenarios today, living on relational platforms. Key considerations of extracting from the operational system of record, especially the merging of multiple systems in different time zones, will be addressed. Furthermore, the integration concerns of an analytics Hadoop platform, using HIVE Acid and Merge, as well as, flattening techniques for dimensional models. Many times a customer is temporarily limited in the range of data their ERP can contain, and older data is often offloaded to secondary systems or cold archiving entities. That goes away, but the opportunities expanded with real-time reporting across all of history, and the expanded use cases with advanced machine learnings methods.
Speakers
Jordan Martz, Director of Tech Solutions, Attunity
David Freriks, Technology Evangelist, Qlik
Thu-310pm-Impetus-SachinAndAjay
This document discusses the emergence of logical data warehouses and how they can help organizations address challenges posed by big data. A logical data warehouse takes a virtualized approach to integrating data from multiple sources like relational databases, NoSQL stores, and file systems. It provides a single, unified view of data while keeping the underlying systems decoupled. The document also describes how organizations can use techniques like data virtualization and offloading to optimize workloads between their enterprise data warehouse and Hadoop data lake. This helps reduce costs while improving query performance and resource utilization.
Hadoop Summit 2012 | Integrating Hadoop Into the Enterprise
The power of Hadoop lies in its ability to help users cost effectively analyze all kinds of data. We are now seeing the emergence of a new class of analytic applications that can only be enabled by a comprehensive big data platform. Such a platform extends the Hadoop framework with built-in analytics, robust developer tools, and the integration, reliability, and security capabilities that enterprises demand for complex, large scale analytics. In this session, we will share innovative analytics use cases from actual customer implementations using an enterprise-class big data analytics platform.
Integrating Hadoop Into the Enterprise
The document discusses integrating Hadoop into the enterprise data infrastructure. It describes common uses of Hadoop including enabling new analytics by joining transactional data from databases with interaction data in Hadoop. The document outlines key aspects of integration like data import/export between Hadoop and existing data stores using tools like Sqoop, various ETL tools, and connecting business intelligence and analytics tools to Hadoop. Example architectures are shown integrating Hadoop with databases, data warehouses, and other systems.
Microsoft's Hadoop Story
This document discusses Hadoop and its relationship to Microsoft technologies. It provides an overview of what Big Data is, how Hadoop fits into the Windows and Azure environments, and how to program against Hadoop in Microsoft environments. It describes Hadoop capabilities like Extract-Load-Transform and distributed computing. It also discusses how HDFS works on Azure storage and support for Hadoop in .NET, JavaScript, HiveQL, and Polybase. The document aims to show Microsoft's vision of making Hadoop better on Windows and Azure by integrating with technologies like Active Directory, System Center, and SQL Server. It provides links to get started with Hadoop on-premises and on Windows Azure.
Big Data in the Microsoft Platform
This document discusses building big data solutions using Microsoft's HDInsight platform. It provides an overview of big data and Hadoop concepts like MapReduce, HDFS, Hive and Pig. It also describes HDInsight and how it can be used to run Hadoop clusters on Azure. The document concludes by discussing some challenges with Hadoop and the broader ecosystem of technologies for big data beyond just Hadoop.
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Find out how Hortonworks and IBM help you address these challenges to enable success to optimize your existing EDW environment.
https://hortonworks.com/webinar/modernize-existing-edw-ibm-big-sql-hortonworks-data-platform/
Big_SQL_3.0_Whitepaper
This document provides an overview of the new features and capabilities of IBM's Big SQL 3.0, an SQL-on-Hadoop solution. Big SQL 3.0 replaces the previous MapReduce-based architecture with a massively parallel processing SQL engine that pushes processing down to HDFS data nodes for low-latency queries. It features a shared-nothing parallel database architecture, rich SQL support including stored procedures and functions, automatic memory management, workload management tools, and fault tolerance. The document discusses the new architecture, performance improvements, and how Big SQL 3.0 represents an important advancement for SQL-on-Hadoop solutions.
Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)
Watch full webinar here: https://bit.ly/3aePFcF
Historically data lakes have been created as a centralized physical data storage platform for data scientists to analyze data. But lately the explosion of big data, data privacy rules, departmental restrictions among many other things have made the centralized data repository approach less feasible. In this webinar, we will discuss why decentralized multipurpose data lakes are the future of data analysis for a broad range of business users.
Attend this session to learn:
- The restrictions of physical single purpose data lakes
- How to build a logical multi purpose data lake for business users
- The newer use cases that makes multi purpose data lakes a necessity
Modern data warehouse
The document discusses the modern data warehouse and key trends driving changes from traditional data warehouses. It describes how modern data warehouses incorporate Hadoop, traditional data warehouses, and other data stores from multiple locations including cloud, mobile, sensors and IoT. Modern data warehouses use multiple parallel processing (MPP) architecture and the Apache Hadoop ecosystem including Hadoop Distributed File System, YARN, Hive, Spark and other tools. It also discusses the top Hadoop vendors and Oracle's technical innovations on Hadoop for data discovery, transformation, discovery and sharing. Finally, it covers the components of big data value assessment including descriptive, predictive and prescriptive analytics.
Similar to Data Lakes on Public Cloud: Breaking Data Management Monoliths (20)
Big Data Analytics with Hadoop, MongoDB and SQL Server
Big Data Analytics with Hadoop, MongoDB and SQL Server
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
Logical Data Warehouse: How to Build a Virtualized Data Services Layer
Logical Data Warehouse: How to Build a Virtualized Data Services Layer
Hadoop Summit 2012 | Integrating Hadoop Into the Enterprise
Hadoop Summit 2012 | Integrating Hadoop Into the Enterprise
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)
Logical Data Lakes: From Single Purpose to Multipurpose Data Lakes (APAC)
More from Itai Yaffe
Mastering Partitioning for High-Volume Data Processing
Yulia Antonovsky (Senior Software Engineer II) @ Akamai:
Our cloud-based ingest pipeline processes over 10 Gb of security events data per second, which demands high-performance processing and analysis. To achieve this, we've implemented efficient partitioning using Java and Spark applications running on AKS and leveraging Kafka. This allows us to provide real-time analytics within two minutes and heavy batch processing for deeper analysis hourly. During this talk, we will cover how we use Kafka to scale our Spark application on K8s, partitioning strategies for high-volume data processing, and how partitioning helps avoid storage throttling issues.
Solving Data Engineers Velocity - Wix's Data Warehouse Automation
This document discusses Wix's solution to automate data warehouse maintenance called Data Warehouse Automation (DWHA). It consists of three components - BI Bank, Metric Collector, and DWHA. BI Bank defines data sources and semantics. Metric Collector extracts metrics from sources efficiently. DWHA understands changes between runs, aggregates data, and handles different table types and changes over time. The presenter demonstrates how DWHA streamlines maintenance by automatically handling differences between runs. They also discuss additional DWHA capabilities and plans for a UI.
Lessons Learnt from Running Thousands of On-demand Spark Applications
Ada Sharoni (Software Engineering Architect) @ Hunters:
Imagine you had to manage thousands of Spark applications that are automatically spinning up on-demand upon every customer interaction.
Our unique constraints in Hunters have led us to adopt an architecture and concepts that we believe many other companies will find useful.
In this lecture we will share our solutions and insights in running many lightweight, cheap Spark applications on Kubernetes, that can easily survive frequent restarts and smartly share resources on Spot EC2 instances.
Planning a data solution - "By Failing to prepare, you are preparing to fail"
Eynav Mass (VP R&D) @ Oribi:
When it comes to data solutions, one-size doesn't fit all.
Choosing the right best-matching database, or data tools, can be a game-changer for your system.
How can you take such a decision effectively?
The system, the company, the product, and probably your team - all are evolving, and the best solution for today may not fit tomorrow's needs.
In order to pick a data solution for longer term, you should evaluate the optional data tools according to several factors.
These factors will reflect the requirements looking forward.
At the session, we will discuss these factors, along with sharing some real-life stories and lessons learned, to help you properly plan & prepare your data solutions.
Evaluating Big Data & ML Solutions - Opening Notes
Esther Sánchez (Global Executive Committee member & Spain Chapter Lead) and Itai Yaffe (Israel Chapter Lead) @ Women in Big Data:
Opening notes about the digital era, diversity, and Women in Big Data (https://www.womeninbigdata.org/).
Big data serving: Processing and inference at scale in real time
Jon Bratseth (VP Architect) @ Verizon Media:
The big data world has mature technologies for offline analysis and learning from data, but have lacked options for making data-driven decisions in real time.
When it is sufficient to consider a single data point model servers such as TensorFlow serving can be used but in many cases you want to consider many data points to make decisions.
This is a difficult engineering problem combining state, distributed algorithms and low latency, but solving it often makes it possible to create far superior solutions when applying machine learning.
This talk will explain why this is a hard problem, show the advantages of solving it, and introduce the open source Vespa.ai platform which is used to implement such solutions in some of the largest scale problems in the world including the world's third largest ad serving system.
Unleashing the Power of your Data
Orit Alul (Sr. Solutions Architect) @ AWS:
As data is growing at an exponential rate, we are interested not only in being able to analyze the past or present but also in predicting the future!
In this session, Orit will talk about the power of data combined with machine learning.
Building a highly scalable and flexible data architecture in the cloud to collect, process, and analyze data, in order to get timely insights and react quickly to new information.
In addition, Orit will present best practices, performance and optimization tips for building a Data Lake in the cloud.
Data Lake on Public Cloud - Opening Notes
Itai Yaffe (Tech Lead, Big Data group) @ Nielsen:
Opening notes about Women in Big Data (https://www.womeninbigdata.org/).
Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the ...
Roi Teveth (Data Engineer) and Itai Yaffe (Tech Lead, Big Data group) @ Nielsen:
At Nielsen Identity Engine, we use Spark to process 10’s of TBs of data. Our ETLs, orchestrated by Airflow, spin-up AWS EMR clusters with thousands of nodes per day.
In this talk, we’ll guide you through migrating Spark workloads to Kubernetes with minimal changes to Airflow DAGs, using the open-sourced GCP Spark-on-K8s operator and the native integration we recently contributed to the Airflow project.
DevTalks Reimagined 2020 - Funnel Analysis with Spark and Druid
Itai Yaffe (Tech Lead, Big Data group) @ Nielsen:
Every day, millions of advertising campaigns are happening around the world.
As campaign owners, measuring the ongoing campaign effectiveness (e.g "how many distinct users saw my online ad VS how many distinct users saw my online ad, clicked it and purchased my product?") is super important.
However, this task (often referred to as "funnel analysis") is not an easy task, especially if the chronological order of events matters.
So, while the combination of Druid and ThetaSketch aggregators can answer some of these questions, it still can’t answer the question "how many distinct users viewed the brand’s homepage FIRST and THEN viewed product X page?"
In this talk, we will discuss how we combine Spark, Druid and ThetaSketch aggregators to answer such questions at scale.
Virtual Apache Druid Meetup: AIADA (Ask Itai and David Anything)
Itai Yaffe (Tech Lead, Big Data group) @ Nielsen and David Bar (Software Architect) @ ForeScout:
At this Ask Me Anything-style virtual meetup, Itai and David answered questions about Apache Druid from the unique perspectives of an open-source Druid user (Itai) and Imply customer (David).
Introducing Kafka Connect and Implementing Custom Connectors
Kobi Hikri (Independent Software Architect and Consultant):
Kobi provides a short intro to Kafka Connect, and then shows an actual code example of developing and dockerizing a custom connector.
A Day in the Life of a Druid Implementor and Druid's Roadmap
This document summarizes a typical day for a Druid architect. It describes common tasks like evaluating production clusters, analyzing data and queries, and recommending optimizations. The architect asks stakeholders questions to understand usage and helps evaluate if Druid is a good fit. When advising on Druid, the architect considers factors like data sources, query types, and technology stacks. The document also provides tips on configuring clusters for performance and controlling segment size.
Scalable Incremental Index for Druid
Dr. Edward (Eddie) Bortnikov (Senior Director of Research) @ Verizon Media:
Ingestion and queries of real-time data in Druid are performed by a core software component named Incremental Index (I^2).
I^2’s scalability is paramount to the speed of the ingested data becoming queryable as well as to the operational efficiency of the Druid cluster.
The current I^2 Implementation is based on the traditional ordered JDK key-value (KV-)map.
We present an experimental I^2 implementation that is based on a novel data structure named OakMap - a scalable thread-safe off-heap KV-map for Big Data applications in Java.
With OakMap, I^2 can ingest data at almost 2x speed while using 30% less RAM.
The project is expected to become GA in 2020.
Funnel Analysis with Spark and Druid
Itai Yaffe (Tech Lead, Big Data group) @ Nielsen:
Every day, millions of advertising campaigns are happening around the world.
As campaign owners, measuring the ongoing campaign effectiveness (e.g “how many distinct users saw my online ad VS how many distinct users saw my online ad, clicked it and purchased my product?”) is super important.
However, this task (often referred to as “funnel analysis”) is not an easy task, especially if the chronological order of events matters. So, while the combination of Druid and ThetaSketch aggregators can answer some of these questions, it still can’t answer the question "how many distinct users viewed the brand’s homepage FIRST and THEN viewed product X page?"
In this talk, we will discuss how we combine Spark, Druid and ThetaSketch aggregators to answer such questions at scale.
The benefits of running Spark on your own Docker
Shir Bromberg (Big Data team leader) @ Yotpo:
Nowadays, many of an organization’s main applications rely on Spark pipelines. As these applications become more significant to businesses, so does the need to quickly deploy, test and monitor them.
The standard way of running spark jobs is to deploy it on a dedicated managed cluster. However, this solution is relatively expensive with potentially high setup time. Therefore, we developed a way to run Spark on any container orchestration platform. This allows us to run Spark in a simple, custom and testable way.
In this talk, we will present our open-source dockers for running Spark on Nomad servers. We will cover:
* The issues we had running spark on managed clusters and the solution we developed.
* How to build a spark docker.
* And finally, what you may achieve by using Spark on Nomad.
Optimizing Spark-based data pipelines - are you up for it?
Etti Gur (Senior Big Data developer) and Itai Yaffe (Tech Lead, Big Data group) @ Nielsen:
At Nielsen Marketing Cloud, we provide our customers (marketers and publishers) real-time analytics tools to measure their ongoing campaigns' efficiency.
To achieve that, we need to ingest billions of events per day into our big data stores and we need to do it in a scalable yet cost-efficient manner.
In this talk, we will discuss how we significantly optimized our Spark-based in-flight analytics daily pipeline, reducing its total execution time from over 20 hours down to 2 hours, resulting in a huge cost reduction.
Topics include:
* Ways to identify optimization opportunities
* Optimizing Spark resource allocation
* Parallelizing Spark output phase with dynamic partition inserts
* Running multiple Spark "jobs" in parallel within a single Spark application
Scheduling big data workloads on serverless infrastructure
Ilai Malka from Nielsen at AWS Community Day TLV, December 2019 (https://awscommunitydaytelaviv2019.splashthat.com/):
Scheduling big data workloads is challenging. It's extra challenging when running on Serverless infrastructure.
At Nielsen Marketing Cloud, we've built a system that uploads 250 billion events per day to partner ad platforms, running on Serverless infrastructure (AWS Lambda and OpenFaaS).
Creating a 'scheduler' for this system required:
1. Rate-limiting to prevent flooding partner platforms.
2. High utilization to keep costs low
3. Careful bottleneck management to keep the system humming
https://www.linkedin.com/in/ilai-malka-93b06172/
https://twitter.com/IlaiMalka
#Nielsen #NielsenMarketingCloud #AWSCommunityDay #Serverless
GraphQL API on a Serverless Environment
The document discusses using GraphQL to build a serverless API for a mobile app that detects construction errors. Key points include:
1. GraphQL provides an efficient way to fetch data from microservices in a single roundtrip and allows flexible field selection.
2. The previous monolithic REST API was split into microservices for data fetching, state management, and serving the API to improve performance and scalability.
3. Serverless is a good fit for the stateless GraphQL architecture as resolvers can trigger other serverless functions and the API can scale up and down easily.
Serverless data processing built for internet SCALE
Ilai Malka (Big Data Developer) & Opher Dubrovsky (Big Data Team lead) @ Nielsen:
You too can build a serverless data pipeline processing 250 billion events/day. In this talk you’ll hear details from a real-life ad delivery system we’ve built running on AWS Lambda serverless infrastructure.
You’ll hear about:
- System design & pitfalls to avoid
- Fault tolerance, self-healing and recoverability
- CI/CD process & avoiding development velocity slowdown
More from Itai Yaffe (20)
Mastering Partitioning for High-Volume Data Processing
Mastering Partitioning for High-Volume Data Processing
Solving Data Engineers Velocity - Wix's Data Warehouse Automation
Solving Data Engineers Velocity - Wix's Data Warehouse Automation
Lessons Learnt from Running Thousands of On-demand Spark Applications
Lessons Learnt from Running Thousands of On-demand Spark Applications
Planning a data solution - "By Failing to prepare, you are preparing to fail"
Planning a data solution - "By Failing to prepare, you are preparing to fail"
Evaluating Big Data & ML Solutions - Opening Notes
Evaluating Big Data & ML Solutions - Opening Notes
Big data serving: Processing and inference at scale in real time
Big data serving: Processing and inference at scale in real time
Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the ...
Airflow Summit 2020 - Migrating airflow based spark jobs to kubernetes - the ...
DevTalks Reimagined 2020 - Funnel Analysis with Spark and Druid
DevTalks Reimagined 2020 - Funnel Analysis with Spark and Druid
Virtual Apache Druid Meetup: AIADA (Ask Itai and David Anything)
Virtual Apache Druid Meetup: AIADA (Ask Itai and David Anything)
Introducing Kafka Connect and Implementing Custom Connectors
Introducing Kafka Connect and Implementing Custom Connectors
A Day in the Life of a Druid Implementor and Druid's Roadmap
A Day in the Life of a Druid Implementor and Druid's Roadmap
Optimizing Spark-based data pipelines - are you up for it?
Optimizing Spark-based data pipelines - are you up for it?
Scheduling big data workloads on serverless infrastructure
Scheduling big data workloads on serverless infrastructure
Serverless data processing built for internet SCALE
Serverless data processing built for internet SCALE
Recently uploaded
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
原版一模一样【微信：741003700 】【(lbs毕业证书)伦敦商学院毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版南十字星大学毕业证(SCU毕业证书)学历如何办理
原版办理【微信号:BYZS866】【南十字星大学毕业证(SCU毕业证书)】【微信号:BYZS866】《成绩单、外壳、雅思、offer、真实留信官方学历认证（永久存档/真实可查）》采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路）我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信号BYZS866】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信号BYZS866】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
一比一原版加拿大渥太华大学毕业证（uottawa毕业证书）如何办理
原版一模一样【微信：741003700 】【渥太华大学毕业证（uottawa毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
How To Control IO Usage using Resource Manager
How To Control IO Usage using Resource Manager
In Multitenant and Exadata architecture
Open Source Contributions to Postgres: The Basics POSETTE 2024
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
一比一原版加拿大麦吉尔大学毕业证（mcgill毕业证书）如何办理
原版一模一样【微信：741003700 】【加拿大麦吉尔大学毕业证（mcgill毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版(Sheffield毕业证书)谢菲尔德大学毕业证如何办理
原版制作【微信:41543339】【(Sheffield毕业证书)谢菲尔德大学毕业证】【微信:41543339】《成绩单、外壳、雅思、offer、留信学历认证（永久存档/真实可查）》采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器一制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
by
Timothy Spann
Principal Developer Advocate
https://budapestdata.hu/2024/en/
https://budapestml.hu/2024/en/
tim.spann@zilliz.com
https://www.linkedin.com/in/timothyspann/
https://x.com/paasdev
https://github.com/tspannhw
https://www.youtube.com/@flank-stack
milvus
vector database
gen ai
generative ai
deep learning
machine learning
apache nifi
apache pulsar
apache kafka
apache flink
一比一原版(UO毕业证)渥太华大学毕业证如何办理
原件一模一样【微信：95270640】【渥太华大学毕业证UO学位证成绩单】【微信：95270640】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信：95270640】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信：95270640】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份【微信：95270640】
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
→ 【关于价格问题（保证一手价格）
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！
办理渥太华大学毕业证毕业证offerUO学位证【微信：95270640 】外观非常精致，由特殊纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理渥太华大学毕业证UO学位证毕业证offer【微信：95270640 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理渥太华大学毕业证毕业证offerUO学位证【微信：95270640 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理渥太华大学毕业证毕业证offerUO学位证【微信：95270640 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
学校原件一模一样【微信：741003700 】《(unimelb毕业证书)墨尔本大学毕业证》【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
Generative Classifiers: Classifying with Bayesian decision theory, Bayes’ rule, Naïve Bayes classifier.
Discriminative Classifiers: Logistic Regression, Decision Trees: Training and Visualizing a Decision Tree, Making Predictions, Estimating Class Probabilities, The CART Training Algorithm, Attribute selection measures- Gini impurity; Entropy, Regularization Hyperparameters, Regression Trees, Linear Support vector machines.
一比一原版马来西亚博特拉大学毕业证（upm毕业证）如何办理
原版一模一样【微信：741003700 】【马来西亚博特拉大学毕业证（upm毕业证）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理马来西亚博特拉大学毕业证（upm毕业证）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理马来西亚博特拉大学毕业证（upm毕业证）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理马来西亚博特拉大学毕业证（upm毕业证）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理马来西亚博特拉大学毕业证（upm毕业证）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版美国帕森斯设计学院毕业证（parsons毕业证书）如何办理
原版一模一样【微信：741003700 】【美国帕森斯设计学院毕业证（parsons毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版兰加拉学院毕业证(Langara毕业证书)学历如何办理
原版办【微信号:BYZS866】【兰加拉学院毕业证(Langara毕业证书)】【微信号:BYZS866】《成绩单、外壳、雅思、offer、真实留信官方学历认证（永久存档/真实可查）》采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路）我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信号BYZS866】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信号BYZS866】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
一比一原版斯威本理工大学毕业证（swinburne毕业证）如何办理
原版一模一样【微信：741003700 】【斯威本理工大学毕业证（swinburne毕业证）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理斯威本理工大学毕业证（swinburne毕业证）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理斯威本理工大学毕业证（swinburne毕业证）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理斯威本理工大学毕业证（swinburne毕业证）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理斯威本理工大学毕业证（swinburne毕业证）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版悉尼大学毕业证如何办理
原版一模一样【微信：741003700 】【悉尼大学毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理悉尼大学毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理悉尼大学毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理悉尼大学毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理悉尼大学毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Recently uploaded (20)
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
DATA COMMS-NETWORKS YR2 lecture 08 NAT & CLOUD.docx
Data Lakes on Public Cloud: Breaking Data Management Monoliths
- 1. Data Lakes in the Public Cloud: Breaking Data Management Monoliths Sharon Dashet, Sr. Data Analytics Solution Lead, GCP https://il.linkedin.com/in/sharon-dashet
- 2. 01Intro to Data Lakes
- 3. It all started with RDBMS….
- 4. Traditional EDW players ~1995 Big data vendors ~2005 Cloud platform vendors ~2010 Specialized Cloud vendors ~2012 Data Management timeline Relational OLTP 80s Database Developer Backend Developer Application DBA Production DBA Data Scientist Data Analysts BI/OLAP Expert SQL Expert Governance MDM Big Data Developer Big Data Architect ML Engineer Hadoop Admin Hadoop Expert AI Scientist CDO Cloud Data Engineer Cloud Data Architect
- 5. HBase ( NoSQL datastore) Flume (Log aggregation and transport) Sqoop (Import and export of relational data) Ambari (Management and monitoring) MapReduce (Cluster data processing) YARN (Cluster resource management) HDFS (Hadoop Distributed File System) HCatalog (Metadata) Oozie (Workflow automation) Zookeeper (Coordination ) Pig (Scripting) Flink (Streams) Mahout & Spark ML (Machine learning) Presto (Distributed SQL query) (Cluster data processing) Hive (SQL DW) The Hadoop ecosystem is very popular for Big Data workloads
- 6. Multi-User, Shared Hadoop Cluster Data (HDFS) Temp Data (HDFS) Metadata (Hive metastore, RDBMS) AuthZ Policies, Audit, Governance (Ranger, Atlas) Compute: YARN Hive Spark MR R AuthN Kerberos, LDAP Kafka, Storm, Flume, Cassandra, Hbase, ELK etc. Typical on-premises deployment
- 7. The apache Data-Processing ecosystem
- 8. Resource utilization and overall TCO of on-prem data lakes becomes unmanageable Data governance and security issues open up compliance concerns Resource intensive data and analytics processing can lead to missed SLAs Analytics experimentation is slow due to resource provisioning time TCO Challenges Governance Challenges Agility ChallengesScaling Challenges On-prem Data Lakes are struggling to deliver value
- 9. Key market players are struggling to convert customers.
- 10. The need is still there AI is now capable of extracting value from unstructured data Cloud is faster, simpler to operate, and less expensive “80 percent of worldwide data will be unstructured by 2025” Data Lake are shifted to the cloud “By connecting data points, we can offer advice like hygiene laws for certain foods, or information on provenance. We can even integrate their local weather forecast so a store doesn't run out of ice cream on a sunny day." Sven Lipowski, Unit Owner Customer Solutions adMETERONOMIDC (source) “The ability to spin up purpose driven Hadoop clusters against our shared datasets and scale them up/down with demand is a game changer for us…” Brett Uyeshiro VP Platform Services, Pandora
- 11. 02 Patterns for Data Lakes in Public Cloud
- 12. Beyond HDFS- Storage and Compute separation Keep your storage on GCS instead of HDFS Benefits: ● Separation of Compute/Storage ● Full HDFS-compliant GCS connector ● Facilitates Job-scoped cost effective workloads (+ephemeral clusters) ● No need to provision x3 storage for replication ● No unused bytes on disks
- 13. Hive Analytics Business ReportingMapReduce ETL Machine Learning Storage Cloud Storage Hive Metastore Cloud Dataproc Clusters Job-Scoped Clusters - Beyond complicated Yarn queues ● Step away from complicated Yarn queues and multi tenancy ● Control cost and performance per workload: ○ Ephemeral Clusters ○ Mix regular and preemptible VMs in the worker pool ○ Different VM types
- 14. Beyond Yarn and into Modern Service Mesh
- 15. AI Platform Notebook s AI Platform AI Platform Notebook s 1. Data sources 2. Data Lake storage 3. Data Pipelines 4. Data Warehouse/Lake 5. ML and analytics workloads Converged Smart Analytics
- 16. Thank you