Hopsworks - The Platform for Data-Intensive AI
•
1 like•454 views
Hopsworks is a platform for data-intensive AI projects that provides: 1. End-to-end machine learning pipelines from data ingestion to model serving. 2. A feature store for organizing machine learning data. 3. Distributed deep learning using multiple GPUs for faster training.
Report
Share
Report
Share
Download to read offline
Recommended
Feature store: Solving anti-patterns in ML-systems
Modern machine learning systems may be very complex and may fall into many pitfalls. It's very easy to unintendedly introduce technical debt into such a complex structure. One of the approaches solving some of anti-patterns is a feature store. Feature store is a missing piece filling a gap between raw data and machine learning models. Not only it will help you to handle technical debt, but even more importantly speeds up time to develop new model.
Managed Feature Store for Machine Learning
All hyperscale AI companies build their machine learning platforms around a Feature Store.
A feature is a measurable property of some data-sample. It could be for example an image-pixel, a word from a piece of text, the age of a person, a coordinate emitted from a sensor, or an aggregate value like the average number of purchases within the last hour. A Feature Store is a central place to store curated features within an organization.
Feature Stores are a fuel for AI systems as we use them to train machine learning models so that we can make predictions for feature values that we have never seen before.
During this presentation you learn:
- About the concept of a Feature Store and how it can help manage feature data for Enterprises and ease the path of data from backend systems and data-lakes to Data Scientists.
- Our take on Feature Stores, including best practices and use cases and:
- How to ensure Consistent Features in both Training and Serving
Governance, Access-Control, and Versioning
- To create Training Data in the File Format of your Choice
Eliminate Inconsistency between Features in Training and Inferencing
Watch the webinar with a demo: https://www.logicalclocks.com/webinars
PyData Meetup - Feature Store for Hopsworks and ML Pipelines
Hopsworks Feature Store Machine Learning Pipelines TensorFlow PySpark
StreamSQL Feature Store (Apache Pulsar Summit)
Input features are the building blocks for machine learning models. You cannot have a great model without great features. By building on top of Apache Pulsar's infinite retention of events, we built infrastructure to serve features in production and to generate training datasets. It allowed our machine learning teams to change, test, and deploy personalization features at an extraordinary rate to 10s of millions of end-users.
This talk will discuss:
- What event-sourcing is and why it's so powerful for machine learning infrastructure.
- How we built the StreamSQL feature store on top of Pulsar, Flink, and Cassandra.
- How a feature store accelerates ML development.
The Bitter Lesson of ML Pipelines
Wasp4All Conference talk nov 2019.
https://wasp-sweden.org/event/wasp4all-future-computing-platforms-for-x/
Recommended
Feature store: Solving anti-patterns in ML-systems
Modern machine learning systems may be very complex and may fall into many pitfalls. It's very easy to unintendedly introduce technical debt into such a complex structure. One of the approaches solving some of anti-patterns is a feature store. Feature store is a missing piece filling a gap between raw data and machine learning models. Not only it will help you to handle technical debt, but even more importantly speeds up time to develop new model.
Managed Feature Store for Machine Learning
All hyperscale AI companies build their machine learning platforms around a Feature Store.
A feature is a measurable property of some data-sample. It could be for example an image-pixel, a word from a piece of text, the age of a person, a coordinate emitted from a sensor, or an aggregate value like the average number of purchases within the last hour. A Feature Store is a central place to store curated features within an organization.
Feature Stores are a fuel for AI systems as we use them to train machine learning models so that we can make predictions for feature values that we have never seen before.
During this presentation you learn:
- About the concept of a Feature Store and how it can help manage feature data for Enterprises and ease the path of data from backend systems and data-lakes to Data Scientists.
- Our take on Feature Stores, including best practices and use cases and:
- How to ensure Consistent Features in both Training and Serving
Governance, Access-Control, and Versioning
- To create Training Data in the File Format of your Choice
Eliminate Inconsistency between Features in Training and Inferencing
Watch the webinar with a demo: https://www.logicalclocks.com/webinars
PyData Meetup - Feature Store for Hopsworks and ML Pipelines
Hopsworks Feature Store Machine Learning Pipelines TensorFlow PySpark
StreamSQL Feature Store (Apache Pulsar Summit)
Input features are the building blocks for machine learning models. You cannot have a great model without great features. By building on top of Apache Pulsar's infinite retention of events, we built infrastructure to serve features in production and to generate training datasets. It allowed our machine learning teams to change, test, and deploy personalization features at an extraordinary rate to 10s of millions of end-users.
This talk will discuss:
- What event-sourcing is and why it's so powerful for machine learning infrastructure.
- How we built the StreamSQL feature store on top of Pulsar, Flink, and Cassandra.
- How a feature store accelerates ML development.
The Bitter Lesson of ML Pipelines
Wasp4All Conference talk nov 2019.
https://wasp-sweden.org/event/wasp4all-future-computing-platforms-for-x/
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
A Feature Store enables machine learning (ML) features to be registered, discovered, and used as part of ML pipelines, thus making it easier to transform and validate the training data that is fed into machine learning systems. Feature stores can also enable consistent engineering of features between training and inference, but to do so, they need a common data processing platform. The first Feature Stores, developed at hyperscale AI companies such as Uber, Airbnb, and Facebook, enabled feature engineering using domain specific languages, providing abstractions tailored to the companies’ feature engineering domains. However, a general purpose Feature Store needs a general purpose feature engineering, feature selection, and feature transformation platform.
In this talk, we describe how we built a general purpose, open-source Feature Store for ML around dataframes and Apache Spark. We will demonstrate how data engineers can transform and engineers features from backend databases and data lakes, while data scientists can use PySpark to select and transform features into train/test data in a file format of choice (.tfrecords, .npy, .petastorm, etc) on a file system of choice (S3, HDFS). Finally, we will show how the Feature Store enables end-to-end ML pipelines to be factored into feature engineering and data science stages that each can run at different cadences.
Bio:
Fabio Buso is the head of engineering at Logical Clocks AB, where he leads the Feature Store development. Fabio holds a master's degree in cloud computing and services with a focus on data intensive applications, awarded by a joint program between KTH Stockholm and TU Berlin.
Topics: feature store, MLOps.
Kim Hammar - Feature Store: the missing data layer in ML pipelines? - HopsML ...
Presentation at HopsML meetup in Stockholm. Talking about Hopsworks feature store. January 29, 2019, Stockholm.
Ml ops and the feature store with hopsworks, DC Data Science Meetup
Ml ops and the feature store with hopsworks, DC Data Science Meetup
END-TO-END MACHINE LEARNING STACK
Autodeploy a complete end-to-end machine learning pipeline on Kubernetes using tools like Spark, TensorFlow, HDFS, etc. - it requires a running Kubernetes (K8s) cluster in the cloud or on-premise.
Machine Learning with Apache Spark
Presented by David Taieb, Architect, IBM Cloud Data Services
Along with Spark Streaming, Spark SQL and GraphX, MLLib is one of the four key architectural components of Spark. It provides easy-to-use (even for beginners), powerful Machine Learning APIs that are designed to work in parallel using Spark RDDs. In this session, we’ll introduce the different algorithms available in MLLib, e.g. supervised learning with classification (binary and multi class) and regression but also unsupervised learning with clustering (K-means) and recommendation systems. We’ll conclude the presentation with a deep dive on a sample machine learning application built with Spark MLLib that predicts whether a scheduled flight will be delayed or not. This application trains a model using data from real flight information. The labeled flight data is combined with weather data from the “Insight for Weather” service available on IBM Bluemix Cloud Platform to form the training, test and blind data. Even if you are not a black belt in machine learning, you will learn in this session how to leverage powerful Machine Learning algorithms available in Spark to build interesting predictive and prescriptive applications.
About the Speaker: For the last 4 years, David has been the lead architect for the Watson Core UI & Tooling team based in Littleton, Massachusetts. During that time, he led the design and development of a Unified Tooling Platform to support all the Watson Tools including accuracy analysis, test experiments, corpus ingestion, and training data generation. Before that, he was the lead architect for the Domino Server OSGi team responsible for integrating the eXpeditor J2EE Web Container in Domino and building first class APIs for the developer community. He started with IBM in 1996, working on various globalization technologies and products including Domino Global Workbench (used to develop multilingual Notes/Domino NSF applications) and a multilingual Content Management system for the Websphere Application Server. David enjoys sharing his experience by speaking at conferences. You’ll find him at various events like the Unicode conference, Eclipsecon, and Lotusphere. He’s also passionate about building tools that help improve developer productivity and overall experience.
Hamburg Data Science Meetup - MLOps with a Feature Store
MLOps is a trend in machine learning (ML) engineering that unifies ML system development (Dev) and ML system operation (Ops). Some ML lifecycle frameworks, such as TensorFlow Extended, are based around end-to-end pipelines that start with raw data and end in production models. During this talk we will introduce the concept of a feature store as the missing piece of ML infrastructure that enables faster lower cost deployment of models. We will show how the Hopsworks Feature Store - factors monolithic end-to-end ML pipelines into feature and model training pipelines that can each run at different cadences. We will show examples of ingestion and training pipelines including hyperparameter optimization and model deployment.
Managing the Complete Machine Learning Lifecycle with MLflow
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models.
To solve for these challenges, Databricks unveiled last year MLflow, an open source project that aims at simplifying the entire ML lifecycle. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
In the past year, the MLflow community has grown quickly: over 120 contributors from over 40 companies have contributed code to the project, and over 200 companies are using MLflow.
In this tutorial, we will show you how using MLflow can help you:
Keep track of experiments runs and results across frameworks.
Execute projects remotely on to a Databricks cluster, and quickly reproduce your runs.
Quickly productionize models using Databricks production jobs, Docker containers, Azure ML, or Amazon SageMaker.
We will demo the building blocks of MLflow as well as the most recent additions since the 1.0 release.
What you will learn:
Understand the three main components of open source MLflow (MLflow Tracking, MLflow Projects, MLflow Models) and how each help address challenges of the ML lifecycle.
How to use MLflow Tracking to record and query experiments: code, data, config, and results.
How to use MLflow Projects packaging format to reproduce runs on any platform.
How to use MLflow Models general format to send models to diverse deployment tools.
Prerequisites:
A fully-charged laptop (8-16GB memory) with Chrome or Firefox
Python 3 and pip pre-installed
Pre-Register for a Databricks Standard Trial
Basic knowledge of Python programming language
Basic understanding of Machine Learning Concepts
Spark ML Pipeline serving
My talk at Data Science Labs conference in Odessa.
Training a model in Apache Spark while having it automatically available for real-time serving is an essential feature for end-to-end solutions.
There is an option to export the model into PMML and then import it into a separated scoring engine. The idea of interoperability is great but it has multiple challenges, such as code duplication, limited extensibility, inconsistency, extra moving parts. In this talk we discussed an alternative solution that does not introduce custom model formats and new standards, not based on export/import workflow and shares Apache Spark API.
AI Modernization at AT&T and the Application to Fraud with Databricks
AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache Spark has rapidly become a key tool for data scientists to explore, understand and transform massive datasets and to build and train advanced machine learning models. The question then becomes, how do I deploy these model to a production environment? How do I embed what I have learned into customer facing data applications?
In this webinar, we will discuss best practices from Databricks on
how our customers productionize machine learning models
do a deep dive with actual customer case studies,
show live tutorials of a few example architectures and code in Python, Scala, Java and SQL.
Hopsworks MLOps World talk june 21
Breaking the end-to-end ML monolithic pipeline with a Feature store.
Erin LeDell, H2O.ai - Scalable Automatic Machine Learning - H2O World San Fra...
This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/ndUtKRzVUCo
In this presentation, Erin LeDell (Chief Machine Learning Scientist, H2O.ai), will provide an overview of the field of "Automatic Machine Learning" and introduce the new AutoML functionality in H2O. H2O's AutoML provides an easy-to-use interface which automates the process of training a large, comprehensive selection of candidate models and a stacked ensemble model which, in most cases, will be the top performing model in the AutoML Leaderboard.
Bio: Erin is the Chief Machine Learning Scientist at H2O.ai. Erin has a Ph.D. in Biostatistics with a Designated Emphasis in Computational Science and Engineering from University of California, Berkeley. Her research focuses on automatic machine learning, ensemble machine learning and statistical computing. She also holds a B.S. and M.A. in Mathematics.
Before joining H2O.ai, she was the Principal Data Scientist at Wise.io (acquired by GE Digital in 2016) and Marvin Mobile Security (acquired by Veracode in 2012), and the founder of DataScientific, Inc.
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...
Open-source technologies allow developers to build microservices framework to build myriad real-time applications. One such application is building the real-time model scoring. In this session,
we will showcase how to architect a microservice framework, in particular how to use it to build a low-latency, real-time model scoring system. At the core of the architecture lies Apache Spark’s Structured
Streaming capability to deliver low-latency predictions coupled with Docker and Flask as additional open source tools for model service. In this session, you will walk away with:
* Knowledge of enterprise-grade model as a service
* Streaming architecture design principles enabling real-time machine learning
* Key concepts and building blocks for real-time model scoring
* Real-time and production use cases across industries, such as IIOT, predictive maintenance, fraud detection, sepsis etc.
Introducing apache prediction io (incubating) (bay area spark meetup at sales...
PredictionIO cofounder and creator Donald Szeto presents what, why and how of Apache PredictionIO as a Machine Learning framework running on top of Apache Spark.
AWS Machine Learning & Google Cloud Machine Learning
A comparison of the AI services provided by AWS and Google.
Multi runtime serving pipelines for machine learning
The talk I gave at Scale By The Bay.
Deploying, Serving and monitoring machine learning models built with different ML frameworks in production. Envoy proxy powered serving mesh. TensorFlow, Spark ML, Scikit-learn and custom functions on CPU and GPU.
The Killer Feature Store: Orchestrating Spark ML Pipelines and MLflow for Pro...
The ‘feature store’ is an emerging concept in data architecture that is motivated by the challenge of productionizing ML applications. The rapid iteration in experimental, data driven research applications creates new challenges for data management and application deployment.
Hopsworks in the cloud Berlin Buzzwords 2019
This talk, given at Berlin Buzzwords 2019, describes the recent progress in making Hopsworks a cloud-native platform, with HA data-center support added for HopsFS.
Hopsworks at Google AI Huddle, Sunnyvale
Hopsworks is a platform for designing and operating End to End Machine Learning using PySpark and TensorFlow/PyTorch. Early access is now available on GCP. Hopsworks includes the industry's first Feature Store. Hopsworks is open-source.
More Related Content
What's hot
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
A Feature Store enables machine learning (ML) features to be registered, discovered, and used as part of ML pipelines, thus making it easier to transform and validate the training data that is fed into machine learning systems. Feature stores can also enable consistent engineering of features between training and inference, but to do so, they need a common data processing platform. The first Feature Stores, developed at hyperscale AI companies such as Uber, Airbnb, and Facebook, enabled feature engineering using domain specific languages, providing abstractions tailored to the companies’ feature engineering domains. However, a general purpose Feature Store needs a general purpose feature engineering, feature selection, and feature transformation platform.
In this talk, we describe how we built a general purpose, open-source Feature Store for ML around dataframes and Apache Spark. We will demonstrate how data engineers can transform and engineers features from backend databases and data lakes, while data scientists can use PySpark to select and transform features into train/test data in a file format of choice (.tfrecords, .npy, .petastorm, etc) on a file system of choice (S3, HDFS). Finally, we will show how the Feature Store enables end-to-end ML pipelines to be factored into feature engineering and data science stages that each can run at different cadences.
Bio:
Fabio Buso is the head of engineering at Logical Clocks AB, where he leads the Feature Store development. Fabio holds a master's degree in cloud computing and services with a focus on data intensive applications, awarded by a joint program between KTH Stockholm and TU Berlin.
Topics: feature store, MLOps.
Kim Hammar - Feature Store: the missing data layer in ML pipelines? - HopsML ...
Presentation at HopsML meetup in Stockholm. Talking about Hopsworks feature store. January 29, 2019, Stockholm.
Ml ops and the feature store with hopsworks, DC Data Science Meetup
Ml ops and the feature store with hopsworks, DC Data Science Meetup
END-TO-END MACHINE LEARNING STACK
Autodeploy a complete end-to-end machine learning pipeline on Kubernetes using tools like Spark, TensorFlow, HDFS, etc. - it requires a running Kubernetes (K8s) cluster in the cloud or on-premise.
Machine Learning with Apache Spark
Presented by David Taieb, Architect, IBM Cloud Data Services
Along with Spark Streaming, Spark SQL and GraphX, MLLib is one of the four key architectural components of Spark. It provides easy-to-use (even for beginners), powerful Machine Learning APIs that are designed to work in parallel using Spark RDDs. In this session, we’ll introduce the different algorithms available in MLLib, e.g. supervised learning with classification (binary and multi class) and regression but also unsupervised learning with clustering (K-means) and recommendation systems. We’ll conclude the presentation with a deep dive on a sample machine learning application built with Spark MLLib that predicts whether a scheduled flight will be delayed or not. This application trains a model using data from real flight information. The labeled flight data is combined with weather data from the “Insight for Weather” service available on IBM Bluemix Cloud Platform to form the training, test and blind data. Even if you are not a black belt in machine learning, you will learn in this session how to leverage powerful Machine Learning algorithms available in Spark to build interesting predictive and prescriptive applications.
About the Speaker: For the last 4 years, David has been the lead architect for the Watson Core UI & Tooling team based in Littleton, Massachusetts. During that time, he led the design and development of a Unified Tooling Platform to support all the Watson Tools including accuracy analysis, test experiments, corpus ingestion, and training data generation. Before that, he was the lead architect for the Domino Server OSGi team responsible for integrating the eXpeditor J2EE Web Container in Domino and building first class APIs for the developer community. He started with IBM in 1996, working on various globalization technologies and products including Domino Global Workbench (used to develop multilingual Notes/Domino NSF applications) and a multilingual Content Management system for the Websphere Application Server. David enjoys sharing his experience by speaking at conferences. You’ll find him at various events like the Unicode conference, Eclipsecon, and Lotusphere. He’s also passionate about building tools that help improve developer productivity and overall experience.
Hamburg Data Science Meetup - MLOps with a Feature Store
MLOps is a trend in machine learning (ML) engineering that unifies ML system development (Dev) and ML system operation (Ops). Some ML lifecycle frameworks, such as TensorFlow Extended, are based around end-to-end pipelines that start with raw data and end in production models. During this talk we will introduce the concept of a feature store as the missing piece of ML infrastructure that enables faster lower cost deployment of models. We will show how the Hopsworks Feature Store - factors monolithic end-to-end ML pipelines into feature and model training pipelines that can each run at different cadences. We will show examples of ingestion and training pipelines including hyperparameter optimization and model deployment.
Managing the Complete Machine Learning Lifecycle with MLflow
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models.
To solve for these challenges, Databricks unveiled last year MLflow, an open source project that aims at simplifying the entire ML lifecycle. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
In the past year, the MLflow community has grown quickly: over 120 contributors from over 40 companies have contributed code to the project, and over 200 companies are using MLflow.
In this tutorial, we will show you how using MLflow can help you:
Keep track of experiments runs and results across frameworks.
Execute projects remotely on to a Databricks cluster, and quickly reproduce your runs.
Quickly productionize models using Databricks production jobs, Docker containers, Azure ML, or Amazon SageMaker.
We will demo the building blocks of MLflow as well as the most recent additions since the 1.0 release.
What you will learn:
Understand the three main components of open source MLflow (MLflow Tracking, MLflow Projects, MLflow Models) and how each help address challenges of the ML lifecycle.
How to use MLflow Tracking to record and query experiments: code, data, config, and results.
How to use MLflow Projects packaging format to reproduce runs on any platform.
How to use MLflow Models general format to send models to diverse deployment tools.
Prerequisites:
A fully-charged laptop (8-16GB memory) with Chrome or Firefox
Python 3 and pip pre-installed
Pre-Register for a Databricks Standard Trial
Basic knowledge of Python programming language
Basic understanding of Machine Learning Concepts
Spark ML Pipeline serving
My talk at Data Science Labs conference in Odessa.
Training a model in Apache Spark while having it automatically available for real-time serving is an essential feature for end-to-end solutions.
There is an option to export the model into PMML and then import it into a separated scoring engine. The idea of interoperability is great but it has multiple challenges, such as code duplication, limited extensibility, inconsistency, extra moving parts. In this talk we discussed an alternative solution that does not introduce custom model formats and new standards, not based on export/import workflow and shares Apache Spark API.
AI Modernization at AT&T and the Application to Fraud with Databricks
AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache Spark has rapidly become a key tool for data scientists to explore, understand and transform massive datasets and to build and train advanced machine learning models. The question then becomes, how do I deploy these model to a production environment? How do I embed what I have learned into customer facing data applications?
In this webinar, we will discuss best practices from Databricks on
how our customers productionize machine learning models
do a deep dive with actual customer case studies,
show live tutorials of a few example architectures and code in Python, Scala, Java and SQL.
Hopsworks MLOps World talk june 21
Breaking the end-to-end ML monolithic pipeline with a Feature store.
Erin LeDell, H2O.ai - Scalable Automatic Machine Learning - H2O World San Fra...
This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/ndUtKRzVUCo
In this presentation, Erin LeDell (Chief Machine Learning Scientist, H2O.ai), will provide an overview of the field of "Automatic Machine Learning" and introduce the new AutoML functionality in H2O. H2O's AutoML provides an easy-to-use interface which automates the process of training a large, comprehensive selection of candidate models and a stacked ensemble model which, in most cases, will be the top performing model in the AutoML Leaderboard.
Bio: Erin is the Chief Machine Learning Scientist at H2O.ai. Erin has a Ph.D. in Biostatistics with a Designated Emphasis in Computational Science and Engineering from University of California, Berkeley. Her research focuses on automatic machine learning, ensemble machine learning and statistical computing. She also holds a B.S. and M.A. in Mathematics.
Before joining H2O.ai, she was the Principal Data Scientist at Wise.io (acquired by GE Digital in 2016) and Marvin Mobile Security (acquired by Veracode in 2012), and the founder of DataScientific, Inc.
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...
Open-source technologies allow developers to build microservices framework to build myriad real-time applications. One such application is building the real-time model scoring. In this session,
we will showcase how to architect a microservice framework, in particular how to use it to build a low-latency, real-time model scoring system. At the core of the architecture lies Apache Spark’s Structured
Streaming capability to deliver low-latency predictions coupled with Docker and Flask as additional open source tools for model service. In this session, you will walk away with:
* Knowledge of enterprise-grade model as a service
* Streaming architecture design principles enabling real-time machine learning
* Key concepts and building blocks for real-time model scoring
* Real-time and production use cases across industries, such as IIOT, predictive maintenance, fraud detection, sepsis etc.
Introducing apache prediction io (incubating) (bay area spark meetup at sales...
PredictionIO cofounder and creator Donald Szeto presents what, why and how of Apache PredictionIO as a Machine Learning framework running on top of Apache Spark.
AWS Machine Learning & Google Cloud Machine Learning
A comparison of the AI services provided by AWS and Google.
Multi runtime serving pipelines for machine learning
The talk I gave at Scale By The Bay.
Deploying, Serving and monitoring machine learning models built with different ML frameworks in production. Envoy proxy powered serving mesh. TensorFlow, Spark ML, Scikit-learn and custom functions on CPU and GPU.
The Killer Feature Store: Orchestrating Spark ML Pipelines and MLflow for Pro...
The ‘feature store’ is an emerging concept in data architecture that is motivated by the challenge of productionizing ML applications. The rapid iteration in experimental, data driven research applications creates new challenges for data management and application deployment.
What's hot (20)
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
MLOps with a Feature Store: Filling the Gap in ML Infrastructure
Kim Hammar - Feature Store: the missing data layer in ML pipelines? - HopsML ...
Kim Hammar - Feature Store: the missing data layer in ML pipelines? - HopsML ...
Ml ops and the feature store with hopsworks, DC Data Science Meetup
Ml ops and the feature store with hopsworks, DC Data Science Meetup
Hamburg Data Science Meetup - MLOps with a Feature Store
Hamburg Data Science Meetup - MLOps with a Feature Store
Managing the Complete Machine Learning Lifecycle with MLflow
Managing the Complete Machine Learning Lifecycle with MLflow
AI Modernization at AT&T and the Application to Fraud with Databricks
AI Modernization at AT&T and the Application to Fraud with Databricks
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Erin LeDell, H2O.ai - Scalable Automatic Machine Learning - H2O World San Fra...
Erin LeDell, H2O.ai - Scalable Automatic Machine Learning - H2O World San Fra...
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...
Introducing apache prediction io (incubating) (bay area spark meetup at sales...
Introducing apache prediction io (incubating) (bay area spark meetup at sales...
AWS Machine Learning & Google Cloud Machine Learning
AWS Machine Learning & Google Cloud Machine Learning
Multi runtime serving pipelines for machine learning
Multi runtime serving pipelines for machine learning
The Killer Feature Store: Orchestrating Spark ML Pipelines and MLflow for Pro...
The Killer Feature Store: Orchestrating Spark ML Pipelines and MLflow for Pro...
Similar to Hopsworks - The Platform for Data-Intensive AI
Hopsworks in the cloud Berlin Buzzwords 2019
This talk, given at Berlin Buzzwords 2019, describes the recent progress in making Hopsworks a cloud-native platform, with HA data-center support added for HopsFS.
Hopsworks at Google AI Huddle, Sunnyvale
Hopsworks is a platform for designing and operating End to End Machine Learning using PySpark and TensorFlow/PyTorch. Early access is now available on GCP. Hopsworks includes the industry's first Feature Store. Hopsworks is open-source.
Hopsworks - Self-Service Spark/Flink/Kafka/Hadoop
ApacheCon Big Data Europe 2016 talk. Hopsworks with secure Spark/Flink/Kafka as a service.
ExtremeEarth: Hopsworks, a data-intensive AI platform for Deep Learning with ...
ExtremeEarth: Hopsworks, a data-intensive AI platform for Deep Learning with ...Big Data Value Association
The main goal of the session is to showcase approaches that greatly simplify the work of a data analyst when performing data analytics, or when employing machine learning algorithms, over Big Data. The session will include presentations on
(a) How data analytics workflows can be easily and graphically composed, and then optimized for execution,
(b) How raw data with great variety can be easily queried using SQL interfaces, and
(c) How complex machine learning operations can be performed efficiently in distributed settings.
After these presentations, the speakers will participate in a discussion with the audience, in order to discuss further tools that could make the work of a data analyst more simple.Data Science und Machine Learning im Kubernetes-Ökosystem
Wer professionell Datenprodukte bereitstellen möchte, benötigt Plattformen für Data Science, Machine Learning und AI, die Nachvollziehbarkeit, Reproduzierbarkeit, Flexibilität und Skalierbarkeit gewährleisten. Dazu werden skalierbare Storage- und Compute-Kapazitäten genauso benötigt wie eine Workflowsteuerung für ML-Pipelines, ein Analysefrontend oder ein Metadatenmanagement.
Wir zeigen, wie auf Basis von Kubernetes eine solche Plattformen aufgebaut werden kann, die sich durch CI/CD-Pipelines flexibel konfigurieren lässt und verschiedene ML-Frameworks und Programmiersprachen unterstützt.
Vorkenntnisse
Grundlegendes Verständnis von Daten, Data-Science-Prozessen und Container-Plattformen ist hilfreich.
Lernziele
Die Zuhörer haben einen Überblick über die Anforderungen an Data-Science-Plattformen und wie eine Architektur mit Kubernetes aussehen kann.
Event: data2day, 26.09.2018
Speaker: Dr. Stefan Igel, Hans-Peter Zorn
Mehr Tech-Vorträge: https://www.inovex.de/de/content-pool/vortraege/
Mehr Tech-Artikel: https://www.inovex.de/blog/
The other Apache Technologies your Big Data solution needs
An overview of the various Apache Technologies to help you build your own Big Data solution
Apache-Flink-What-How-Why-Who-Where-by-Slim-Baltagi
This introductory level talk is about Apache Flink: a multi-purpose Big Data analytics framework leading a movement towards the unification of batch and stream processing in the open source.
With the many technical innovations it brings along with its unique vision and philosophy, it is considered the 4 G (4th Generation) of Big Data Analytics frameworks providing the only hybrid (Real-Time Streaming + Batch) open source distributed data processing engine supporting many use cases: batch, streaming, relational queries, machine learning and graph processing.
In this talk, you will learn about:
1. What is Apache Flink stack and how it fits into the Big Data ecosystem?
2. How Apache Flink integrates with Hadoop and other open source tools for data input and output as well as deployment?
3. Why Apache Flink is an alternative to Apache Hadoop MapReduce, Apache Storm and Apache Spark.
4. Who is using Apache Flink?
5. Where to learn more about Apache Flink?
ApacheCon 2021 Apache Deep Learning 302
ApacheCon 2021 Apache Deep Learning 302
Tuesday 18:00 UTC
Apache Deep Learning 302
Timothy Spann
This talk will discuss and show examples of using Apache Hadoop, Apache Kudu, Apache Flink, Apache Hive, Apache MXNet, Apache OpenNLP, Apache NiFi and Apache Spark for deep learning applications. This is the follow up to previous talks on Apache Deep Learning 101 and 201 and 301 at ApacheCon, Dataworks Summit, Strata and other events. As part of this talk, the presenter will walk through using Apache MXNet Pre-Built Models, integrating new open source Deep Learning libraries with Python and Java, as well as running real-time AI streams from edge devices to servers utilizing Apache NiFi and Apache NiFi - MiNiFi. This talk is geared towards Data Engineers interested in the basics of architecting Deep Learning pipelines with open source Apache tools in a Big Data environment. The presenter will also walk through source code examples available in github and run the code live on Apache NiFi and Apache Flink clusters.
Tim Spann is a Developer Advocate @ StreamNative where he works with Apache NiFi, Apache Pulsar, Apache Flink, Apache MXNet, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Principal Field Engineer at Cloudera, a senior solutions architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science.
* https://github.com/tspannhw/ApacheDeepLearning302/
* https://github.com/tspannhw/nifi-djl-processor
* https://github.com/tspannhw/nifi-djlsentimentanalysis-processor
* https://github.com/tspannhw/nifi-djlqa-processor
* https://www.linkedin.com/pulse/2021-schedule-tim-spann/
End-to-End ML pipelines with Beam, Flink, TensorFlow and Hopsworks.
Apache Beam meetup Stockholm Ju;y 2019
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
In this deck from the Stanford HPC Conference, DK Panda from Ohio State University presents: Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Processing.
"This talk will provide an overview of challenges in accelerating Hadoop, Spark and Memcached on modern HPC clusters. An overview of RDMA-based designs for Hadoop (HDFS, MapReduce, RPC and HBase), Spark, Memcached, Swift, and Kafka using native RDMA support for InfiniBand and RoCE will be presented. Enhanced designs for these components to exploit NVM-based in-memory technology and parallel file systems (such as Lustre) will also be presented. Benefits of these designs on various cluster configurations using the publicly available RDMA-enabled packages from the OSU HiBD project (http://hibd.cse.ohio-state.edu) will be shown."
Watch the video: https://youtu.be/iLTYkTandEA
Learn more: http://web.cse.ohio-state.edu/~panda.2/
and
http://hpcadvisorycouncil.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...DataWorks Summit/Hadoop Summit
Accelerating Apache Hadoop through High-Performance Networking and I/O TechnologiesLarge-Scale Data Science in Apache Spark 2.0
Data science is one of the only fields where scalability can lead to fundamentally better results. Scalability allows users to train models on more data or to experiment with more types of models, both of which result in better models. It is no accident that the organizations most successful with AI have been those with huge distributed computing resources. In this talk, Matei Zaharia will describe how Apache Spark is democratizing large-scale data science to make it easier for more organizations to build high-quality data and AI products. Matei Zaharia will talk about the new structured APIs in Spark 2.0 that enable more optimization underneath familia programming interfaces, as well as libraries to scale up deep learning or traditional machine learning libraries on Apache Spark.
Speaker: Matei Zaharia
Big Data , Big Problem?
This presentation has been prepared based on the first chapter of "Scalable Big Data Architecture" written by "Bahaaldine Azarmi"
Big Data with hadoop, Spark and BigQuery (Google cloud next Extended 2017 Kar...
Google Next Extended (https://cloudnext.withgoogle.com/) is an annual Google event focusing on Google cloud technologies. This presentation is from tech talk held in Google Next Extended 2017 Karachi event
Scaling Data Science on Big Data
Data science holds tremendous potential for organizations to uncover new insights and drivers of revenue and profitability. Big Data has brought the promise of doing data science at scale to enterprises, however this promise also comes with challenges for data scientists to continuously learn and collaborate. Data Scientists have many tools at their disposal such as notebooks like Juypter and Apache Zeppelin & IDEs such as RStudio with languages like R, Python, Scala and frameworks like Apache Spark. Given all the choices how do you best collaborate to build your model and then work through the development lifecycle to deploy it from test into production ?
In this session learn the attributes of a modern data science platform that empowers data scientists to build models using all the data in their data lake and foster continuous learning and collaboration. We will show a demo of DSX with HDP with the focus on integration, security and model deployment and management.
Speakers:
Sriram Srinivasan, Senior Technical Staff Member, Analytics Platform Architect, IBM
Vikram Murali, Program Director, Data Science and Machine Learning, IBM
Open Source Data Orchestration for AI, Big Data, and Cloud
Beijing Meetup
06/22/2019
Haoyuan Li
For more Alluxio events: https://www.alluxio.io/events/
Similar to Hopsworks - The Platform for Data-Intensive AI (20)
ExtremeEarth: Hopsworks, a data-intensive AI platform for Deep Learning with ...
ExtremeEarth: Hopsworks, a data-intensive AI platform for Deep Learning with ...
Data Science und Machine Learning im Kubernetes-Ökosystem
Data Science und Machine Learning im Kubernetes-Ökosystem
The other Apache Technologies your Big Data solution needs
The other Apache Technologies your Big Data solution needs
Big Data, Ingeniería de datos, y Data Lakes en AWS
Big Data, Ingeniería de datos, y Data Lakes en AWS
Apache-Flink-What-How-Why-Who-Where-by-Slim-Baltagi
Apache-Flink-What-How-Why-Who-Where-by-Slim-Baltagi
End-to-End ML pipelines with Beam, Flink, TensorFlow and Hopsworks.
End-to-End ML pipelines with Beam, Flink, TensorFlow and Hopsworks.
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
Big Data Meets HPC - Exploiting HPC Technologies for Accelerating Big Data Pr...
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Accelerating Apache Hadoop through High-Performance Networking and I/O Techno...
Big Data with hadoop, Spark and BigQuery (Google cloud next Extended 2017 Kar...
Big Data with hadoop, Spark and BigQuery (Google cloud next Extended 2017 Kar...
Open Source Data Orchestration for AI, Big Data, and Cloud
Open Source Data Orchestration for AI, Big Data, and Cloud
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
Big Data Essentials meetup @ IBM Ljubljana 23.06.2015
More from QAware GmbH
Mit ChatGPT Dinosaurier besiegen - Möglichkeiten und Grenzen von LLM für die ...
Generative AI for Developers, 27.05.2024, Bonn (Martin Binder, Lead IT Consultant bei QAware)
Die kritische Infrastruktur wird nach wie vor von Dinosauriersystemen aus dem letzten Jahrtausend getragen. Die Ablösung dieser Altsysteme ist ein drängendes Problem der Digitalisierung in Deutschland. In meinem Projekt musste ich in wenigen Wochen ein 30 Jahre altes System auf Basis von IBM iSeries (AS/400) mit RPG-Code analysieren. Überraschenderweise kannte ChatGPT die RGP-Referenz von IBM. Mithilfe von ChatGPT konnte ich mich schnell einarbeiten und so ein brauchbares Arbeitsverständnis des Systems erreichen. Aber wo Licht ist, ist auch Schatten: ChatGPT hat kein Kontextverständnis, ist unvollständig und neigt zu Fehlschlüssen. Als Vorgeschmack hier der Powermove: ChatGPT nach jeder Antwort grundsätzlich danach fragen, was es falsch gemacht hat. Du wirst überrascht sein!
50 Shades of K8s Autoscaling #JavaLand24.pdf
JavaLand 2024, April 2024, Mario-Leander Reimer
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Abstract:
Elasticity and unlimited scalability are the holy grail for any cloud-native application. Now you might think: “That’s easy!!! I just run my containers on a Kubernetes cluster and I am done.” But is it really that simple?
Turns out it’s not! Your application as well as your cloud infrastructure and K8s cluster need to address and support these non-functional requirements.
This session will have a detailed and also practical look at the different ways of autoscaling in Kubernetes. We will give an overview of the technical foundations and prerequisites, and then showcase several frameworks and technologies that can be used to flexibly autoscale your cluster and your cloud-native workloads.
Make Agile Great - PM-Erfahrungen aus zwei virtuellen internationalen SAFe-Pr...
GPM Regionalgruppe Chemnitz (Patrick Albert)
Wegen ihres Umfangs und Komplexität sind größere SAFe-Programme bereits in Präsenz hinsichtlich ihres Managements und ihrer Steuerung anspruchsvoll. Aufgrund von COVID19 jedoch war eine Verlegung in den virtuellen Raum im beschriebenen Praxisfall unausweichlich. Das Management hatte hierbei sicherzustellen, dass die Programmziele trotz des verminderten Kontaktes allen beteiligten Teams dauerhaft klar und präsent sind und dass die in den Teams umgesetzten Funktionen außerdem den genannten Programmzielen dienen.
Besonders wichtig ist dieses Alignment im Rahmen der regelmäßigen PI-Plannings, in welchen alle Teams gleichzeitig die jeweils kommenden Iterationen planen und dabei auch teamübergreifende Abhängigkeiten zuverlässig berücksichtigen müssen.
Es werden Erfolgsfaktoren für den virtuellen Einsatz von SAFe herausgearbeitet und beleuchtet.
Fully-managed Cloud-native Databases: The path to indefinite scale @ CNN Mainz
When it comes to the question: "Where do we actually store our application data?", we are spoilt for choice, especially when it comes to the major cloud providers.
The simple and often completely valid answer is still the classic relational database! It is very suitable for many areas of application, as the technology is tried and tested and can cover a very broad spectrum. It is therefore not surprising that all major cloud providers offer this as a "managed service".
For some years now, however, there have also been so-called cloud-native databases that have been specially developed for the requirements of the cloud. The big promise: "Infinite scalability"
In a large customer project, we have been using such a database productively for over 4 years with Azure CosmosDB. The presentation will deal with the following questions, among others
What does "upscalability" mean in practice ?
What do you have to pay attention to when designing?
What are the actual limits?
What other special features do I get?
When do I need a cloud-native database?
But that's not all! We also look beyond Azure to the other two major cloud providers: AWS and Google Cloud. With DynamoDB and Datastore/Firestore, they have similar products on offer.
Down the Ivory Tower towards Agile Architecture
iSAQB Software Architecture Gathering – Digital 2023, November 2023
Architecture for agile projects must be defined and described differently, as well as continuously developed and evolved. Not all decisions are made at once, nor are they all known right from the start of the project. This sessions presents various useful and lightweight methods, tools and team topologies that can be applied in (large) agile projects to avoid uncontrolled growth and architectural erosion but without acting from the ivory tower and thus suffocating team autonomy.
"Mixed" Scrum-Teams – Die richtige Mischung macht's!
IT-Tage 2023, Frankfurt am Main (Patrick Albert)
Die Rollen und insbesondere deren Zuständigkeiten sind in Scrum recht klar geregelt: Der Product Owner sorgt für das "Was", der Scrum Master für das "Wie" und die Developer für die eigentliche Umsetzung. Solange es sich dabei um ein internes Projekt mit einem komplett internen Team handelt, sind damit bereits die zentralen Fragen beantwortet. Ein wenig differenzierter zu betrachten sind allerdings Teams mit mehreren Parteien wie etwa beim Einsatz von Dienstleistern. Muss etwa zwingend der Kunde den Product Owner stellen oder kann dieser auch auf der Seite des Dienstleisters stehen? Falls alle Developer vom gleichen Dienstleister bereitgestellt werden, würde diese Konstellation sicherlich einige Kommunikationswege verkürzen. Allerdings hat ein Product Owner auf der Seite eines Dienstleisters sicher nicht die gleichen Verbindungen zu den Stakeholdern (Nutzer, Geldgeber, ...) des Produkts wie ein interner Product Owner – wie also könnte er ihre Anforderungen dann gut vor dem Team vertreten? Ähnliche Fragen stellen sich auch für den Scrum Master und das Development-Team. In den meisten Fällen gibt es für verschiedene Konstellationen jeweils Vor- und Nachteile – und zwar sowohl für den Kunden als auch für den Dienstleister. Wichtig bei der Entscheidung für eine dieser Konstellationen ist außerdem die Art des Projekts, der Kreis der Stakeholder, das zur Verfügung stehende Budget, der Zeitrahmen und noch einiges mehr.
Dieser Vortrag beleuchtet verschiedene dieser Varianten und zeigt Vorteile, Nachteile und Risiken auf.
Make Developers Fly: Principles for Platform Engineering
Make Developers Fly – Helping developers to build better applications
Cloud Native Night, Mainz, November 2023, Alex Krause
Platform Engineering is the next stage of DevOps and accelerates software developers even more to build applications faster and bring products rapidly to the customers. In this meetup, we show you the key principles of platform engineering, as we experienced them in our projects, and additionally show you a better way to manage your internal software platforms.
PRINCIPLES FOR PLATFORM ENGINEERING, Alex Krause
How do we help our developers to fly instead of crashing miserablely? The answer is Platform Engineering, a discipline for building internal developer platforms (IDPs) to simplify software delivery for product teams. In this talk, you'll learn how Platform Engineering evolved from the DevOps movement and what principles and best practices make for a good implementation. Finally, we'll take a look at reference architectures that can support your platform.
Der Tod der Testpyramide? – Frontend-Testing mit Playwright
Codineers Rosenheim Meetup, 2. November 2023 (Dominik Haas, QAware)
Wer kennt es nicht: Langsame, instabile und wartungsintensive Tests von Frontends und die Schmerzen damit.
Aber in den letzten Jahren und Monaten hat sich hier viel getan und Playwright ist der vielversprechende neue Stern der Web-Automatisierung.
Ich werde in meinem Talk über das Testen von Frontends sprechen und speziell auf Playwright als spannendes Werkzeug eingehen.
Neben der praktischen Einführung möchte ich auch mit Euch über Best Practices sprechen und einen Versuch wagen die Testpyramide zu töten (oder sie zumindest etwas zu beschädigen).
Was kommt nach den SPAs
Sie kennen doch bestimmt das "Gesetz des Instruments": Wer als Werkzeug nur ein modernes Frontend-Framework hat, löst jedes Problem mit einer Single Page-Applikation. So oder so ähnlich, nur halt mit Hammer und Nagel lautet es, beschreibt jedoch ganz gut die aktuelle Situation der JavaScript-Welt. Auf nahezu jede Anforderung wird mit einer aufgeblähten, clientseitig gerenderten SPA geantwortet. Doch ist es schön langsam an der Zeit, dass wir uns fragen sollten: Ist das wirklich alles? Und die Antwort lautet ziemlich sicher "Nein". Doch genau diesem Thema widmen wir uns und werfen einen Blick auf die Alternativen und da gibt es viele.
Im React-Ökosystem findet aktuell ein kleiner Umbruch statt. Mit Server Side Rendering, Static Site Generation, Server Components und Frameworks wie Next verschiebt sich ein Teil der Arbeit in Richtung Server. Dieser Trend ist auch bei Vue, Svelte und Angular zu beobachten. Und genau das ist es, was die sogenannten Meta-Frameworks ausmacht. Uns als EntwicklerInnen gibt das mehr Flexibilität, um auf Anforderungen reagieren zu können. Sie müssen nicht mehr den kompletten Quellcode zum Client übertragen, haben bessere Caching-Möglichkeiten und auch die Suchmaschinen sind Ihnen dankbar.
Dieser Vortrag gibt Ihnen einen Überblick über die wichtigsten Features von Meta-Frameworks und wo und vor allem wie sie gewinnbringend eingesetzt werden können.
Cloud Migration mit KI: der Turbo
Digital Future Congress (DFC) 2023, September 2023, München, Andreas Zitzelsberger (Business Unit Director bei QAware)
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Alt-Systeme in die Cloud migrieren – mit sechs Leitplanken geht das effizient und sicher. Neu: Künstliche Intelligenz kann jetzt zum Turbo werden! Wir nutzen KI bei der Cloud-Migration und zeigen in diesem Vortrag, was geht und was nicht.
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
MedTech Stars, Webkonferenz, September 2023, Josef Adersberger (CEO bei QAware)
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Der Vortrag „Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die Seele verkaufen oder Himmel auf Erden?“ erkundet das wachsende Bedürfnis und die Herausforderungen, medizinische Anwendungen in die Cloud zu migrieren.
Bei den entsprechenden Anwendungen stellt laut einer Umfrage der Datenschutz für 82% der Anwendungen die größte Herausforderung dar, um den Schritt in die Cloud zu machen. Trotzdem wächst der Druck, diesen Schritt zu vollziehen, aufgrund des Innovations- und Digitalisierungsbedarfs, der durch gesetzliche Initiativen wie das Krankenhaus-Zukunftsgesetz, gestiegene Patientenerwartungen und den immer klareren medizinischen Impact der Digitalisierung angetrieben wird.
Die zentrale Frage, die in diesem Vortrag erörtert wird, ist, wie man hochregulierte Anwendungen schrittweise in die Cloud migrieren kann. Hierzu werden wir einen Ansatz vorstellen und anhand von exemplarischen Anwendungen demonstrieren, wie eine solche Migration erfolgreich durchgeführt werden kann. Abschließend fassen wir den Vortrag mit fünf leitenden Prinzipien zusammen, die bei der Migration von stark regulierten Anwendungen in die Cloud beachtet werden sollten. Diese Prinzipien bieten einen nützlichen Leitfaden für Unternehmen und Institutionen, die den Umzug ihrer Anwendungen in die Cloud erwägen, und sollen dazu beitragen, die Herausforderungen der Cloud-Migration in diesem hochregulierten Bereich erfolgreich zu bewältigen.
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
SAA 2023 | Software Architecture Alliance, September 2023, München, Mario-Leander Reimer (@LeanderReimer, CTO bei QAware)
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Die Cloud hat bereits heute einen größeren CO2-Fußabdruck als die Luftfahrtindustrie, mit steigender Digitalisierung und Cloudifizierung wird sich dieser Trend fortsetzen, wenn wir nichts dagegen unternehmen. Viele Kubernetes-basierte Installationen sind gemessen am eigentlich benötigten Ressourcen Bedarf stark überdimensioniert und tragen so unnötig zur globalen Erwärmung bei. Wie sieht die Energiebilanz Ihres Clusters und Workloads aus?
In diesem Vortrag zeigen wir Ansätze und Technologien, die dabei helfen K8s-Cluster grün(er) zu machen. Zunächst braucht es Transparenz: Wie sieht die Energiebilanz des Clusters und seiner Workloads aus? Erst danach lassen sich diese gezielt auf ihre Energiesparsamkeit hin optimieren. Und das ist gar nicht so schwer, also packen wir es an!
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.
SAA 2023 | Software Architecture Alliance, September 2023, München, Ildikó Tárkányi (Senior Software Engineer bei QAware)
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Der Traum von den unendlichen Weiten der Daten ist schnell ausgeträumt, wenn wir unsere APIs nicht im Griff haben. Dabei ist es egal, ob wir uns APIs zwischen unseren Services, zwischen Frontend und Backend oder zu Fremdsystemen anschauen - APIs sind ein fundamentaler Bestandteil unserer Softwarearchitektur und ohne Testautomatisierung geht gar nichts.
Schlecht geschriebene Integrationstests mit selbst erfundenen Testfällen für unsere APIs können richtig nerven: Wir rufen die APIs auf und hoffen, dass genau das zurückkommt, was wir erwarten.
Das geht auch anders: Wir werfen einen Blick auf andere Testansätze, bewerten, welche Aspekte diese jeweils testen und schauen zum Beispiel auf:
• Contract Based Testing
• Black Box Testing der OpenAPI Specs
• Trace-basiertes Testing
Kubernetes with Cilium in AWS - Experience Report!
Cloud Native Night, Munich, September 2023, Bernhard Schaidhammer
=== Please download slides if blurred! ===
Cilium is a powerful tool for network policies and also encryption between the Kubernetes nodes. Cilium hooks deep into Kubernetes in the network stack as an plugin and can even replace the AWS CNI Plugin. This talk will share our project experiences.
Topics involve:
- Network Policies
- Encryption
- Hubble (Observability)
- Installation
- CLI Usage (Hubble / Cilium)
50 Shades of K8s Autoscaling
Container Days 2023, September 2023, Hamburg, Mario-Leander Reimer (@LeanderReimer, CTO @QAware).
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Elasticity and unlimited scalability are the holy grail for any cloud-native application. Now you might think: “That’s easy!!! I just run my containers on a Kubernetes cluster and I am done.” But is it really that simple?
Turns out it’s not! Your application as well as your cloud infrastructure and K8s cluster need to address and support these non-functional requirements.
This session will have a detailed and also practical look at the different ways of autoscaling in Kubernetes. We will give an overview of the technical foundations and prerequisites, and then showcase several frameworks and technologies that can be used to flexibly autoscale your cluster and your cloud-native workloads.
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP
Heise DevSec 2023, September 2023, Karlsruhe, Mario-Leander Reimer (@LeanderReimer, CTO @QAware).
== Dokument bitte herunterladen, falls unscharf! Please download slides if blurred! ==
Continuous Delivery ist allgegenwärtig. Wirklich? Viele Teams straucheln immer noch dabei, regelmäßig gut getestete und vor allem sichere Software auszuliefern. Immer mit der gleichen, guten alten Ausrede: die nicht-funktionalen Tests seien zu aufwändig und zu teuer umzusetzen. Doch genau das Gegenteil ist der Fall!
In diesem Vortrag gehen wir kurz auf die aktuellen Bedrohungen und die Bedeutung früher und regelmäßiger Sicherheitstests von APIs ein. Anschließend zeigen wir, wie einfach es ist, diese Tests kontinuierlich und asynchron mit OWASP ZAP und Testkube gegen REST- und GraphQL-APIs direkt auf einem Kubernetes-Cluster auszuführen; immer dann wenn sich die API und der Service ändern.
Service Mesh Pain & Gain. Experiences from a client project.
Cloud Native Night, Mainz, September 2023, Markus Zimmermann
=== Please download slides if blurred! ===
The topic of service mesh is still present at every major DevOps conference and is the subject of controversial discussions.
A service mesh comes with the promise of implementing cross-functional requirements of microservices such as observability or secure communication without changes within the services. The operational effort and the integration of the services within the service mesh should also be problem-free.
But do the common service mesh implementations keep these promises? In a client project we have gained experience with Linkerd and can say: we did not achieve the benefits without pain. We want to share these experiences and what you can take away for your next project!
50 Shades of K8s Autoscaling
WeAreDevelopers World Congress 2023, July 2023, Mario-Leander Reimer
=== Please download slides if blurred! ===
Elasticity and unlimited scalability are the holy grail for any cloud-native application. Now you might think: “That’s easy!!! I just run my containers on a Kubernetes cluster and I am done.” But is it really that simple?
Turns out it’s not! Your application as well as your cloud infrastructure and K8s cluster need to address and support these non-functional requirements.
This session will have a detailed and also practical look at the different ways of autoscaling in Kubernetes.
We will give an overview of the technical foundations and prerequisites, and then showcase several frameworks and technologies that can be used to flexibly autoscale your cluster and your cloud-native workloads.
Blue turns green! Approaches and technologies for sustainable K8s clusters.
Kubernetes Community Days Munich 2023, Juli 2023, Mario-Leander Reimer
The cloud already has a larger CO2 footprint than the worldwide aviation industry. With increasing digitalization and cloudification this trend will continue if we don't do anything about it. Many Kubernetes installations are greatly oversized in terms of the resources actually required and thus contribute unnecessarily to global warming. What is the energy balance of your cluster and its workloads?
In this session we will discuss approaches and technologies that help to make K8s clusters green(er). First of all, transparency is needed: what is the energy balance of the cluster and its workloads? Only then we can start to optimize for better energy efficiency. And it's not that difficult, so let's get started!
Per Anhalter zu Cloud Nativen API Gateways
Mastering Kubernetes 2023, Juli 2023, Sonja Wegner
Gute APIs sind das Herzstück erfolgreicher digitaler Produkte und Cloud-nativer Anwendungen. Doch schlecht verwaltete APIs werden schnell zum Albtraum. Damit es kein böses Erwachen gibt, setzen wir auf API Gateways: Diese sind etabliert und bekannt und helfen uns bei der Verwaltung der APIs. Sie regeln unter anderem Traffic Management, Rollout-Szenarien, Versionierung, Zugriffskontrolle und Diagnostizierbarkeit.
In diesem Vortrag werden wir das Cloud-native API-Gateway-Ökosystem näher betrachten: Gloo, KrakenD, Kong, Envoy et al. Aber welches davon ist das Richtige für den Einsatz im nächsten Projekt? Lasst es uns herausfinden!
More from QAware GmbH (20)
Mit ChatGPT Dinosaurier besiegen - Möglichkeiten und Grenzen von LLM für die ...
Mit ChatGPT Dinosaurier besiegen - Möglichkeiten und Grenzen von LLM für die ...
Make Agile Great - PM-Erfahrungen aus zwei virtuellen internationalen SAFe-Pr...
Make Agile Great - PM-Erfahrungen aus zwei virtuellen internationalen SAFe-Pr...
Fully-managed Cloud-native Databases: The path to indefinite scale @ CNN Mainz
Fully-managed Cloud-native Databases: The path to indefinite scale @ CNN Mainz
"Mixed" Scrum-Teams – Die richtige Mischung macht's!
"Mixed" Scrum-Teams – Die richtige Mischung macht's!
Make Developers Fly: Principles for Platform Engineering
Make Developers Fly: Principles for Platform Engineering
Der Tod der Testpyramide? – Frontend-Testing mit Playwright
Der Tod der Testpyramide? – Frontend-Testing mit Playwright
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.
Kubernetes with Cilium in AWS - Experience Report!
Kubernetes with Cilium in AWS - Experience Report!
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP
Service Mesh Pain & Gain. Experiences from a client project.
Service Mesh Pain & Gain. Experiences from a client project.
Blue turns green! Approaches and technologies for sustainable K8s clusters.
Blue turns green! Approaches and technologies for sustainable K8s clusters.
Recently uploaded
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness in Kansas City, MO
To: Tom Tresser
From: Roger Warren
State of Artificial intelligence Report 2023
Artificial intelligence (AI) is a multidisciplinary field of science and engineering whose goal is to create intelligent machines.
We believe that AI will be a force multiplier on technological progress in our increasingly digital, data-driven world. This is because everything around us today, ranging from culture to consumer products, is a product of intelligence.
The State of AI Report is now in its sixth year. Consider this report as a compilation of the most interesting things we’ve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future.
We consider the following key dimensions in our report:
Research: Technology breakthroughs and their capabilities.
Industry: Areas of commercial application for AI and its business impact.
Politics: Regulation of AI, its economic implications and the evolving geopolitics of AI.
Safety: Identifying and mitigating catastrophic risks that highly-capable future AI systems could pose to us.
Predictions: What we believe will happen in the next 12 months and a 2022 performance review to keep us honest.
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
原版定制【微信:41543339】【(UofS毕业证书)萨省大学毕业证】【微信:41543339】成绩单、外壳、offer、留信学历认证（永久存档真实可查）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
→ 【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
原版定制【微信:41543339】【(BCU毕业证书)伯明翰城市大学毕业证】【微信:41543339】成绩单、外壳、offer、留信学历认证（永久存档真实可查）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
→ 【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
Analysis insight about a Flyball dog competition team's performance
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Global Situational Awareness of A.I. and where its headed
You can see the future first in San Francisco.
Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added to the boardroom plans. Behind the scenes, there’s a fierce scramble to secure every power contract still available for the rest of the decade, every voltage transformer that can possibly be procured. American big business is gearing up to pour trillions of dollars into a long-unseen mobilization of American industrial might. By the end of the decade, American electricity production will have grown tens of percent; from the shale fields of Pennsylvania to the solar farms of Nevada, hundreds of millions of GPUs will hum.
The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word. Along the way, national security forces not seen in half a century will be un-leashed, and before long, The Project will be on. If we’re lucky, we’ll be in an all-out race with the CCP; if we’re unlucky, an all-out war.
Everyone is now talking about AI, but few have the faintest glimmer of what is about to hit them. Nvidia analysts still think 2024 might be close to the peak. Mainstream pundits are stuck on the wilful blindness of “it’s just predicting the next word”. They see only hype and business-as-usual; at most they entertain another internet-scale technological change.
Before long, the world will wake up. But right now, there are perhaps a few hundred people, most of them in San Francisco and the AI labs, that have situational awareness. Through whatever peculiar forces of fate, I have found myself amongst them. A few years ago, these people were derided as crazy—but they trusted the trendlines, which allowed them to correctly predict the AI advances of the past few years. Whether these people are also right about the next few years remains to be seen. But these are very smart people—the smartest people I have ever met—and they are the ones building this technology. Perhaps they will be an odd footnote in history, or perhaps they will go down in history like Szilard and Oppenheimer and Teller. If they are seeing the future even close to correctly, we are in for a wild ride.
Let me tell you what we see.
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
原版定制【微信:41543339】【(Adelaide毕业证书)阿德莱德大学毕业证】【微信:41543339】成绩单、外壳、offer、留信学历认证（永久存档真实可查）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
→ 【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
The affect of service quality and online reviews on customer loyalty in the E...
Journal of Management and Creative Business.published book./
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
原版定制【Q微信:741003700】《(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书》【Q微信:741003700】成绩单 、雅思、外壳、留信学历认证永久存档查询，采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【Q微信741003700】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信741003700】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
UIUC毕业证offer【微信95270640】☀《伊利诺伊大学|厄巴纳-香槟分校毕业证购买》GoogleQ微信95270640《UIUC毕业证模板办理》加拿大文凭、本科、硕士、研究生学历都可以做,二、业务范围：
★、全套服务：毕业证、成绩单、化学专业毕业证书伪造《伊利诺伊大学|厄巴纳-香槟分校大学毕业证》Q微信95270640《UIUC学位证书购买》
(诚招代理)办理国外高校毕业证成绩单文凭学位证,真实使馆公证（留学回国人员证明）真实留信网认证国外学历学位认证雅思代考国外学校代申请名校保录开请假条改GPA改成绩ID卡
1.高仿业务:【本科硕士】毕业证,成绩单（GPA修改）,学历认证（教育部认证）,大学Offer,,ID,留信认证,使馆认证,雅思,语言证书等高仿类证书；
2.认证服务: 学历认证（教育部认证）,大使馆认证（回国人员证明）,留信认证（可查有编号证书）,大学保录取,雅思保分成绩单。
3.技术服务：钢印水印烫金激光防伪凹凸版设计印刷激凸温感光标底纹镭射速度快。
办理伊利诺伊大学|厄巴纳-香槟分校伊利诺伊大学|厄巴纳-香槟分校毕业证offer流程：
1客户提供办理信息：姓名生日专业学位毕业时间等（如信息不确定可以咨询顾问：我们有专业老师帮你查询）；
2开始安排制作毕业证成绩单电子图；
3毕业证成绩单电子版做好以后发送给您确认；
4毕业证成绩单电子版您确认信息无误之后安排制作成品；
5成品做好拍照或者视频给您确认；
6快递给客户（国内顺丰国外DHLUPS等快读邮寄）
-办理真实使馆公证（即留学回国人员证明）
-办理各国各大学文凭（世界名校一对一专业服务,可全程监控跟踪进度）
-全套服务：毕业证成绩单真实使馆公证真实教育部认证。让您回国发展信心十足！
（详情请加一下 文凭顾问+微信:95270640）欢迎咨询！的鬼地方父亲的家在高楼最底屋最下面很矮很黑是很不显眼的地下室父亲的家安在别人脚底下须绕过高楼旁边的垃圾堆下八个台阶才到父亲的家很狭小除了一张单人床和一张小方桌几乎没有多余的空间山娃一下子就联想起学校的男小便处山娃很想笑却怎么也笑不出来山娃很迷惑父亲的家除了一扇小铁门连窗户也没有墓穴一般阴森森有些骇人父亲的城也便成了山娃的城父亲的家也便成了山娃的家父亲让山娃呆在屋里做作业看电视最多只能在门口透透气间
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
原版定制【微信:41543339】【(Dalhousie毕业证书)达尔豪斯大学毕业证】【微信:41543339】成绩单、外壳、offer、留信学历认证（永久存档真实可查）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
→ 【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
一比一原版(Bradford毕业证书)布拉德福德大学毕业证如何办理
原版定制【微信:41543339】【(Bradford毕业证书)布拉德福德大学毕业证】【微信:41543339】成绩单、外壳、offer、留信学历认证（永久存档真实可查）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【我们承诺采用的是学校原版纸张（纸质、底色、纹路），我们拥有全套进口原装设备，特殊工艺都是采用不同机器制作，仿真度基本可以达到100%，所有工艺效果都可提前给客户展示，不满意可以根据客户要求进行调整，直到满意为止！】
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信41543339】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信41543339】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
留信网服务项目：
1、留学生专业人才库服务（留信分析）
2、国（境）学习人员提供就业推荐信服务
3、留学人员区块链存储服务
→ 【关于价格问题（保证一手价格）】
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：客户在留信官方认证查询网站查询到认证通过结果后付款，不成功不收费！
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
CBU毕业证offer【微信95270640】《卡普顿大学毕业证书》《QQ微信95270640》学位证书电子版：在线制作卡普顿大学毕业证成绩单GPA修改（制作CBU毕业证成绩单CBU文凭证书样本）、卡普顿大学毕业证书与成绩单样本图片、《CBU学历证书学位证书》、卡普顿大学毕业证案例毕业证书制作軟體、在线制作加拿大硕士学历证书真实可查.
如果您是以下情况，我们都能竭诚为您解决实际问题：【公司采用定金+余款的付款流程，以最大化保障您的利益，让您放心无忧】
1、在校期间，因各种原因未能顺利毕业，拿不到官方毕业证+微信95270640
2、面对父母的压力，希望尽快拿到卡普顿大学卡普顿大学毕业证成绩单；
3、不清楚流程以及材料该如何准备卡普顿大学卡普顿大学毕业证成绩单；
4、回国时间很长，忘记办理；
5、回国马上就要找工作，办给用人单位看；
6、企事业单位必须要求办理的；
面向美国乔治城大学毕业留学生提供以下服务:
【★卡普顿大学卡普顿大学毕业证成绩单毕业证、成绩单等全套材料，从防伪到印刷，从水印到钢印烫金，与学校100%相同】
【★真实使馆认证（留学人员回国证明），使馆存档可通过大使馆查询确认】
【★真实教育部认证，教育部存档，教育部留服网站可查】
【★真实留信认证，留信网入库存档，可查卡普顿大学卡普顿大学毕业证成绩单】
我们从事工作十余年的有着丰富经验的业务顾问，熟悉海外各国大学的学制及教育体系，并且以挂科生解决毕业材料不全问题为基础，为客户量身定制1对1方案，未能毕业的回国留学生成功搭建回国顺利发展所需的桥梁。我们一直努力以高品质的教育为起点，以诚信、专业、高效、创新作为一切的行动宗旨，始终把“诚信为主、质量为本、客户第一”作为我们全部工作的出发点和归宿点。同时为海内外留学生提供大学毕业证购买、补办成绩单及各类分数修改等服务；归国认证方面，提供《留信网入库》申请、《国外学历学位认证》申请以及真实学籍办理等服务，帮助众多莘莘学子实现了一个又一个梦想。
专业服务，请勿犹豫联系我
如果您真实毕业回国，对于学历认证无从下手，请联系我，我们免费帮您递交
诚招代理：本公司诚聘当地代理人员，如果你有业余时间，或者你有同学朋友需要，有兴趣就请联系我
你赢我赢，共创双赢
你做代理，可以帮助卡普顿大学同学朋友
你做代理，可以拯救卡普顿大学失足青年
你做代理，可以挽救卡普顿大学一个个人才
你做代理，你将是别人人生卡普顿大学的转折点
你做代理，可以改变自己，改变他人，给他人和自己一个机会道银边山娃摸索着扯了扯灯绳小屋顿时一片刺眼的亮瞅瞅床头的诺基亚山娃苦笑着摇了摇头连他自己都感到奇怪居然又睡到上午点半掐指算算随父亲进城已一个多星期了山娃几乎天天起得这么迟在乡下老家暑假五点多山娃就醒来在爷爷奶奶嘁嘁喳喳的忙碌声中一骨碌爬起把牛驱到后龙山再从莲塘里采回一蛇皮袋湿漉漉的莲蓬也才点多点半早就吃过早餐玩耍去了山娃的家在闽西山区依山傍水山清水秀门前潺潺流淌的蜿蜒小溪一直都是山娃和小伙伴们盛试
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Recently uploaded (20)
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdf
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Influence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business Plan
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Hopsworks - The Platform for Data-Intensive AI
- 1. Hopsworks The Platform for Data-Intensive AI Steffen Grohsschmiedt Head of Cloud steffen@logicalclocks.com @grohsschmiedt
- 2. Hopsworks Timeline “If you’re working with big data and Hadoop, this one paper could repay your investment in the Morning Paper many times over.... HopsFS is a huge win.” - Adrian Colyer, The Morning Paper World’s fastest Hadoop Published at USENIX FAST with Oracle and Spotify World’s First #1 GPUs-as-a-Resource support in the Hopsworks platform World’s First #3 Open Source Feature Store for Machine Learning World’s First #2 Distributed File System to store small files in metadata on NVMe disks Winner of IEEE.. .. Scale Challenge 2017 with HopsFS - 1.2m ops/sec 2017 2018 2019 World’s most scalable Filesystem with Multi Data Center Availability
- 3. Example workflow in Hopsworks at Scale 1. Insert 1m images (<100kb) in seconds 2. Train a DNN classifier using 100s of GPUs 3. Run a Spark job to identify all objects in the 1m images and add the image annotations (JSON) as extended metadata to HopsFS 4. “show me the images with >3 bicycles” and get a sub-second response. Data scientists: Do it all in Jupyter notebooks and Python (if you want)! Ops folks: Remove the image directory, and elasticsearch is auto-cleaned up! Images Train DNN HopsFS Image Search App 1. 2. 3. 4. Elastic
- 4. Data validation Distributed Training Model Serving A/B Testing Monitoring Pipeline Management HyperParameter Tuning Feature Engineering Data Collection Hardware Management Data Model Prediction φ(x) Hopsworks hides the Complexity of Deep Learning *Figure from “Technical Debt in Machine Learning Systems”, Google research paper
- 5. Data validation Distributed Training Model Serving A/B Testing Monitoring Pipeline Management HyperParameter Tuning Feature Engineering Data Collection Hardware Management Data Model Prediction φ(x) Hopsworks hides the Complexity of Deep Learning Hopsworks Feature Store
- 6. Data validation Distributed Training Model Serving A/B Testing Monitoring Pipeline Management HyperParameter Tuning Feature Engineering Data Collection Hardware Management Data Model Prediction φ(x) Hopsworks hides the Complexity of Deep Learning Hopsworks Feature Store Hopsworks REST API
- 11. What is Hopsworks? Efficiency & Performance Security & GovernanceUsability & Process Secure Multi-Tenancy Project-based restricted access Encryption At-Rest, In-Motion TLS/SSL everywhere AI-Asset Governance Models, experiments, data, GPUs Data/Model/Feature Lineage Discover/track dependencies Jupyter/Python Development Notebooks in pipelines Version Everything Code, Infrastructure, Data Model Serving on Kubernetes TF Serving, MLeap, SkLearn End-to-End ML Pipelines Orchestrated by Airflow Feature Store Data warehouse for ML Distributed Deep Learning Faster with more GPUs HopsFS NVMe speed with Big Data Horizontally Scalable Ingestion, DataPrep, Training, Serving FS
- 12. Which services require Distributed Metadata (HopsFS)? Efficiency & Performance Security & GovernanceUsability & Process Secure Multi-Tenancy Project-based restricted access Encryption At-Rest, In-Motion TLS/SSL everywhere AI-Asset Governance Models, experiments, data, GPUs Data/Model/Feature Lineage Discover/track dependencies Jupyter/Python Development Notebooks in pipelines Version Everything Code, Infrastructure, Data Model Serving on Kubernetes TF Serving, MLeap, SkLearn End-to-End ML Pipelines Orchestrated by Airflow Feature Store Data warehouse for ML Distributed Deep Learning Faster with more GPUs HopsFS NVMe speed with Big Data Horizontally Scalable Ingestion, DataPrep, Training, Serving FS
- 13. End-to-End ML Pipelines in Hopsworks
- 14. End-to-End Pipelines can be factored into stages
- 15. Typical Feature Store Pipelines
- 17. Dev View: Pipelines of Jupyter Notebooks in Airflow
- 19. First Class Python: Conda in the Cluster Conda Repo Hopsworks Cluster No need to write Dockerfiles
- 20. Demo
- 21. How to get started with Hopsworks? @hopsworks Register for a free account at: www.hops.site Images available for AWS, GCE, Virtualbox. https://www.logicalclocks.com/ https://github.com/logicalclocks/hopsworks Reach us