Consistency without Consensus: CRDTs in Production at SoundCloudC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1DKnwXr.
Peter Bourgon provides a practical introduction to Conflict-free Replicated Data Types (CRDTs) and describes a production CRDT system built at SoundCloud to serve several product features. Filmed at qconsf.com.
Peter Bourgon is a distributed systems engineer who has seen things. He works at SoundCloud, building and improving the infrastructure that powers the world's largest audio platform.
Consistency without Consensus: CRDTs in Production at SoundCloudC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1DKnwXr.
Peter Bourgon provides a practical introduction to Conflict-free Replicated Data Types (CRDTs) and describes a production CRDT system built at SoundCloud to serve several product features. Filmed at qconsf.com.
Peter Bourgon is a distributed systems engineer who has seen things. He works at SoundCloud, building and improving the infrastructure that powers the world's largest audio platform.
The presentation is made before a group of young people who have some knowledge of climate change but do not think that they have any role to play in reducing its effect. They are of the view that it’s the government to deal with climate change issues. The presentation aims to show them that they too can contribute in their own way without having to spend as much while going about their daily routine.
Aleksei Milovidov "Let's optimize one aggregate function in ClickHouse"Fwdays
Let's calculate an average of one column for each key, like the following query: SELECT key, avg(value) FROM table GROUP BY key. What can be more simple? But the question is: what is the most efficient way to do it? How to write code to achieve maximum performance on a variety of hardware?
Опыт разработки статического анализатора кодаAndrey Karpov
Один из основателей проекта PVS-Studio расскажет об опыте разработки статического анализатора кода C++. У инструментов статического анализа кода существует "проблема айсберга". От пользователей скрыты сложные механизмы анализа кода, и иногда им кажется, что статические анализаторы – это просто какие-то утилиты, ищущие опечатки с помощью регулярных выражений. Автор доклада постарается в общих чертах описать, как всё обстоит на самом деле. Он покажет на примерах, почему нормальный анализ с помощью регулярных выражений нереализуем, что такое Data Flow анализ, а также расскажет о других технологиях, применяемых при анализе кода. Вкратце будет затронут вопрос использования нейронных сетей, обсуждение которых сейчас является очень модной темой, и рассказано, почему с точки зрения анализа кода отношение к этому направлению является очень скептическим.
Павел Артёмкин — Разработка C++ API для реализации алгоритмов на больших графахYandex
В докладе рассказано о вычислительной модели на графах, в основе которой лежит механизм передачи сообщений между вершинами, а также о реализации в рамках данной модели API для написания алгоритмов на C++.
Всеволод Поляков "История одного мониторинга"Fwdays
«Мир изменился… Я чувствую это в воде… Я чувствую это в земле…»
Галадриэль
«Какой-то отсталый у неё мониторинг»
Сева Поляков
В этом докладе я хочу рассказать вам историю о современном мониторинге, на примере выбора для моего текущего проекта. Когда нужен prometheus, когда нужен SaaS и почему графит не умрёт. Также я постараюсь пройтись по всем новинкам и важным изменениям в современном мире мониторинга.
Евгений Крутько — Опыт внедрения технологий параллельных вычислений для повыш...Yandex
Евгений Крутько, НИЦ «Курчатовский институт».
В докладе на примере программы моделирования динамики движения конструкций по методу конечных элементов рассматриваются возможности и практика распараллеливания вычислений. Речь в нём пойдёт как о технике создания новых вычислительных потоков, так и об использовании стандартов openMP и MPI.
Looking to make your document processing operations more effective and cost-efficient with AI/ML? Learn from the experts of Provectus and Amazon Web Services (AWS) how to choose the right solution for your company! We will look into the management and engineering perspectives of AI document processing, from industry use cases and the solution map to our unique methodology for assessing available document processing solutions to Provectus IDP. Whether you are looking for a ready-made solution or you plan to build a custom solution of your own, this webinar will help you find the best option for your business.
Agenda
- Introductions
- Industry use cases
- Intelligent Document Processing (IDP) overview
- IDP Solutions map
- AWS IDP Solution
- Provectus IDP Platform
- Q&A
Intended Audience
Technology executives and decision makers, including such roles as CIO, CCO, COO, and CDO; digital transformation managers; data and ML engineers.
Presenters
Almir Davletov, IDP Subject Matter Expert, Provectus
Yaroslav Tarasyuk, Business Development, Provectus
Sonali Sahu, Sr. Solutions Architect, AWS
Interested? Learn more about Provectus Intelligent Document Processing Solution: https://provectus.com/document-processing-solution/
Intelligent Document Processing in Healthcare. Choosing the Right Solutions.Provectus
Healthcare organizations generate piles of documents and forms in different formats, making it difficult to achieve operational excellence and streamline business processes. Manual entry and OCR are no longer viable, and healthcare entities are looking for new solutions to handle documents.
In this presentation you can learn about:
- Healthcare document types and use cases
- IDP framework: building blocks for document processing solutions
- The document processing market landscape
- Methodology for solution evaluation: comparing apples to apples
Whether you are looking for a ready-made solution or plan to build a custom solution of your own, this webinar will help you find the best fit for your healthcare use cases.
Choosing the Right Document Processing Solution for Healthcare OrganizationsProvectus
Looking to automate document processing in your healthcare organization? Learn from Provectus & AWS experts how to make data capture, conversion, and analytics more efficient. Process and manage documents faster and on a larger scale with AI & Machine Learning.
In this presentation, we offer management and engineering perspectives on document processing with AI, to help you explore available options. Whether you are looking for a ready-made solution or plan to build a custom solution of your own, this webinar will help you find the best fit for your healthcare use cases.
MLOps and Data Quality: Deploying Reliable ML Models in ProductionProvectus
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
The presentation is made before a group of young people who have some knowledge of climate change but do not think that they have any role to play in reducing its effect. They are of the view that it’s the government to deal with climate change issues. The presentation aims to show them that they too can contribute in their own way without having to spend as much while going about their daily routine.
Aleksei Milovidov "Let's optimize one aggregate function in ClickHouse"Fwdays
Let's calculate an average of one column for each key, like the following query: SELECT key, avg(value) FROM table GROUP BY key. What can be more simple? But the question is: what is the most efficient way to do it? How to write code to achieve maximum performance on a variety of hardware?
Опыт разработки статического анализатора кодаAndrey Karpov
Один из основателей проекта PVS-Studio расскажет об опыте разработки статического анализатора кода C++. У инструментов статического анализа кода существует "проблема айсберга". От пользователей скрыты сложные механизмы анализа кода, и иногда им кажется, что статические анализаторы – это просто какие-то утилиты, ищущие опечатки с помощью регулярных выражений. Автор доклада постарается в общих чертах описать, как всё обстоит на самом деле. Он покажет на примерах, почему нормальный анализ с помощью регулярных выражений нереализуем, что такое Data Flow анализ, а также расскажет о других технологиях, применяемых при анализе кода. Вкратце будет затронут вопрос использования нейронных сетей, обсуждение которых сейчас является очень модной темой, и рассказано, почему с точки зрения анализа кода отношение к этому направлению является очень скептическим.
Павел Артёмкин — Разработка C++ API для реализации алгоритмов на больших графахYandex
В докладе рассказано о вычислительной модели на графах, в основе которой лежит механизм передачи сообщений между вершинами, а также о реализации в рамках данной модели API для написания алгоритмов на C++.
Всеволод Поляков "История одного мониторинга"Fwdays
«Мир изменился… Я чувствую это в воде… Я чувствую это в земле…»
Галадриэль
«Какой-то отсталый у неё мониторинг»
Сева Поляков
В этом докладе я хочу рассказать вам историю о современном мониторинге, на примере выбора для моего текущего проекта. Когда нужен prometheus, когда нужен SaaS и почему графит не умрёт. Также я постараюсь пройтись по всем новинкам и важным изменениям в современном мире мониторинга.
Евгений Крутько — Опыт внедрения технологий параллельных вычислений для повыш...Yandex
Евгений Крутько, НИЦ «Курчатовский институт».
В докладе на примере программы моделирования динамики движения конструкций по методу конечных элементов рассматриваются возможности и практика распараллеливания вычислений. Речь в нём пойдёт как о технике создания новых вычислительных потоков, так и об использовании стандартов openMP и MPI.
Looking to make your document processing operations more effective and cost-efficient with AI/ML? Learn from the experts of Provectus and Amazon Web Services (AWS) how to choose the right solution for your company! We will look into the management and engineering perspectives of AI document processing, from industry use cases and the solution map to our unique methodology for assessing available document processing solutions to Provectus IDP. Whether you are looking for a ready-made solution or you plan to build a custom solution of your own, this webinar will help you find the best option for your business.
Agenda
- Introductions
- Industry use cases
- Intelligent Document Processing (IDP) overview
- IDP Solutions map
- AWS IDP Solution
- Provectus IDP Platform
- Q&A
Intended Audience
Technology executives and decision makers, including such roles as CIO, CCO, COO, and CDO; digital transformation managers; data and ML engineers.
Presenters
Almir Davletov, IDP Subject Matter Expert, Provectus
Yaroslav Tarasyuk, Business Development, Provectus
Sonali Sahu, Sr. Solutions Architect, AWS
Interested? Learn more about Provectus Intelligent Document Processing Solution: https://provectus.com/document-processing-solution/
Intelligent Document Processing in Healthcare. Choosing the Right Solutions.Provectus
Healthcare organizations generate piles of documents and forms in different formats, making it difficult to achieve operational excellence and streamline business processes. Manual entry and OCR are no longer viable, and healthcare entities are looking for new solutions to handle documents.
In this presentation you can learn about:
- Healthcare document types and use cases
- IDP framework: building blocks for document processing solutions
- The document processing market landscape
- Methodology for solution evaluation: comparing apples to apples
Whether you are looking for a ready-made solution or plan to build a custom solution of your own, this webinar will help you find the best fit for your healthcare use cases.
Choosing the Right Document Processing Solution for Healthcare OrganizationsProvectus
Looking to automate document processing in your healthcare organization? Learn from Provectus & AWS experts how to make data capture, conversion, and analytics more efficient. Process and manage documents faster and on a larger scale with AI & Machine Learning.
In this presentation, we offer management and engineering perspectives on document processing with AI, to help you explore available options. Whether you are looking for a ready-made solution or plan to build a custom solution of your own, this webinar will help you find the best fit for your healthcare use cases.
MLOps and Data Quality: Deploying Reliable ML Models in ProductionProvectus
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
AI Stack on AWS: Amazon SageMaker and BeyondProvectus
Looking to learn more about AWS AI stack? Join experts from Provectus & AWS to find out how to use Amazon SageMaker (with combination with other tools and services) to enable enterprise-wide AI.
Companies are looking to scale and become more productive when it comes to AI and data initiatives. They seek to launch AI projects more rapidly, which, among many other factors, requires a robust machine learning infrastructure. In this webinar, you will learn how to create a canonical SageMaker workflow, expand the SageMaker workflow to a holistic implementation, enhance and expand the implementation using best practices for feature store, data versioning, ML pipeline orchestration, and model monitoring.
Agenda
- Introductions
- Amazon SageMaker Overview
- Real-World Use Case
- Data Lake for Machine Learning
- Amazon SageMaker Experiments
- Orchestration Beyond SageMaker Experiments
- Amazon SageMaker Debugger
- Amazon SageMaker Model Monitor
- Webinar Takeaways
Intended audience
Technology executives & decision makers, manager-level tech roles, data engineers & data scientists, ML practitioners & ML engineers, and developers
Presenters
- Stepan Pushkarev, Chief Technology Officer, Provectus
- Pritpal Sahota, Technical Account Manager, Provectus
- Christopher A. Burns, Sr. AI/ML Solution Architect, AWS
Feel free to share this presentation with your colleagues and don't hesitate to reach out to us at info@provectus.com if you have any questions!
REQUEST WEBINAR: https://provectus.com/ai-stack-on-aws-sagemaker-and-beyond-mar-2020/
Feature Store as a Data Foundation for Machine LearningProvectus
Looking to design and build a centralized, scalable Feature Store for your Data Science & Machine Learning teams to take advantage of? Come and learn from experts of Provectus and Amazon Web Services (AWS) how to!
Feature Store is a key component of the ML stack and data infrastructure, which enables feature engineering and management. By having a Feature Store, organizations can save massive amounts of resources, innovate faster, and drive ML processes at scale. In this webinar, you will learn how to build a Feature Store with a data mesh pattern and see how to achieve consistency between real-time and training features, to improve reproducibility with time-traveling for data.
Agenda
- Modern Data Lakes & Modern ML Infrastructure
- Existing and Emerging Architectural Shifts
- Feature Store: Overview and Reference Architecture
- AWS Perspective on Feature Store
Intended Audience
Technology executives & decision makers, manager-level tech roles, data architects & analysts, data engineers & data scientists, ML practitioners & ML engineers, and developers
Presenters
- Stepan Pushkarev, Chief Technology Officer, Provectus
- Gandhi Raketla, Senior Solutions Architect, AWS
- German Osin, Senior Solutions Architect, Provectus
Feel free to share this presentation with your colleagues and don't hesitate to reach out to us at info@provectus.com if you have any questions!
REQUEST WEBINAR: https://provectus.com/webinar-feature-store-as-data-foundation-for-ml-nov-2020/
MLOps and Reproducible ML on AWS with Kubeflow and SageMakerProvectus
Looking to implement MLOps using AWS services and Kubeflow? Come and learn about machine learning from the experts of Provectus and Amazon Web Services (AWS)!
Businesses recognize that machine learning projects are important but go beyond just building and deploying models, which is mostly done by organizations. Successful ML projects entail a complete lifecycle involving ML, DevOps, and data engineering and are built on top of ML infrastructure.
AWS and Amazon SageMaker provide a foundation for building infrastructure for machine learning while Kubeflow is a great open source project, which is not given enough credit in the AWS community. In this webinar, we show how to design and build an end-to-end ML infrastructure on AWS.
Agenda
- Introductions
- Case Study: GoCheck Kids
- Overview of AWS Infrastructure for Machine Learning
- Provectus ML Infrastructure on AWS
- Experimentation
- MLOps
- Feature Store
Intended Audience
Technology executives & decision makers, manager-level tech roles, data engineers & data scientists, ML practitioners & ML engineers, and developers
Presenters
- Stepan Pushkarev, Chief Technology Officer, Provectus
- Qingwei Li, ML Specialist Solutions Architect, AWS
Feel free to share this presentation with your colleagues and don't hesitate to reach out to us at info@provectus.com if you have any questions!
REQUEST WEBINAR: https://provectus.com/webinar-mlops-and-reproducible-ml-on-aws-with-kubeflow-and-sagemaker-aug-2020/
Cost Optimization for Apache Hadoop/Spark Workloads with Amazon EMRProvectus
Considering new ways and options for reducing operational costs and scaling flexibility of your Apache Hadoop/Spark? Try migrating to Amazon EMR!
On-premises Apache Hadoop/Spark clusters are among the top sources of financial pressure for businesses. IT organizations want to reduce spend while still meeting demand, to keep their legacy data applications up and running. Come and learn from experts at Provectus & AWS how you can use Amazon EMR to start driving cost efficiencies in your organization!
Agenda
- Hadoop market and cost optimizations using Amazon EMR
- Cost related and other challenges of on-prem Hadoop clusters
- Cost optimizations by using Amazon EMR and migration best practices
Intended audience
Technology executives & decision makers, manager-level tech roles, data engineers & data scientists, and developers
Presenters
- Stepan Pushkarev, Chief Technology Officer, Provectus
- Pritpal Sahota, Technical Account Manager, Provectus
- Nirav Shah, Senior Solutions Architect, AWS
- Perry Peterson, Business Development Manager, AWS
Feel free to share this presentation with your colleagues and don't hesitate to reach out to us at info@provectus.com if you have any questions!
REQUEST WEBINAR: https://provectus.com/cost-optimization-for-apache-hadoop-spark-workloads-with-amazon-emr-june-2020/
ODSC webinar "Kubeflow, MLFlow and Beyond — augmenting ML delivery" Stepan Pu...Provectus
What's a machine learning workflow? What open source tools can you use to automate ML workflow?
Reproducible ML pipelines in research and production with monitoring insights from live inference clusters could enable and accelerate the delivery of AI solutions for enterprises. There is a growing ecosystem of tools that augment researchers and machine learning engineers in their day to day operations.
Still, there are big gaps in the machine learning workflow when it comes to training dataset versioning, training performance and metadata tracking, integration testing, inferencing quality monitoring, bias detection, concept drift detection and other aspects that prevent the adoption of AI in organizations of all sizes.
"Building a Modern Data platform in the Cloud", Alex Casalboni, AWS Dev Day K...Provectus
AWS Dev Day Kyiv 2019
Track: Analytics & Machine Learning
Session: "Building a Modern Data platform in the Cloud"
Speaker: Alex Casalboni, AWS Technical Evangelist
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/HIDnAG9AxZo
"How to build a global serverless service", Alex Casalboni, AWS Dev Day Kyiv ...Provectus
AWS Dev Day Kyiv 2019
Track: Modern Application Development
Session: "How to build a global serverless service"
Speaker: Alex Casalboni, AWS Technical Evangelist
Level: 400
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/Q19B-NTkMfk
"Automating AWS Infrastructure with PowerShell", Martin Beeby, AWS Dev Day Ky...Provectus
AWS Dev Day Kyiv 2019
Track: Backend & Architecture
Session: "Automating AWS Infrastructure with PowerShell"
Speaker: Martin Beeby, AWS Principle Evangelist
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/rgIjjK2J4dQ
"Analyzing your web and application logs", Javier Ramirez, AWS Dev Day Kyiv 2...Provectus
AWS Dev Day Kyiv 2019
Track: Analytics & Machine Learning
Session: "Analyzing your web and application logs"
Speaker: Javier Ramirez, AWS Technical Evangelist
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/IpEhEs1sXeg
"Resiliency and Availability Design Patterns for the Cloud", Sebastien Storma...Provectus
AWS Dev Day Kyiv 2019
Track: Backend & Architecture
Session: "Resiliency and Availability Design Patterns for the Cloud"
Speaker: Sebastien Stormacq, AWS Technical Evangelist
Level: 400
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/O8gonQCJawU
"Architecting SaaS solutions on AWS", Oleksandr Mykhalchuk, AWS Dev Day Kyiv ...Provectus
AWS Dev Day Kyiv 2019
Track: Backend & Architecture
Session: ""Architecting SaaS solutions on AWS""
Speaker: Oleksandr Mykhalchuk, Director of DevOps & Cloud Services at Softserve
Level: 300
Video: https://youtu.be/3lKoe-ts8Qs
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
"Developing with .NET Core on AWS", Martin Beeby, AWS Dev Day Kyiv 2019Provectus
AWS Dev Day Kyiv 2019
Track: Modern Application Development
Session: "Developing with .NET Core on AWS"
Speaker: Martin Beeby, AWS Principle Evangelist
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/OzM8L7H1LmA
"How to build real-time backends", Martin Beeby, AWS Dev Day Kyiv 2019Provectus
AWS Dev Day Kyiv 2019
Track: Backend & Architecture
Session: "How to build real-time backends"
Speaker: Martin Beeby, AWS Principle Evangelist
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://youtu.be/bsZYA6V3bDA
"Integrate your front end apps with serverless backend in the cloud", Sebasti...Provectus
AWS Dev Day Kyiv 2019
Track: Modern Application Development
Session: "Integrate your front end apps with serverless backend in the cloud"
Speaker: Sebastien Stormacq, AWS Technical Evangelist
Level: 200
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://www.youtube.com/watch?v=6z43H11qoU8&t=1s
"Scaling ML from 0 to millions of users", Julien Simon, AWS Dev Day Kyiv 2019Provectus
AWS Dev Day Kyiv 2019
Track: Analytics & Machine Learning
Session: ""Scaling ML from 0 to millions of users""
Speaker: Julien Simon, Global AI & Machine Learning Evangelist at AWS
Level: 300
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
Video: https://www.youtube.com/watch?v=N73u1mx9DqY
How to implement authorization in your backend with AWS IAMProvectus
AWS Dev Day Kyiv 2019
Track: Backend & Architecture
Session: ""How to implement authorization in your backend with AWS IAM""
Speaker: Stas Ivaschenko, AWS solutions architect at Provectus
Level: 400
Video: https://www.youtube.com/watch?v=4Jje_WJ4V7Q
AWS Dev Day is a free, full-day technical event where new developers will learn about some of the hottest topics in cloud computing, and experienced developers can dive deep on newer AWS services.
Provectus has organized AWS Dev Day Kyiv in close collaboration with Amazon Web Services: 800+ participants, 18 sessions, 3 tracks, a really AWSome Day!
Now, together with Zeo Alliance, we're building and nurturing AWS User Group Ukraine — join us on Facebook to stay updated about cloud technologies and AWS services: https://www.facebook.com/groups/AWSUserGroupUkraine
"
3. Распределённые системы
• Много общающихся машин
• Общее состояние
• Бывает разное
• CAP теорема
• Consistency, Availability, Partitioning – выберите любые два
• CALM теорема
• Consistency as logical monotonicity – система может быть распределённой,
если она не удаляет исторические факты
4.
5. CRDT
• Тип структур данных для eventually consistent систем
• Формально доказано: состояние системы сходится
• CAP теорема: выкидываем строгую consistency
• CALM теорема: строгая гарантия eventual consistency
6. CRDT: State-based
• Множество <A, combine>
• a combine (b combine c) == (a combine b) combine c
• a combine b == b combine a
• a combine a == a
• Необходимо есть нулевой элемент
• Ограниченная полурешетка AKA коммутативный идемпотентный моноид
• Частичный порядок
7. G-Counter
• Можно только увеличивать счётчик
case class GCounter[R, E](state: Map[R, E])(implicit num: Numeric[E])
import com.machinomy.crdt.state._
import cats.syntax.all._
import cats._
val counter = Monoid[GCounter[Int, Int]].empty // empty G-Counter
val firstReplica = counter + (1 -> 1) // increment replica 1
val secondReplica = counter + (2 -> 2) // increment replica 2
val firstReplicacombined = firstReplica |+| secondReplica // combine
val secondReplicacombined = secondReplica |+| firstReplica // combine
firstReplicacombined == secondReplicacombined // the result is independent of combine order
8. PN-Counter
• Можно делать инкремент и декремент
class PNCounter[K, E: Numeric](increments: GCounter[K, E], decrements: GCounter[K, E])
import com.machinomy.crdt.state._
import cats.syntax.all._
import cats._
val counter = Monoid[PNCounter[Int, Int]].empty
val firstReplica = counter + (1 -> 1) // increment replica 1
val secondReplica = counter + (2 -> -2) // decrement replica 2
val firstReplicacombined = firstReplica |+| secondReplica // combine
val secondReplicacombined = secondReplica |+| firstReplica // combine
firstReplicacombined == secondReplicacombined // the result is independent of combine
order
9. G-Set
• Только добавление, `combine` объединяет множества
• class GSet[E](val state: Set[E])
import com.machinomy.crdt.state._
import cats.syntax.all._
import cats._
val counter = Monoid[GSet[Int]].empty // empty G-Set
val firstReplica = counter + 1 // add element
val secondReplica = counter + 2 // add element
val firstReplicacombined = firstReplica |+| secondReplica // combine
val secondReplicacombined = secondReplica |+| firstReplica // combine
firstReplicacombined == secondReplicacombined // the result is independent of
combine order
14. 2P2P-Graph
case class TPTPGraph[A, V <: VertexLike[A], E <: EdgeLike[A, V]]
(va: Set[V], vr: Set[V], ea: Set[E], er: Set[E])
• Можно добавлять и удалять элементы
• Но никаких гарантий!
16. Monotonic DAG
case class MonotonicDag[V, E[X] <: DiEdgeLikeIn[X], G <: Graph[V, E]](graph: G)
type UpdateResult[V, E[X] <: DiEdgeLikeIn[X], G <: Graph[V, E]] = (MonotonicDag[V, E, G],
Option[Update[V, E, G]])
def add(e: E[V] with OuterEdge[V, E]): MonotonicDag.UpdateResult[V, E, G] = {
val from = graphLike.fromVertex(graph, e)
val to = graphLike.toVertex(value, e)
val containsSource = graphLike.contains(value, from)
val containsDestination = graphLike.contains(value, to)
val effective = graphLike.addEdge(value, e)
if (containsSource && containsDestination && graphLike.existsPath(effective, from, to)) {
(MonotonicDag[V, E, G](effective), Some(MonotonicDag.AddEdge[V, E, G](e)))
} else {
(this, None)
}
}
17. Pro et Contra
• Простая логика
• Стабильное теоретическое
обоснование
• Лучше, чем OT
• Социальное доказательство
• Непонятно, что делать с BFT
• Необходимо обеспечить
стабильный канал
• Сложно найти подходящую
реализацию
https://github.com/machinomy/crdt
https://www.mendeley.com/groups/9275881/crdt/papers/
sergey.ukustov@machinomy.com