Blox is a collection of open source projects for container management and orchestration on Amazon ECS. Blox gives you more control over how your containerized applications run on Amazon ECS, and it enables you to build schedulers and integrate third-party schedulers on top of ECS, while leveraging Amazon ECS to fully manage and scale your clusters.
Amazon EC2 allows you to bid for and run spare EC2 capacity, known as Spot instances, in a dynamically priced market. On average, customers save 80% to 90% compared to On Demand prices by using Spot instances. Achieving these savings has historically required time and effort to find the best deals while managing compute capacity as supply and demand fluctuate.
Using AWS Batch and AWS Step Functions to Design and Run High-Throughput Work...Amazon Web Services
Learning Objectives:
- How to simply scale out your batch workflows on AWS
- How to think about container/job management within managed, high-throughput workflows
- How to build a scalable orchestration framework within AWS Step Functions
An introduction to serverless architectures (February 2017)Julien SIMON
An introduction to serverless
AWS Lambda
Amazon API Gateway
Demo: writing your first Lambda function
Demo: building a serverless pipeline
Additional resources
AWS Batch is a service that enables developers, scientists, and engineers to easily and efficiently run batch computing workloads at scale on AWS. AWS Batch automatically provisions compute resources and optimizes the workload distribution based on the quantity and scale of the workloads, allowing you to focus on analyzing results and solving problems.
In this session, led by the AWS Batch service team, you will learn core concepts behind AWS Batch and details of how the service functions. We will cover multiple patterns used by customers to leverage storage and GPUs as part of their batch workloads. We will also cover how to integrate AWS Batch with other services such as AWS Step Functions for decision based workloads or Amazon CloudWatch Events to trigger batch jobs based on events or schedules.
Scaling your web app horizontally and vertically (ahmedabad amazon aws cloud...Jhalak Modi
Auto Scaling helps you maintain application availability and allows you to scale your Amazon EC2 capacity up or down automatically according to conditions you define. You can use Auto Scaling to help ensure that you are running your desired number of Amazon EC2 instances. Auto Scaling can also automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs. Auto Scaling is well suited both to applications that have stable demand patterns or that experience hourly, daily, or weekly variability in usage.
Amazon EC2 Container Service is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. Part of ECS is Amazon EC2 Container Registry (ECR). Amazon ECR is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. This session will describe how you can use ECS and ECR for your applications.
Speaker: Sascha Möllering, Solutions Architect, AWS
Amazon Web Services (AWS) provides Elastic Load Balancing to automatically distribute incoming web traffic across multiple Amazon Elastic Compute Cloud (Amazon EC2) instances.
With Elastic Load Balancing, you can add and remove EC2 instances as your needs change without disrupting the overall flow of information. If one EC2 instance fails, Elastic Load Balancing automatically reroutes the traffic to the remaining running EC2 instances. If the failed EC2 instance is restored, Elastic Load Balancing restores the traffic to that instance.
Elastic Load Balancing offers clients a single point of contact, and it can also serve as the first line of defense against attacks on your network. You can offload the work of encryption and decryption to Elastic Load Balancing, so your servers can focus on their main task.
Building A Dynamic Website - 31st Jan 2015Jhalak Modi
Amazon Web Services offers cloud website hosting solutions that provides businesses, non-profits, and governmental organizations with a flexible, highly scalable, and low-cost way to deliver their websites and web applications. Our agenda is "How to deploy a dynamic website using Amazon Web Services". We will discuss some special services on amazon that is AWS Elastic Cloud Compute (EC2), Relational Database Service (RDS), Elastic Load Balancing (ELB), Route 53 (R53).
Blox is a collection of open source projects for container management and orchestration on Amazon ECS. Blox gives you more control over how your containerized applications run on Amazon ECS, and it enables you to build schedulers and integrate third-party schedulers on top of ECS, while leveraging Amazon ECS to fully manage and scale your clusters.
Amazon EC2 allows you to bid for and run spare EC2 capacity, known as Spot instances, in a dynamically priced market. On average, customers save 80% to 90% compared to On Demand prices by using Spot instances. Achieving these savings has historically required time and effort to find the best deals while managing compute capacity as supply and demand fluctuate.
Using AWS Batch and AWS Step Functions to Design and Run High-Throughput Work...Amazon Web Services
Learning Objectives:
- How to simply scale out your batch workflows on AWS
- How to think about container/job management within managed, high-throughput workflows
- How to build a scalable orchestration framework within AWS Step Functions
An introduction to serverless architectures (February 2017)Julien SIMON
An introduction to serverless
AWS Lambda
Amazon API Gateway
Demo: writing your first Lambda function
Demo: building a serverless pipeline
Additional resources
AWS Batch is a service that enables developers, scientists, and engineers to easily and efficiently run batch computing workloads at scale on AWS. AWS Batch automatically provisions compute resources and optimizes the workload distribution based on the quantity and scale of the workloads, allowing you to focus on analyzing results and solving problems.
In this session, led by the AWS Batch service team, you will learn core concepts behind AWS Batch and details of how the service functions. We will cover multiple patterns used by customers to leverage storage and GPUs as part of their batch workloads. We will also cover how to integrate AWS Batch with other services such as AWS Step Functions for decision based workloads or Amazon CloudWatch Events to trigger batch jobs based on events or schedules.
Scaling your web app horizontally and vertically (ahmedabad amazon aws cloud...Jhalak Modi
Auto Scaling helps you maintain application availability and allows you to scale your Amazon EC2 capacity up or down automatically according to conditions you define. You can use Auto Scaling to help ensure that you are running your desired number of Amazon EC2 instances. Auto Scaling can also automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs. Auto Scaling is well suited both to applications that have stable demand patterns or that experience hourly, daily, or weekly variability in usage.
Amazon EC2 Container Service is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. Part of ECS is Amazon EC2 Container Registry (ECR). Amazon ECR is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. This session will describe how you can use ECS and ECR for your applications.
Speaker: Sascha Möllering, Solutions Architect, AWS
Amazon Web Services (AWS) provides Elastic Load Balancing to automatically distribute incoming web traffic across multiple Amazon Elastic Compute Cloud (Amazon EC2) instances.
With Elastic Load Balancing, you can add and remove EC2 instances as your needs change without disrupting the overall flow of information. If one EC2 instance fails, Elastic Load Balancing automatically reroutes the traffic to the remaining running EC2 instances. If the failed EC2 instance is restored, Elastic Load Balancing restores the traffic to that instance.
Elastic Load Balancing offers clients a single point of contact, and it can also serve as the first line of defense against attacks on your network. You can offload the work of encryption and decryption to Elastic Load Balancing, so your servers can focus on their main task.
Building A Dynamic Website - 31st Jan 2015Jhalak Modi
Amazon Web Services offers cloud website hosting solutions that provides businesses, non-profits, and governmental organizations with a flexible, highly scalable, and low-cost way to deliver their websites and web applications. Our agenda is "How to deploy a dynamic website using Amazon Web Services". We will discuss some special services on amazon that is AWS Elastic Cloud Compute (EC2), Relational Database Service (RDS), Elastic Load Balancing (ELB), Route 53 (R53).
Talk on Amazon Redshift, Meetup Les Nouvelles Organisations, 11/02/2016, Paris - http://www.meetup.com/fr-FR/lesnouvellesorganisations/events/227195680/
Microservices is a software architectural method where you decompose complex applications into smaller, independent services. Containers are great for running small decoupled services, but how do you coordinate running microservices in production at scale and what AWS services do you use?
In this session, we will explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications. We will also demonstrate how you can easily launch microservices on Amazon EC2 Container Service and how you can use ELB and Route 53 to easily do service discovery between microservices.
Availability & Scalability with Elastic Load Balancing & Route 53 (CPN204) | ...Amazon Web Services
Elastic Load Balancing provides a scalable and highly-available load balancer that automatically distributes incoming application traffic across multiple Amazon EC2 instances. It enables you to achieve even greater fault tolerance in your applications, seamlessly providing the amount of load balancing capacity needed in response to incoming application traffic. In this session, we take a deeper look at some of the existing and newer features that enable application developers to architect highly-available architectures that are resilient to load spikes and application failures. We also explore some of the features that allow seamless integration with services such as Auto Scaling and Amazon Route 53 to further improve the scalability and resilience of your applications.
(ISM303) Migrating Your Enterprise Data Warehouse To Amazon RedshiftAmazon Web Services
Learn how Boingo Wireless and online media provider Edmunds gained substantial business insights and saved money and time by migrating to Amazon Redshift. Get an inside look into how they accomplished their migration from on-premises solutions. Learn how they tuned their schema and queries to take full advantage of the columnar MPP architecture in Amazon Redshift, how they leveraged third party solutions, and how they met their business intelligence needs in record time.
Many of our customers have adopted DevOps for faster and reliable software delivery. Applying software engineering best practices such as revision control and continuous delivery to your infrastructure is essential for adopting DevOps.
In this session, find out how AWS CloudFormation and the associated AWS tools enable DevOps by allowing you to treat infrastructure as code and applying those software engineering best practices to your infrastructure.
Speakers:
Steven Bryen, AWS Solutions Architect
Bruce Jackson, Chief Technology Officer, Myriad Group
Rajpal Singh Wilkhu,Principal Engineer, Just Eat
(SDD408) Amazon Route 53 Deep Dive: Delivering Resiliency, Minimizing Latency...Amazon Web Services
Learn how to utilize Amazon Route 53 latency-based routing, weighted round-robin, and other features in conjunction with DNS failover to direct traffic to the least latent, most available endpoints across a global infrastructure. We explore topics such as balancing traffic between endpoints in terms of load and latency, and discuss how to provide multi-record answers to improve client-side resiliency. As part of this session, Loggly will present how they utilize Route 53 for their traffic management needs.
Learn how the Blue/Green Deployment methodology combined with AWS tools and services can help reduce the risks associated with software deployment. We will illustrate common patterns and highlight ways deployment risks are mitigated by each pattern. Topics will include how services like AWS CloudFormation, AWS Elastic Beanstalk, Amazon EC2 Container Service, Amazon Route53, Auto Scaling and Elastic Load Balancing can help automate deployment. We will also address how to effectively manage deployments in the context of data model and schema changes. Learn how you can adopt blue/green for your software release processes in a cost-effective and low-risk way.
"Conceptually, a data lake is a flat data store to collect data in its original form, without the need to enforce a predefined schema. Instead, new schemas or views are created “on demand”, providing a far more agile and flexible architecture while enabling new types of analytical insights. AWS provides many of the building blocks required to help organizations implement a data lake. In this session, we will introduce key concepts for a data lake and present aspects related to its implementation. We will discuss critical success factors, pitfalls to avoid as well as operational aspects such as security, governance, search, indexing and metadata management. We will also provide insight on how AWS enables a data lake architecture.
A data lake is a flat data store to collect data in its original form, without the need to enforce a predefined schema. Instead, new schemas or views are created ""on demand"", providing a far more agile and flexible architecture while enabling new types of analytical insights. AWS provides many of the building blocks required to help organizations implement a data lake. In this session, we introduce key concepts for a data lake and present aspects related to its implementation. We discuss critical success factors and pitfalls to avoid, as well as operational aspects such as security, governance, search, indexing, and metadata management. We also provide insight on how AWS enables a data lake architecture. Attendees get practical tips and recommendations to get started with their data lake implementations on AWS."
Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the AWS cloud where you can launch AWS resources in a virtual network that you define. In this talk, we discuss advanced tasks in Amazon VPC, including the implementation of Amazon VPC peering, the creation of multiple network zones, the establishment of private connections, and the use of multiple routing tables. We also provide information for current Amazon EC2-Classic network customers and help you prepare to adopt Amazon VPC.
Speakers:
Steve Seymour, AWS Solutions Architect
Eamonn O'Neill, Director, Lemongrass Consulting
Jackie Wong, Head of Networks, Financial Times
Building prediction models with Amazon Redshift and Amazon Machine Learning -...Amazon Web Services
Mining data with Redshift, using this data to build a prediction model with Amazon ML, performing batch predictions & real-time predictions (with a Java app).
Businesses are generating more data than ever before.
Doing real time data analytics requires IT infrastructure that often needs to be scaled up quickly and running an on-premise environment in this setting has its limitations.
Organisations often require a massive amount of IT resources to analyse their data and the upfront capital cost can deter them from embarking on these projects.
What’s needed is scalable, agile and secure cloud-based infrastructure at the lowest possible cost so they can spin up servers that support their data analysis projects exactly when they are required. This infrastructure must enable them to create proof-of-concepts quickly and cheaply – to fail fast and move on.
최근 데이터의 폭증과 이를 기반한 빅데이터 분석이 기업 비지니스 성패에 큰 영향을 끼치고 있습니다. 다양한 기업의 데이터 기반 의사 결정을 위한 요구를 수용하는 분석 플랫폼과 인공 지능 기술의 도입은 큰 화두입니다. 본 세션에서는 기업의 비지니스 전략 및 기획을 담당하시는 분들을 위해 클라우드 기반 데이터 분석 플랫폼을 쉽게 접근하고 사용할 수 있는 방법을 사례 위주로 소개합니다.국내외 주요 기업들이 어떻게 AWS기반 데이터 분석 및 기계 학습 서비스로 비지니스 혁신에 활용하고 있는지 알아보시기 바랍니다.
다시보기 링크: https://youtu.be/24YgdrJ9r-A
AWS re:Invent 2016: Migrating Your Data Warehouse to Amazon Redshift (DAT202)Amazon Web Services
Amazon Redshift is a fast, simple, cost-effective data warehousing solution, and in this session, we look at the tools and techniques you can use to migrate your existing data warehouse to Amazon Redshift. We will then present a case study on Scholastic’s migration to Amazon Redshift. Scholastic, a large 100-year-old publishing company, was running their business with older, on-premise, data warehousing and analytics solutions, which could not keep up with business needs and were expensive. Scholastic also needed to include new capabilities like streaming data and real time analytics. Scholastic migrated to Amazon Redshift, and achieved agility and faster time to insight while dramatically reducing costs. In this session, Scholastic will discuss how they achieved this, including options considered, technical architecture implemented, results, and lessons learned.
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...Amazon Web Services
Amazon Redshift is a fast and powerful, fully managed, petabyte-scale data warehouse service in the cloud. In this session we'll give an introduction to the service and its pricing before diving into how it delivers fast query performance on data sets ranging from hundreds of gigabytes to a petabyte or more.
AWS re:Invent 2016: What’s New with Amazon Redshift (BDA304)Amazon Web Services
In this session, you learn about the latest and hottest features of Amazon Redshift. Join Vidhya Srinivasan, General Manager of Amazon Redshift, to take a deep dive into the architecture and inner workings of Amazon Redshift. You discover how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your end user experience. You also get a glimpse of what we are working on and our plans for the future.
Amazon Redshift in Action: Enterprise, Big Data, and SaaS Use Cases (DAT205) ...Amazon Web Services
Since Amazon Redshift launched last year, it has been adopted by a wide variety of companies for data warehousing. In this session, learn how customers NASDAQ, HauteLook, and Roundarch Isobar are taking advantage of Amazon Redshift for three unique use cases: enterprise, big data, and SaaS. Learn about their implementations and how they made data analysis faster, cheaper, and easier with Amazon Redshift.
Over 90% of today’s data has been generated in the last two years, and growth rates continue to climb. In this session, we’ll step through challenges and best practices with data capturing, how to derive meaningful insights to help predict the future, and common pitfalls in data analysis.
Come discover how integrated solutions involving Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon Machine Learning/Deep Learning result in effective data systems for data scientists and business users, alike.
Using real time big data analytics for competitive advantageAmazon Web Services
Many organisations find it challenging to successfully perform real-time data analytics using their own on premise IT infrastructure. Building a system that can adapt and scale rapidly to handle dramatic increases in transaction loads can potentially be quite a costly and time consuming exercise.
Most of the time, infrastructure is under-utilised and it’s near impossible for organisations to forecast the amount of computing power they will need in the future to serve their customers and suppliers.
To overcome these challenges, organisations can instead utilise the cloud to support their real-time data analytics activities. Scalable, agile and secure, cloud-based infrastructure enables organisations to quickly spin up infrastructure to support their data analytics projects exactly when it is needed. Importantly, they can ‘switch off’ infrastructure when it is not.
BluePi Consulting and Amazon Web Services (AWS) are giving you the opportunity to discover how organisations are using real time data analytics to gain new insights from their information to improve the customer experience and drive competitive advantage.
講師: Ivan Cheng, Solution Architect, AWS
Join us for a series of introductory and technical sessions on AWS Big Data solutions. Gain a thorough understanding of what Amazon Web Services offers across the big data lifecycle and learn architectural best practices for applying those solutions to your projects.
We will kick off this technical seminar in the morning with an introduction to the AWS Big Data platform, including a discussion of popular use cases and reference architectures. In the afternoon, we will deep dive into Machine Learning and Streaming Analytics. We will then walk everyone through building your first Big Data application with AWS.
최근 데이터의 폭증과 이를 기반한 빅데이터 분석이 기업 비지니스 성패에 큰 영향을 끼치고 있습니다. 다양한 기업의 데이터 기반 의사 결정을 위한 요구를 수용하는 분석 플랫폼과 인공 지능 기술의 도입은 큰 화두입니다. 본 세션에서는 기업의 비지니스 전략 및 기획을 담당하시는 분들을 위해 클라우드 기반 데이터 분석 플랫폼을 쉽게 접근하고 사용할 수 있는 방법을 사례 위주로 소개합니다.국내외 주요 기업들이 어떻게 AWS기반 데이터 분석 및 기계 학습 서비스로 비지니스 혁신에 활용하고 있는지 알아보시기 바랍니다.
Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...Amazon Web Services
In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Amazon Web Services
• Overview of database services to elevate your applications, analytic services to engage your data, and migration services to help you reach database freedom.
• Survey of how Canadian and other organizations are using the cloud to make data scalable, reliable, and secure.
With AWS you can choose the right database technology and software for the job. Given the myriad of choices, from relational databases to non-relational stores, this session provides details and examples of some of the choices available to you. This session also provides details about real-world deployments from customers using Amazon RDS, Amazon ElastiCache, Amazon DynamoDB, and Amazon Redshift.
In this session, we will introduce Amazon RedShift, a new petabyte scale data warehouse service. We'll walk through the basics of the Redshift architecture, launching a new cluster and run SQL queries across a large scale, public dataset. After demonstrating how easy it is to get started with RedShift, we will show how to visualize and query large scale datasets, running queries, reports, and analytics against millions of rows of records in just a few seconds.
Data warehousing in the era of Big Data: Deep Dive into Amazon RedshiftAmazon Web Services
Analyzing big data quickly and efficiently requires a data warehouse optimized to handle and scale for large datasets. Amazon Redshift is a fast, petabyte-scale data warehouse that makes it simple and cost-effective to analyze all of your data for a fraction of the cost of traditional data warehouses. In this session, we take an in-depth look at data warehousing with Amazon Redshift for big data analytics. We cover best practices to take advantage of Amazon Redshift's columnar technology and parallel processing capabilities to deliver high throughput and query performance. We also discuss how to design optimal schemas, load data efficiently, and use work load management.
Why Scale Matters and How the Cloud is Really Different (at scale)Amazon Web Services
Cloud computing gives you a number of advantages, such as being able to scale your application on demand. As a new business looking to use the cloud, you inevitably ask yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We will show you how to best combine different AWS services, make smarter decisions for architecting your application, and best practices for scaling your infrastructure in the cloud.
Presenter:
Santanu Dutt, Solution Architect, Amazon Internet Services
Vinayak Hegde, Vice President – Engineering, Helpshift
Sunny Saxena, Product Lead, Sprinklr
Similar to Building a data warehouse with Amazon Redshift … and a quick look at Amazon Machine Learning (20)
An introduction to computer vision with Hugging FaceJulien SIMON
In this code-level talk, Julien will show you how to quickly build and deploy computer vision applications based on Transformer models. Along the way, you'll learn about the portfolio of open source and commercial Hugging Face solutions, and how they can help you deliver high-quality solutions faster than ever before.
Starting your AI/ML project right (May 2020)Julien SIMON
In this talk, we’ll see how you can put your AI/ML project on the right track from the get-go. Applying common sense and proven best practices, we’ll discuss skills, tools, methods, and more. We’ll also look at several real-life projects built by AWS customers in different industries and startups.
Building Machine Learning Inference Pipelines at Scale (July 2019)Julien SIMON
Talk at OSCON, Portland, 18/07/2019
Real-life Machine Learning applications require more than a single model. Data may need pre-processing: normalization, feature engineering, dimensionality reduction, etc. Predictions may need post-processing: filtering, sorting, combining, etc.
Our goal: build scalable ML pipelines with open source (Spark, Scikit-learn, XGBoost) and managed services (Amazon EMR, AWS Glue, Amazon SageMaker)
Optimize your Machine Learning Workloads on AWS (July 2019)Julien SIMON
Talk at Floor 28, Tel Aviv.
Infrastructure, tips to speed up training, hyperparameter optimization, model compilation, Amazon SageMaker Neo, cost optimization, Amazon Elastic Inference
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
4. Collect Store Analyze Consume
A
iOS
Android
Web Apps
Logstash
Amazon
RDS
Amazon
DynamoDB
Amazon
ES
Amazon
S3
Apache
Kafka
Amazon
Glacier
Amazon
Kinesis
Amazon
DynamoDB
Amazon
Redshift
Impala
Pig
Amazon ML
Streaming
Amazon
Kinesis
AWS
Lambda
AmazonElasticMapReduce
Amazon
ElastiCache
SearchSQLNoSQLCache
StreamProcessing
Batch
Interactive
Logging
StreamStorage
IoT
Applications
FileStorage
Analysis&Visualization
Hot
Cold
Warm
Hot
Slow
Hot
ML
Fast
Fast
Amazon
QuickSight
Transactional Data
File Data
Stream Data
Notebooks
Predictions
Apps & APIs
Mobile
Apps
IDE
Search Data
ETL
5. Amazon Redshift
a relational, petabyte scale, fully managed data warehousing service
• SQL is all you need to know
• ODBC and JDBC drivers available
• Parallel processing on multiple nodes
• No system administration
• Free tier: 750 hours / month for 2 months
• Available on-demand from $0.25 / hour / node
• As low as $1,000 / Terabyte / year
“ Come for the cost, stay for the performance ”
10. Case study: Photobox
Maxime Mezin, Data & Photo Science Director:
“L’entrepôt de données ne comportait que les données du site e-commerce liées aux ventes.
Alors que nous avions la volonté d’intégrer des données du service clients et des données
d’analyse (…) Nous avions atteint la limite du stockage de la base Oracle, et cela ne marchait
pas très bien en termes de performances”
“Avec Redshift, la rapidité d’exécution des traitements a été multipliée par 10. Sans parler de
la vitesse de chargement des données”
“On paie en fonction de la quantité de données que l’on va stocker. Chez Google, cela était
plus compliqué”
• 2 Redshift clusters : 1 for historical data, 1 for real-time processing (SSD)
• TCO divided by 7 (90K€à13K€)
http://www.lemagit.fr/etude/Photobox-consolide-et-analyse-ses-donnees-avec-AWS-RedShift
11. Case study: Financial Times
• BI analysis of reader traffic, in order to decide which stories to cover
• Conventional data warehouse running on Microsoft technologies
• Scalability issues, impossible to perform real-time analytics à Amazon Redshift PoC
• Amazon Redshift performed so quickly that some analysts thought it was malfunctioning J
John O’Donovan, CTO: “Amazon Redshift is the single source of truth for our user data.”
“Some of the queries we’re running are 98 percent faster, and most things are running 90
percent faster (…) and the ability to try Redshift out before having to invest a significant
amount of capital was a huge bonus.”
“Being able to explore near-real-time data improves our decision making massively. We can
make decisions based on what’s happening now rather than what happened three or four
days ago.”
• Total Cost of Ownership divided by 4
https://aws.amazon.com/solutions/case-studies/financial-times/
12. Case study: Boingo
https://www.youtube.com/watch?v=58URZbp1voY
• Largest operator of airport wireless hotspots in the world: 1M+ hotspots,100+ countries
• About 15 TB of data, growing at 2-3 TB per year
• Platform running on SAP (ETL) & Oracle 11g: low performance, heavy admin, high cost
• Evaluated Oracle Exadata, SAP HANA and Amazon Redshift
• Selected Amazon Redshift and migrated in 2 months
Queries
180x faster
Data load
480x faster
6-7x less expensive than alternatives
13. Amazon Redshift performance
No indexes, no partitioning, no wizardry.
Distribution key
• How data is spread across nodes
• EVEN (default), ALL, KEY
Sort key
• How data is sorted inside of disk blocks
• Compound and interleaved keys are possible
Both are crucial to query performance! Universal Pictures
14. DEMO #1
Demo gods, I’m your humble servant,
please be good to me
8-node cluster
1 billion lines of CSV data (45GB)
à 1 billion rows in a single table
15. Amazon Machine Learning
A managed service for building ML models
and generating predictions
Integration with Amazon S3, Redshift and RDS
Data transformation, visualization and exploration
Model evaluation and interpretation tools
API for batch and real-time predictions
$0.42 / hour for analysis and model building (eu-west-1)
$0.10 per 1000 batch predictions
$0.0001 per real-time prediction
https://aws.amazon.com/machine-learning
16. Case study: BuildFax
“Amazon Machine Learning
democratizes the process
of building predictive
models. It's easy and fast to
use, and has machine-
learning best practices
encapsulated in the
product, which lets us
deliver results significantly
faster than in the past”
Joe Emison, Founder &
Chief Technology Officer
https://aws.amazon.com/solutions/case-studies/buildfax/
17. DEMO #2
Demo gods, I know I’m pushing it,
but please don’t let me down now
Load data from Amazon Redshift
Train and evaluate a regression model with Amazon ML
Create a real-time prediction API
Perform real-time prediction from Java app