Charles is a Lead ML platforms engineer at MavenCode. He has well over 15 years of experience building large-scale, distributed applications. Topic: Enterprise MLOps in Practice. How to efficiently get your Machine Learning Models from Notebooks to Production!
GDG Cloud Southlake #2 Jez Humble DevOps Transformation:Building & Scaling H...James Anderson
Our speaker, Jez Humble, is co-author of several books on software including Shingo Publication Award winner Accelerate, The DevOps Handbook, Lean Enterprise, and Jolt Award winner Continuous Delivery. Jez's talk is titled "The DevOps Transformation: Building and Scaling High Performing Technology Organizations."
Explain DevOps To Me Like I’m Five: DevOps for ManagersGene Gotimer
Organizations and leaders are often supportive of DevOps, but they don’t always understand what DevOps is and what it will change. It isn’t a one-size-fits-all issue; different environments need different benefits from a DevOps transformation. Join Gene Gotimer as he explains the most important parts of understanding DevOps. We'll discuss how to determine what parts of DevOps your organization needs to concentrate on first and how you should measure improvement. This session boils DevOps down to its most basic parts and makes sure you have a foundation for understanding how to make it work for your situation and organization.
These slides, based on the webinar, provide insights into DevOps research, including:
- How fast is DevOps being adopted and what is the sequencing and actual/planned penetration of DevOps capabilities across 20 markets?
- Deep insight into the highest growth areas of DevOps which include value stream management (VSM), operational analytics (OA), and test environment management (TEM).
- What strategies and approaches are enterprises leveraging to address their VSM, POA, and TEM needs?
- Where is the DevOps market headed, and what are likely to be the next important growth areas?
SRE (service reliability engineer) on big DevOps platform running on the clou...DevClub_lv
SRE (service reliability engineer). The talk is to explain the SRE philosophy and the principles of production engineering and operations in clouds.
(Language – English)
Pavlo is ADOP (Accenture DevOps Platform) Service Reliability Team Lead, SRE practitioner. Has more then 18 years of IT experience in Ops and Dev.
SQL Server 2012 is the most crucial release of SQL Server to-date. In this slideshow, you'll see how SQL Server 2012 supports mission critical applications 24x7 and gives significant insight into business operations. Presented by Subhash Jawahrani of Microsoft to the Silicon Valley SQL Server User Group in March 2012.
You'll learn about:
* Mission Critical Apps
* New Business Intelligence features
* Improving business agility with Cloud computing
Software Alliance - Hire Remote Developers Team EuropeSoftware Alliance
Software Alliance is a full-service web and app development company that provides best solutions for your business. We are a team of highly experienced and skilled developers who can bring your idea to life. From design to development and team, we work to provide the best results. Our headquarter is in Denmark, but we have teams in Pakistan, the USA, UK, and UAE. Hire the best remote developers for whatever technology you want from Software Alliance.
We are offering complete solutions for your business needs. Whether you are looking for game development services, web or app development, or other digital solutions to promote your business, we are here to help you. Our expert blockchain development team is here if you want trading bots or your own cryptocurrency. Our developers have the experience of more than 5 years and they have successfully delivered hundreds of projects as we are working for 10+ years.
GDG Cloud Southlake #2 Jez Humble DevOps Transformation:Building & Scaling H...James Anderson
Our speaker, Jez Humble, is co-author of several books on software including Shingo Publication Award winner Accelerate, The DevOps Handbook, Lean Enterprise, and Jolt Award winner Continuous Delivery. Jez's talk is titled "The DevOps Transformation: Building and Scaling High Performing Technology Organizations."
Explain DevOps To Me Like I’m Five: DevOps for ManagersGene Gotimer
Organizations and leaders are often supportive of DevOps, but they don’t always understand what DevOps is and what it will change. It isn’t a one-size-fits-all issue; different environments need different benefits from a DevOps transformation. Join Gene Gotimer as he explains the most important parts of understanding DevOps. We'll discuss how to determine what parts of DevOps your organization needs to concentrate on first and how you should measure improvement. This session boils DevOps down to its most basic parts and makes sure you have a foundation for understanding how to make it work for your situation and organization.
These slides, based on the webinar, provide insights into DevOps research, including:
- How fast is DevOps being adopted and what is the sequencing and actual/planned penetration of DevOps capabilities across 20 markets?
- Deep insight into the highest growth areas of DevOps which include value stream management (VSM), operational analytics (OA), and test environment management (TEM).
- What strategies and approaches are enterprises leveraging to address their VSM, POA, and TEM needs?
- Where is the DevOps market headed, and what are likely to be the next important growth areas?
SRE (service reliability engineer) on big DevOps platform running on the clou...DevClub_lv
SRE (service reliability engineer). The talk is to explain the SRE philosophy and the principles of production engineering and operations in clouds.
(Language – English)
Pavlo is ADOP (Accenture DevOps Platform) Service Reliability Team Lead, SRE practitioner. Has more then 18 years of IT experience in Ops and Dev.
SQL Server 2012 is the most crucial release of SQL Server to-date. In this slideshow, you'll see how SQL Server 2012 supports mission critical applications 24x7 and gives significant insight into business operations. Presented by Subhash Jawahrani of Microsoft to the Silicon Valley SQL Server User Group in March 2012.
You'll learn about:
* Mission Critical Apps
* New Business Intelligence features
* Improving business agility with Cloud computing
Software Alliance - Hire Remote Developers Team EuropeSoftware Alliance
Software Alliance is a full-service web and app development company that provides best solutions for your business. We are a team of highly experienced and skilled developers who can bring your idea to life. From design to development and team, we work to provide the best results. Our headquarter is in Denmark, but we have teams in Pakistan, the USA, UK, and UAE. Hire the best remote developers for whatever technology you want from Software Alliance.
We are offering complete solutions for your business needs. Whether you are looking for game development services, web or app development, or other digital solutions to promote your business, we are here to help you. Our expert blockchain development team is here if you want trading bots or your own cryptocurrency. Our developers have the experience of more than 5 years and they have successfully delivered hundreds of projects as we are working for 10+ years.
As companies have adopted faster development methodologies a new constraint has emerged in the journey to digital transformation: data. Data has long been the neglected discipline, the weakest link in the tool chain, with provisioning times still counted in days, weeks, or even months. In addition, most companies are still using decades-old processes to manage and deploy database changes, further anchoring development teams.
Case Study: How The Home Depot Built Quality Into Software DevelopmentCA Technologies
This session will cover how The Home Depot built quality into its software development as it migrated from waterfall to agile delivery.
For more information on DevOps: Continuous Delivery, please visit: http://ow.ly/hAXz50g62ZM
To successfully implement continuous delivery in an enterprise, there are specific needs and obstacles which must be addressed. In this webinar, we’ll address the pain points that most enterprises face, and how they can be overcome.
Integrating SAP into DevOps Pipelines: Why and HowDevOps.com
Teams practicing DevOps don’t usually have to spend much time thinking about applications like SAP, and SAP often remains a DevOps-free zone that is resolutely difficult to change. But SAP systems enable critical operational processes and in an increasingly interconnected technology stack, need to adapt at high speed if a business is going to be truly agile.
DevOps expertise from outside SAP teams is helping to accelerate change in SAP so that digital transformation of products, processes and business models isn’t held back by dependence on slow, unresponsive ‘systems of record’. In this webinar we’ll look at why it’s important to include SAP in cross-application CI/CD pipelines, and how to do so. Join us to learn:
Why DevOps teams should care about SAP
Key SAP differences that DevOps teams need to understand
How to get started with DevOps for SAP and successfully integrate SAP into wider DevOps pipelines
Real-world examples of SAP DevOps adoption
How We Do DevOps at Walmart: OneOps OSS Application Lifecycle Management Plat...WalmartLabs
Recently, Dr. Qingsong Zhang spoke at a Meetup about how Walmart is using DevOps.
Within this slide deck, you'll learn about our DataOps, DevOps and OneOps, an application lifecycle management (ALM), and open source DevOps platform for cloud which was developed by Walmart Labs.
Feel free to follow us on Twitter: @one_ops!
Contribute to One_Ops: www.oneops.com
Following on from the success of last year, this annual event for London's architect community will have architectural innovation as a theme this year, and particularly CQRS. At the DDD eXchange we will feature leading thinkers and architects who will share their experience and Eric Evans is the programme lead.
We will delve into the creation of the GSA's DevSecOps guide, progression towards componentized and lego-pieced ATO's (leveraging reusable Infrastructure and Configuration as-Code modules), Cloud.gov "Heroku for government", "how to" be Cloud agnostic, and more.
Our DevSecOps meetup:
https://www.meetup.com/DevSecOps-NoVA
The Handbook:
https://tech.gsa.gov/guides/dev_sec_ops_guide/
Our speakers group:
https://handbook.tts.gsa.gov/tech-portfolio/
His team's areas of responsibility:
https://digital.gov/services/
Integrating DevOps and ALM tools to speed deliveryTasktop
Test and build automation are important pieces of your DevOps toolchain, but these tools need to be integrated with your issue trackers and test management tools in order to optimize your software delivery.
Learn how to:
* Create defects in HPE Quality Center automatically when a Selenium test fails
* Update JIRA issues with build fail/pass information from Jenkins
* Create visibility and traceability across your value stream with data from all of your tools
<p>From <a href="https://en.wikipedia.org/wiki/Site_reliability_engineering" target="_blank">Wikipedia</a>: Site reliability engineering (SRE) is a discipline that incorporates aspects of software engineering and applies that to operations whose goals are to create ultra-scalable and highly reliable software systems.<p>
<p>Over the past year Acquia has built their own SRE team to help their products and services scale with the demand of our growing number of customers. We wish to share our experience so that others are enabled to do the same and reap the rewards.</p>
<p>This presentation will discuss how the SRE team came about at Acquia, what achievements we have made so far, and the lessons we have learned along the way. We will then show the steps on how to introduce SRE to your workplace so you can deliver more reliable and scalable services to your customers! We will specifically cover:</p>
<ul>
<li>SRE's basic concepts and history from Google</li>
<li>The management support you will need to get started</li>
<li>Introducing the idea of service level objectives and error budgets</li>
<li>Operational Responsibility Assessments as a tool to measure risk</li>
<li>Creating a Launch Readiness Checklist to standardize and improve product launches</li>
<li>Finding ideal candidates for your SRE team</li></ul>
<p>The intended audience are software engineers, system administrators, and managers that have a desire to improve how they do their work and how their products/services perform.</p>
MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.
As companies have adopted faster development methodologies a new constraint has emerged in the journey to digital transformation: data. Data has long been the neglected discipline, the weakest link in the tool chain, with provisioning times still counted in days, weeks, or even months. In addition, most companies are still using decades-old processes to manage and deploy database changes, further anchoring development teams.
Case Study: How The Home Depot Built Quality Into Software DevelopmentCA Technologies
This session will cover how The Home Depot built quality into its software development as it migrated from waterfall to agile delivery.
For more information on DevOps: Continuous Delivery, please visit: http://ow.ly/hAXz50g62ZM
To successfully implement continuous delivery in an enterprise, there are specific needs and obstacles which must be addressed. In this webinar, we’ll address the pain points that most enterprises face, and how they can be overcome.
Integrating SAP into DevOps Pipelines: Why and HowDevOps.com
Teams practicing DevOps don’t usually have to spend much time thinking about applications like SAP, and SAP often remains a DevOps-free zone that is resolutely difficult to change. But SAP systems enable critical operational processes and in an increasingly interconnected technology stack, need to adapt at high speed if a business is going to be truly agile.
DevOps expertise from outside SAP teams is helping to accelerate change in SAP so that digital transformation of products, processes and business models isn’t held back by dependence on slow, unresponsive ‘systems of record’. In this webinar we’ll look at why it’s important to include SAP in cross-application CI/CD pipelines, and how to do so. Join us to learn:
Why DevOps teams should care about SAP
Key SAP differences that DevOps teams need to understand
How to get started with DevOps for SAP and successfully integrate SAP into wider DevOps pipelines
Real-world examples of SAP DevOps adoption
How We Do DevOps at Walmart: OneOps OSS Application Lifecycle Management Plat...WalmartLabs
Recently, Dr. Qingsong Zhang spoke at a Meetup about how Walmart is using DevOps.
Within this slide deck, you'll learn about our DataOps, DevOps and OneOps, an application lifecycle management (ALM), and open source DevOps platform for cloud which was developed by Walmart Labs.
Feel free to follow us on Twitter: @one_ops!
Contribute to One_Ops: www.oneops.com
Following on from the success of last year, this annual event for London's architect community will have architectural innovation as a theme this year, and particularly CQRS. At the DDD eXchange we will feature leading thinkers and architects who will share their experience and Eric Evans is the programme lead.
We will delve into the creation of the GSA's DevSecOps guide, progression towards componentized and lego-pieced ATO's (leveraging reusable Infrastructure and Configuration as-Code modules), Cloud.gov "Heroku for government", "how to" be Cloud agnostic, and more.
Our DevSecOps meetup:
https://www.meetup.com/DevSecOps-NoVA
The Handbook:
https://tech.gsa.gov/guides/dev_sec_ops_guide/
Our speakers group:
https://handbook.tts.gsa.gov/tech-portfolio/
His team's areas of responsibility:
https://digital.gov/services/
Integrating DevOps and ALM tools to speed deliveryTasktop
Test and build automation are important pieces of your DevOps toolchain, but these tools need to be integrated with your issue trackers and test management tools in order to optimize your software delivery.
Learn how to:
* Create defects in HPE Quality Center automatically when a Selenium test fails
* Update JIRA issues with build fail/pass information from Jenkins
* Create visibility and traceability across your value stream with data from all of your tools
<p>From <a href="https://en.wikipedia.org/wiki/Site_reliability_engineering" target="_blank">Wikipedia</a>: Site reliability engineering (SRE) is a discipline that incorporates aspects of software engineering and applies that to operations whose goals are to create ultra-scalable and highly reliable software systems.<p>
<p>Over the past year Acquia has built their own SRE team to help their products and services scale with the demand of our growing number of customers. We wish to share our experience so that others are enabled to do the same and reap the rewards.</p>
<p>This presentation will discuss how the SRE team came about at Acquia, what achievements we have made so far, and the lessons we have learned along the way. We will then show the steps on how to introduce SRE to your workplace so you can deliver more reliable and scalable services to your customers! We will specifically cover:</p>
<ul>
<li>SRE's basic concepts and history from Google</li>
<li>The management support you will need to get started</li>
<li>Introducing the idea of service level objectives and error budgets</li>
<li>Operational Responsibility Assessments as a tool to measure risk</li>
<li>Creating a Launch Readiness Checklist to standardize and improve product launches</li>
<li>Finding ideal candidates for your SRE team</li></ul>
<p>The intended audience are software engineers, system administrators, and managers that have a desire to improve how they do their work and how their products/services perform.</p>
MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.
When it comes to Large Scale data processing and Machine Learning, Apache Spark is no doubt one of the top battle-tested frameworks out there for handling batched or streaming workloads. The ease of use, built-in Machine Learning modules, and multi-language support makes it a very attractive choice for data wonks. However bootstrapping and getting off the ground could be difficult for most teams without leveraging a Spark cluster that is already pre-provisioned and provided as a managed service in the Cloud, while this is a very attractive choice to get going, in the long run, it could be a very expensive option if it’s not well managed.
As an alternative to this approach, our team has been exploring and working a lot with running Spark and all our Machine Learning workloads and pipelines as containerized Docker packages on Kubernetes. This provides an infrastructure-agnostic abstraction layer for us, and as a result, it improves our operational efficiency and reduces our overall compute cost. Most importantly, we can easily target our Spark workload deployment to run on any major Cloud or On-prem infrastructure (with Kubernetes as the common denominator) by just modifying a few configurations.
In this talk, we will walk you through the process our team follows to make it easy for us to run a production deployment of our Machine Learning workloads and pipelines on Kubernetes which seamlessly allows us to port our implementation from a local Kubernetes set up on the laptop during development to either an On-prem or Cloud Kubernetes environment
Navigating the ML Pipeline Jungle with MLflow: Notes from the Field with Thun...Databricks
Plumbing has been a key focus of modern software engineering, with our API/services/containers/devops driven landscape so it may come as a surprise that plumbing is where AI projects tend to fail. But it is precisely because our modern software development focuses on decoupled plumbing that we have struggled to handle the rise of AI.
Specifically, companies are able to use AI effectively when they are able to create end-to-end AI model factories that explicitly account for coupling between data, models, and code.
In this talk, I will be walking through what a model factory is and how MLFlow’s design supports the creation of end-to-end model factories as well as sharing best practices I’ve observed helping customers from startups to Fortune 50s create, productionize, and scale end-to-end ML pipelines, and watching those pipelines produce serious, game changing business impact.
MLOps and Data Quality: Deploying Reliable ML Models in ProductionProvectus
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
BigQuery ML - Machine learning at scale using SQLMárton Kodok
With BigQuery ML, you can build machine learning models without leaving the data warehouse environment and training it on massive datasets. We are going to demonstrate how to build, train, eval and predict, your own scalable machine learning models using standard SQL language in Google BigQuery.
We will see how can we use CREATE MODEL sql syntax to build different models such as:
-Linear regression
-Multiclass logistic regression for classification
-K-means clustering
-Import TensorFlow models for prediction in BigQuery
We will see how we can apply these models on tabular data in retail and marketing use cases.
Models are trained and accessed in BigQuery using SQL — a language data analysts know. This enables business decision making through predictive analytics across the organization without leaving the query editor.
Arocom is a consulting and solution engineering company with expertise in providing engineering services for AI & Machine Learning, Data Operations & Analytics, MLOps and Cloud Computing.
Our clients include companies within biotech, drug discovery, therapeutics, manufacturing, retail and startups. Our consultants are best in their skills and offer hands-on talent to our clients in achieving their goals.
Applying BigQuery ML on e-commerce data analyticsMárton Kodok
With BigQuery ML, you can build machine learning models without leaving the database environment and training it on massive datasets. We are going to demonstrate common marketing Machine Learning use cases we do at REEA.net to build, train, eval and predict, your own scalable machine learning models using SQL language in Google BigQuery and to address the following use cases:
Customer Segmentation
Customer Lifetime Value (LTV) prediction
Conversion/Purchase prediction
The audience will get first hand experience how to write CREATE MODEL sql syntax to build machine learning models such as:
Multiclass logistic regression for classification
K-means clustering
Import TensorFlow models for prediction in BigQuery
Models are trained and accessed in BigQuery using SQL — a language data analysts know. This enables business decision making through predictive analytics across the organization without leaving the query editor
While the adoption of machine learning and deep learning techniques continue to grow, many organizations find it difficult to actually deploy these sophisticated models into production. It is common to see data scientists build powerful models, yet these models are not deployed because of the complexity of the technology used or lack of understanding related to the process of pushing these models into production.
As part of this talk, I will review several deployment design patterns for both real-time and batch use cases. I’ll show how these models can be deployed as scalable, distributed deployments within the cloud, scaled across hadoop clusters, as APIs, and deployed within streaming analytics pipelines. I will also touch on topics related to security, end-to-end governance, pitfalls, challenges, and useful tools across a variety of platforms. This presentation will involve demos and sample code for the the deployment design patterns.
Building machine learning muscle in your team & transitioning to make them do machine learning at scale. We also discuss about Spark & other relevant technologies.
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...James Anderson
Do you know The Cloud Girl? She makes the cloud come alive with pictures and storytelling.
The Cloud Girl, Priyanka Vergadia, Chief Content Officer @Google, joins us to tell us about Scaleable Data Analytics in Google Cloud.
Maybe, with her explanation, we'll finally understand it!
Priyanka is a technical storyteller and content creator who has created over 300 videos, articles, podcasts, courses and tutorials which help developers learn Google Cloud fundamentals, solve their business challenges and pass certifications! Checkout her content on Google Cloud Tech Youtube channel.
Priyanka enjoys drawing and painting which she tries to bring to her advocacy.
Check out her website The Cloud Girl: https://thecloudgirl.dev/ and her new book: https://www.amazon.com/Visualizing-Google-Cloud-Illustrated-References/dp/1119816327
Democratizing AI/ML with GCP - Abishay Rao (Google) at GoDataFest 2019GoDataDriven
Every company today is talking about AI/ML, but when most companies talk about AI/ML in their transformation journey, you hear terms like Proof of Concept, Feasibility Study, Pilot, A/B Test. We are at the peak of AI's hype, but only 12% of enterprises have deployed AI in production. Google aims to make big data processing available for everyone, the possiblities of Big Query ML are endless: Marketing, retail, industrial and IoT, media, gaming, and so fort.
Why do the majority of Data Science projects never make it to production?Itai Yaffe
María de la Fuente (Solutions Architect Manager for IMEA) @ Databricks
While most companies understand the value creation of leveraging data and are taking on board an AI strategy, only 13% of the data science projects make it to production successfully.
Besides the well-known skills gap in the market, we need to level up our end-to-end approach and cover all aspects involved when working with AI.
In this session, we will discuss the main obstacles to overcome and how we can avoid the major pitfalls to ensure our data science journey becomes successful.
BigQuery ML - Machine learning at scale using SQLMárton Kodok
With BigQuery ML, you can build machine learning models without leaving the data warehouse environment and training it on massive datasets. We are going to demonstrate how to build, train, eval and predict, your own scalable machine learning models using standard SQL language in Google BigQuery.
We will see how can we use CREATE MODEL sql syntax to build different models such as:
Linear regression
Multiclass logistic regression for classification
K-means clustering
Import TensorFlow models for prediction in BigQuery
We will see how we can apply these models on tabular data in retail and marketing use cases.
Models are trained and accessed in BigQuery using SQL — a language data analysts know. This enables business decision making through predictive analytics across the organization without leaving the query editor.
Introduction to Machine Learning - WeCloudDataWeCloudData
In this talk, WeCloudData introduces the lifecycle of machine learning and its tools/ecosystems. For more detail about WeCloudData's machine learning course please visit: https://weclouddata.com/data-science/
Similar to GDG Cloud Southlake #3 Charles Adetiloye: Enterprise MLOps in Practice (20)
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GDG Cloud Southlake 32: Kyle Hettinger: Demystifying the Dark WebJames Anderson
It's important to remember that accessing the dark web can be risky and requires specialized skills and tools. Many organizations leverage threat intelligence companies that have a safe and legal way to monitor these areas and extract valuable information.
Let's shine some light on the Dark Web.
Kyle Hettinger from Recorded Future's Dark Web research team joins GDG Cloud Southlake joins to Demystify the Dark Web.
Kyle has been doing cybercrime investigations for over 10 years, and has collaborated with both public and private sector partners to identify, mitigate, and neutralize cybercriminals.
GDG Cloud Southlake 31: Santosh Chennuri and Festus Yeboah: Empowering Develo...James Anderson
GDG Cloud Southlake #31: Santosh Chennuri and Festus Yeboah: Empowering Developers: Gen AI's Impact on Productivity
In this interactive presentation and demo, we'll explore how Generative AI is revolutionizing the entire software development lifecycle (SDLC), empowering developers to work smarter, innovate faster, and deliver cutting-edge features to the market with unprecedented speed.
Santosh is the Lead Customer Engineer passionate about exploring the potential of Gen AI for enterprise clients. With a background in cloud migrations, DevOps, and application modernization, Santosh is committed to finding new ways to leverage generative AI for increased efficiency and problem-solving.
Festus is a Customer Engineer at Google Cloud, specializing in data and AI. He advises organizations on harnessing the potential of generative AI for innovation and enhanced customer experiences. With a strong background in data engineering and machine learning, Festus offers a unique perspective on improving developer productivity using GenAI solutions. Outside of work, he enjoys spending time with his family and is an avid fan of the Marvel Cinematic Universe.
#gdg #gdgcloudsouthlake #gdgcloud #google #genai #duetai #DeveloperProductivity #SDLC
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...James Anderson
Breeding 10x Developers with Developer Productivity Engineering
Sasquatch. Yeti. The Loch Ness Monster. The 10x Developer. You may think of these as mythical creatures that can’t possibly exist, but the 10x Organization is very real. In this session, Gradle’s Brian Demers will explain how a dedicated Developer Productivity Engineering (DPE) organization can breed 10x Developers. By reducing the toil, friction, and frustration of slow builds, flaky tests, and other avoidable failures, a DPE team enables a level of developer productivity that you may have thought impossible. Brian will help you explore DPE technologies, including build and test acceleration, failure analytics, and easily analyzed build records to show how to create an environment in which 10x Developers not only exist, but thrive.
Brian Demers is a Java Champion, Developer Advocate at Gradle and a PMC member for the Apache Shiro project. He spends much of his day contributing to OSS projects in the form of writing code, tutorials, blogs, and answering questions. Along with typical software development, Brian also has a passion for fast builds and automation. You can see the various topics he speaks on here.
Away from the keyboard, Brian is a beekeeper and can likely be found playing board games. You can find him on Twitter at @BrianDemers and most other places as ‘bdemers’.
GDG Cloud Southlake 29 Jimmy Mesta OWASP Top 10 for KubernetesJames Anderson
Given the growth and adoption of Kubernetes, a number of projects have been published in the OWASP community to help practitioners assess and secure the security of their containerized infrastructure including the recently released Top 10 for Kubernetes (https://owasp.org/www-project-kubernetes-top-ten/) led by KSOC CTO & Co-Founder, Jimmy Mesta. When adopting Kubernetes, we introduce new risks to our applications and infrastructure. The OWASP Kubernetes Top 10 is aimed at helping security practitioners, system administrators, and software developers prioritize risks around the Kubernetes ecosystem. The Top 10 is a prioritized list of these risks. In the future, we hope for this to be backed by data collected from organizations varying in maturity and complexity. This session will discuss the project in detail, examples for each of the risks in the list, and how you can get involved.
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...James Anderson
GDG Cloud Southlake #28: Brad Taylor and Shawn Augenstein: Old Problems in the New Frontiers of AI
• Brad discusses how decades-old laws and expanding regulation have new implications in the ML and Large Model age, and will touch on:
• Legal and Regulatory: Data usage rights, cautionary tale of stability.ai and Getty Images, EU's planned expansion of GDPR re models
• How Neural Networks, zero and one-shot learning, and LLMs have increased the need for better data governance, lineage management
• Shawn speaks on the coming "Data Renaissance"
• The New IP: Prompts and Internatl Interaction Data
• Where GenAI can be used right now and where it maybe shouldn't be used yet
• The Power of the Diversity of Insight
• What is making the future look bright!
Brad has been an intrapreneur and entrepreneur in data, AI, and IoT and has led teams in the creation of NLP, data products and predictive analytics for retention, churn, driver safety, traffic, CX and fleet risk. He has built solutions on global hyperscalers GCP, AWS, Azure, and IBM. Brad is a former founding partner at Tech Wildcatters, and worked with dozens of mobile, SaaS and AI start-ups, many of which became both job creators and profitable exits for TW investors. He is currently a Senior Manager in Pepsico's global Strategy and Transformation group, where he focuses on delivering AI/ML driven solutions.
Shawn Augenstein is a dynamic and highly experienced professional, who is driven by educating, providing equal access to technology and equitable access to information. Currently, Shawn serves as Principal Data & AI Consultant at CDW, where he develops the curriculum and architectures for understanding and furthering the use of AI, as well as developing solutions for both partners and clients. In his spare time, he enjoys exploring new frontiers of Diffusers, capturing moments through photography, and listening to music as a passionate melophile.
GDG SLK - Why should devs care about container security.pdfJames Anderson
Title: Why should developers care about container security?
Abstract: Container scanning tools, industry publications, and application security experts are constantly telling us about best practices for how to build our images and run our containers. Often these non-functional requirements seem abstract and are not described well enough for those of us that don’t have an appsec background to fully understand why they are important. In this session, we will go over several of the most common practices, show examples of how your workloads can be exploited if not followed and, most importantly, how to easily find and fix your Dockerfiles and deployment manifests (i.e. Kubernetes config's) before you commit your code.
Speaker: Eric is a 30+ year enterprise software developer, architect, and consultant with a focus on CI/CD, DevOps, and container-based solutions over the last decade. He is a Docker Captain, is certified in Kubernetes (CKA, CKAD, CKS), and has been a Docker user since 2013. As a Senior Developer Advocate at Snyk, Eric helps developers implement proactive and scalable security practices with a focus on container and cloud-native technologies.
Catch the video: https://youtu.be/lBNcUBdY-VM
GraphQL Insights Deck ( Sabre_GDG - Sept 2023).pdfJames Anderson
GraphQL - Industry insights on the rise of the supergraph
Exploring what we’ve learned from hundreds of organizations transforming their business and customer experiences with GraphQL & the supergraph.
In his talk and Q&A session, Dan Boerner will share insights and best practices from his experience working with hundreds of companies working to unblock their teams and backlogs with the supergraph—a new layer of the stack. He’ll share real-world examples to explore why GraphQL and its architectural advantages must be coupled with leadership, vision, team empowerment, and mindset shifts to truly transform the way enterprises build, deliver and organize themselves to create digital products. As Apollo’s Graph Champion, Dan leads Apollo’s community of 800+ GraphQL leaders from 350 companies. Before joining Apollo, Dan led Expedia Group’s effort to radically accelerate the delivery of improved customer experiences with a company-wide supergraph.
Dan is passionate about helping graph champions harness the transformative power of the supergraph to improve product development and digital customer experiences. At Apollo, he leads a community of hundreds of GraphQL champions working to drive transformation within their organizations. He joined Apollo after a long tenure at Expedia Group where he led the effort to create a company-wide supergraph transforming product development and delivery, and enabling the organization to roll out their new trips platform in 1 year instead of 3.
https://youtu.be/0Vucl1qVecM
GDG Cloud Southlake #25: Jacek Ostrowski & David Browne: Sabre's Journey to ...James Anderson
GDG Cloud Southlake #25: Ostrowski/Browne: Sabre's Journey to the Cloud
Brief overview of Sabre's journey from private datacenters, through multi-cloud to mono-cloud and beyond. Review of the drivers, expectations, and results with plenty of time for Q&A.
Jacek Ostrowski
Sabre
Sr Director Platform Engineering
In 1998 Jacek received MS in Computer Science from Jagiellonian University, Poland and started a developer career.
From 2001 to 2007 he honed his java and architecture skills while building systems supporting data warehouses with Asseco Poland.
In 2007 joined Sabre as a senior java engineer, and a few years later moved to enterprise architecture. After a few years as an EA, he started championing platform product management and took the platform product manager position. In 2018 took a leadership position over a team of platform product managers.
From 2020 Jacek leads Platform Engineering and uses his developer experience and product mindset to make Sabre's developers happier and more productive.
David Browne
Sabre
Senior Principal SRE Architecture
Graduated from the University of Waterloo with Joint degrees in Computer Science and Actuarial Science. Has spent 20 years doing software development and Enterprise Architecture work with IBM, Travelocity, and Sabre.
Experienced in implementing enterprise DevOps solutions to deploy software into on-prem and cloud-based environments such as AWS, Azure and GCP.
Currently working as an SRE architect with Sabre where he is an advocate for designing and implementing enterprise DevOps solutions that can run at scale. Enabling hundreds of teams to get their software products to market faster and more efficiently while meeting today’s current reliability regulatory and security requirements.
https://gdg.community.dev/events/details/google-gdg-cloud-southlake-presents-gdg-cloud-southlake-25-ostrowskibrowne-sabres-journey-to-the-cloud/cohost-gdg-cloud-southlake
This is the white paper behind the GDG Cloud Southlake #24 presentation by Arty Starr:
Enabling Powerful Software Insights by Visualizing Friction and Flow
In an Agile software development process, a software team will typically meet on a regular basis in a “retrospective meeting” to reflect on the challenges faced by the team and opportunities for improvement. On the surface, this challenge might seem straight-forward, but modern software projects are complex endeavors, and developers are human – identifying what’s most important in a complex sociotechnical system is a task humans struggle to do well. What if developers had tools that recorded and helped them explore their historical experiences with the code, and they could identify hotspots of team friction, worthy of discussion, based on empirical data? This talk will explore the possibility and impact of such tools through a design fiction and working prototype of an Augmented Reality (AR) Code Planetarium powered by FlowInsight developer tools.
Arty Starr, PhD student, University of Victoria & Founder, FlowInsight
Arty is a recognized Flow Experience expert, researcher, speaker and thought leader, and the author of Idea Flow. This expertise, along with her experience as a former CTO and software engineer inspired Arty’s mission to improve the efficiency and morale of engineering teams, culminating in her founding FlowInsight.
Arty teaches system models for better understanding the Flow Experience of software development, and the practice of using Flow Metrics to systematically optimize programming flow. “Flow as a practice” is the art of getting in and staying in flow state to optimize productivity.
The company she founded, FlowInsight, is on a mission to bring back joy to our everyday work.
GDG Cloud Southlake #24: Arty Starr: Enabling Powerful Software Insights by V...James Anderson
Enabling Powerful Software Insights by Visualizing Friction and Flow
In an Agile software development process, a software team will typically meet on a regular basis in a “retrospective meeting” to reflect on the challenges faced by the team and opportunities for improvement. On the surface, this challenge might seem straight-forward, but modern software projects are complex endeavors, and developers are human – identifying what’s most important in a complex sociotechnical system is a task humans struggle to do well. What if developers had tools that recorded and helped them explore their historical experiences with the code, and they could identify hotspots of team friction, worthy of discussion, based on empirical data? This talk will explore the possibility and impact of such tools through a design fiction and working prototype of an Augmented Reality (AR) Code Planetarium powered by FlowInsight developer tools.
Arty Starr, PhD student, University of Victoria & Founder, FlowInsight
Arty is a recognized Flow Experience expert, researcher, speaker and thought leader, and the author of Idea Flow. This expertise, along with her experience as a former CTO and software engineer inspired Arty’s mission to improve the efficiency and morale of engineering teams, culminating in her founding FlowInsight.
Arty teaches system models for better understanding the Flow Experience of software development, and the practice of using Flow Metrics to systematically optimize programming flow. “Flow as a practice” is the art of getting in and staying in flow state to optimize productivity.
The company she founded, FlowInsight, is on a mission to bring back joy to our everyday work.
GDG Cloud Southlake #23:Ralph Lloren: Social Engineering Large Language ModelsJames Anderson
Each day, the world continues to get smaller and smaller. The Cybersecurity and Data Science domains have converged, and we are now at a crossroads. Soft skills and effective communication are in higher demand than ever, with new roles such as Prompt Engineering being created. So, where do humans go from here?
Dive with us into the hidden depths of Social Engineering, a topic often considered taboo to explore. We must have the hard conversations now to tackle the Fear, Uncertainty, and Doubt that AI/ML brings. Is resistance really futile? No matter what, have fun during the event, and be sure to join us afterward for the social networking hour so we can practice our verbal judo on each other.
GDG Cloud Southlake no. 22 Gutta and Nayer GCP Terraform Modules Scaling Your...James Anderson
GCP Terraform Modules: Scaling Your Infrastructure the easy way
With GCP Terraform Modules, you can take advantage of pre-built modules that simplify the process of creating and managing GCP resources, such as virtual machines, load balancers, databases, and more. These modules are designed to be reusable, scalable, and customizable, allowing you to quickly and easily deploy complex infrastructure configurations with just a few lines of code.
Whether you're just getting started with GCP or you're looking for a more efficient way to manage your infrastructure, GCP Terraform Modules are a great way to streamline your operations and scale your infrastructure with ease. Join us as we cover details on why to use modules, how to use and where to find more helpful resources.
Anita Gutta is Cloud Infrastructure Engineer in Google Cloud Professional Services Organization (PSO). She provides technical guidance to customers adopting Google Cloud Platform services. She works closely with clients to understand their business needs and recommends the best cloud solutions to meet those needs. She has hands-on terraform experience and leads the SME TF Community in Google Cloud. Prior to Google Anita worked in the IT industry for 25 years, the majority focused in the finance sector.
Imran Nayer is a Senior Technical Solutions Consultant at Google Cloud Professional Services. He has been working on Google Cloud since 2019. Helped companies in the healthcare, financial, and retail sectors with projects including cloud foundation, migration, and automation. He is a regular contributor to the official GCP Terraform module, aka the Cloud Foundation Toolkit. He developed the Cloud Armor Security Module and several other CFT submodules.
GDG Cloud Southlake #21:Alexander Snegovoy: Master Continuous Resiliency in C...James Anderson
Mastering Continuous Resiliency in Cloud: Chaos Engineering
No one likes downtime. It can be detrimental in today’s competitive environment. It isn’t cheap either. Many companies have been using traditional DR strategies. However, their testing is costly, limited, and complex. In the modern agile environment, the latest DR exercise becomes invalid not long after it is done and there’s a greater variety of disruptions that can occur. In this demo, we’ll explore how to use chaos engineering techniques to: quantify reliability and resiliency, gain valuable insights, and build systems that can withstand the unexpected. By applying these practices, you can gain confidence, prove resiliency, and be sure you are ready to face the unexpected.
Our speaker is Alexander Snegovoy, Lead of DevOps & Cloud Center of Competence at DataArt.
Alex spearheads DataArt’s drive toward innovation, with more than 10 years of professional experience across the financial services, healthcare, travel, and IoT industries. After joining DataArt as a software engineer in 2016, he became a leading member of the DevOps & Cloud Center of Competence. His role also includes identifying and communicating technology trends, cementing alliances and strategic partnerships with other companies, and coaching and mentoring new talent.
There is a “dark side” to Kubernetes that makes it difficult to ensure the desired performance and resilience of cloud-native applications, while also keeping their costs under control. Indeed, the combined effect of Kubernetes resource management mechanisms and application runtime heuristics may cause serious performance and resilience risks. See Akamas' AI-powered optimizations solve this!
GDG Cloud Southlake #19: Sullivan and Schuh: Design Thinking Primer: How to B...James Anderson
Brian Sullivan and J Schuh GDG Cloud Southlake #19: Design Thinking Primer: How to Build Better Ideas
Video and other items from the event are here: https://gdg.community.dev/events/details/google-gdg-cloud-southlake-presents-gdg-cloud-southlake-19-sullivan-and-schuh-design-thinking-primer-how-to-build-better-ideas/
GDG Cloud Southlake #18 Yujun Liang Crawl, Walk, Run My Journey into Google C...James Anderson
Crawl, Walk, Run. An exciting journey from 0 to fully certified on Google Cloud. A story of inspiration, entertainment, and struggle. You don't want to miss it.
@YujunLiang is an Associate Director at Accenture. He started his Google Cloud journey in 2017 and had been on many challenging projects including leading roles on some of them. His expertise spans Cloud Infrastructure and Data analytics. Currently, Yujun works as the cloud architect on a Data Analytics Platform and helps the team remove roadblocks in networking and security.
He is also known as the certification king on LinkedIn. He holds all 11 Google Cloud certifications and all 14 AWS certifications. His dedication to learning has created a sensation.
Yujun is a Google Cloud Champion Innovator with a specialization in Data Analytics, Databases, Security, and Networking.
Video on YouTube: https://youtu.be/RkMCn6ukfZg
Check out past and future GDG Cloud Southlake events: https://gdg.community.dev/gdg-cloud-s...
#cloud #gdg #gdgcloudsouthlake #sabre #google #careerjourney
GDG Cloud Southlake #17: Meg Dickey-Kurdziolek: Explainable AI is for EveryoneJames Anderson
If Artificial Intelligence (AI) is a black-box, how can a human comprehend and trust the results of Machine Learning (ML) alogrithms? Explainable AI (XAI) tries to shed light into that AI black-box so humans can trust what is going on. Our speaker Meg Dickey-Kurdziolek is currently a UX Researcher for Google Cloud AI and Industry Solutions, where she focuses her research on Explainable AI and Model Understanding. Recording of the presentation: https://youtu.be/6N2DNN_HDWU
GDG Cloud Southlake #15: Mihir Mistry: Cybersecurity and Data Privacy in an A...James Anderson
Addressing Cybersecurity and Data Privacy concerns in the evolving world of AR, VR and Metaverse.
Mihir Mistry is a leader in Strategy, Governance, Risk and Compliance. Leads the delivery service lines for Controls & Compliance, Risk & Advisory, Cloud and Architecture. Takes a very programmatic approach to solving and delivering security based projects. Proven business and entrepreneurial skills to deliver custom, highly visible projects in front of the C-suite and Board of Directors. Diverse knowledge base and framework expertise that includes NIST, HIPAA, HITRUST, CIS, GLBA, ISO, GDPR, CLOUD, PCI and others. Global experience across North America, Europe, Asia and Australia.
https://gdg.community.dev/events/details/google-gdg-cloud-southlake-presents-gdg-cloud-southlake-15-mihir-mistry-cybersecurity-and-data-privacy-in-an-arvr-metaverse-world/
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
2. About MavenCode
MavenCode Confidential and Proprietary
MavenCode is a Artificial Intelligence Solutions company located Southlake, Texas - We do training, product
development and consulting services with specialization in
● Provisioning Scalable AI and ML platforms - OnPrem and in the Cloud
● Deployment & Development of Machine Learning Platforms - OnPrem and in the Cloud
● Enterprise Feature Store Development and Management
● Model Management and Governance
● Streaming Data Analytics and Edge IoT Model Deployments
● Document Understanding and Natural Language Processing with Artificial Intelligence
3. Industry Verticals We Serve
Retail Industry
● Recommendation Engines
● Customer Management
● Demand Analysis and Planning
● Logistics and Supply Management
Insurance Industry
● AI Infrastructure Tooling
● Claims Analysis and Processing
● Document Processing
● Damage Detection and Identification
Automotive Industry
● AI infrastructure Tooling
● Near Real Time Car Telemetry Analysis
● Preemptive maintenance
recommendation
MavenCode Confidential and Proprietary
Healthcare Industry
● Medical insurance claim analysis
● X-ray image analysis and diagnostics
● Data Driven decision making enablement
Energy Industry
● Capacity Planning and Demand
Forecasting
● Preemptive Equipment Maintenance
Travel & Hospitality Industry
● Planning and Logistics
● Customer Recommendations
● Logistics, Planning and Forecasting
Telecom Industry
● Utilization Forecasting
● Churn Rate Analysis
● Preemptive Maintenance of Equipments
Agriculture Industry
● Precision Farming
● Mechanical Utilization Rate and Planning
● Capacity Planning
6. Agenda
MavenCode Confidential and Proprietary
1 Overview of Machine Learning Ops
2 MLOps Roles
3 MLOps Landscape
4 Discuss a Use Case
5 Questions and Answers
10. MLOps is not easy!
MavenCode Confidential and Proprietary
Launching a rocket is easy, but the ongoing
operations of guiding it successfully into Space
afterward is hard
11. MavenCode Confidential and Proprietary
“It took me 3 weeks to develop the model. It’s been > 11 months, and it’s still
not deployed”
@ginablaber
“On average, 40% of companies said it takes more than a month to deploy
ML models into production”
thenewstack.io
12. MavenCode Confidential and Proprietary
Machine Learning Operations, or MLOps, helps simplify the processes involved in the deployment
of machine learning models between operations team and machine learning researchers or data
scientists in the organization
What is Machine Learning Operations?
13. MavenCode Confidential and Proprietary
● The goal is to standardize and streamline the Machine Learning Life Cycle management
● Is a critical component of any successful Machine Learning project in the Enterprise
● Organizations generate long term value and mitigate risk associated with Machine Learning
projects
So we can say with MLOps ...
14. MavenCode Confidential and Proprietary
Challenges In Enterprise ML
Reproducibility
● Not Easy to Reproduce ML Model Output
on each iterative runs
● Constantly Changing Training Data
● Consistent Environment Configuration
Issues
Reusability
● Training Pipelines are not
Componentized for Reusability
● No well defined way of doing Model
versioning and tagging
● Collaboration and sharing of source
code is not well defined
Manageability
● Managing model deployment and serving
between environments is difficult
● Versioning and Tracking model artifacts is
very difficult and complex
● No defined way to visually track updates
and changes
Automation
● A lot of deployment process is still
manual
● Steps needed to update model
parameters are not not automated
● Most data science teams are not
equipped with the right knowledge to
take models to production
16. What People Think about Machine Learning
Machine Learning Code
MavenCode Confidential and Proprietary
17. Hidden Technical Debt of ML Deployment
Data Verification
Configuration
Feature
Extraction
Data Validation
Machine Resource
Management
Serving
Infrastructure
Monitoring
Analysis Tool
Machine Learning Code
MavenCode Confidential and Proprietary
18. ● Ensure a scalable and
flexible environment for ML
model pipelines
● Introduce new technologies
that improve ML model
performance in production
● Identify bottlenecks in the
production system and
pinpoint solutions for long
term improvements
ML Architects
● Analyze initial business
goals and model
outcomes
● Minimize overall risk as a
result of ML models in
production
● Ensure compliance with
internal and external
requirements before
pushing ML models to
production
Model Risk
Managers/Auditors
● Conduct and build
operational systems
● Test systems for security,
performance and
availability
● CI/CD pipeline
management
DevOps
● Integrate ML models in
company’s applications
● Ensure seamless working of
ML models with non-ML
based applications
● Maintain functional ML
models in production
ML Engineers
● Identify the right data for a
project
● Optimize the retrieval and
use of data to power ML
models
● Resolve underlying issues in
data pipelines
Data Engineers
● Build models that address
business needs
● Deliver operationalizable
models for production
environment
● Access model quality
Data Scientists
MavenCode Confidential and Proprietary
● Provide business
questions for framing ML
models
● Define business KPIs to
be achieved
● Evaluate Model
performance
Subject Matter Experts
MLOps Roles and Responsibilities
19. Data scientists
Model risk
managers/auditors
Subject Matter
Experts
Business Questions
Data Acquisition Feature Engineering
Data Preparation
Model
Training/Experimentation
Model Evaluation and
Comparison
Develop Models
Runtime
Environment
Risk Evaluation
QA
Scabilibility
Containerization
Continuous
Integration
Prepare for
Production
Subject Matter
Experts
Development
to Production
Logging/Alerting
Input drift tracking
Online Evaluation
Monitoring &
Feedback
Performance Drift
MavenCode Confidential and Proprietary
DevOps Data Engineers
Data Engineers
Data scientists
Software Engineers
ML Architects
Data Engineers
DevOps
1
2
3
4
ML Team Workflow
Model risk
managers/auditors
21. Machine Learning Pipeline
MavenCode Confidential and Proprietary
Data Extraction
Data Preparation &
Analysis
Data QA and Validation
Feature Engineering
Streaming Source
Batch Job Operations
Datasource with
Streaming sources like
MQTT, Kafka, Pubsub etc
Batch Operations on
Databases, FileStorage,
Distributed Storage etc
Model
Training/Validation
Model Training
Model Serving
Model Versioning
Prediction Service
Monitoring
Logging
App
Integration
Deployment / Inferencing
22. Typical ML Engineer or Data Scientist Workflow
Data
Sourcing
Pre
Processing
Feature
Engineering
Model
Training /
Evaluation
Model Scoring
/Management
Model
Inferencing
Azure Storage
Google Storage
AWS S3 Storage
Raw Data Transformation Processed Data
Storage Compute
GCP Vertex AWS SageMaker Azure ML
Data Scientist / ML Engineers works
on pulling or processing data first
before starting ML training on a
Managed Cloud Service
Raw Data Processing and
Transformation Pipeline
Cloud Training Platforms
on-prem KF
23. Team A
Team B
Team C
Team D
Google Cloud AI
AWS SageMaker
KF on prem
Azure ML
Running ML workflow across
the enterprise with multiple
teams using different Cloud
Provider technology stacks
Data
Sourcing
Pre
Processing
Feature
Engineering
Azure Storage
Google Storage
AWS S3 Storage
Raw Data Transformation Processed Data
Storage Compute
At scale, it gets complex ...
MavenCode Confidential and Proprietary
24. To simplify the Complexities can we abstract our ML Pipeline...
Data
Sourcing
Pre
Processing
Feature
Engineering
Model Training
/ Evaluation
Model Scoring
/Management
Model
Inferencing
Storage Compute
1 2
Feature Store
Kubernetes
MavenCode Confidential and Proprietary
25. To simplify the Complexities can we abstract our ML Pipeline...
Data Sourcing Pre
Processing
Feature
Engineering
Model Training /
Evaluation
ModelScoring
/Management
Model
Inferencing
Storage Compute
1 2
Feature Store
Kubeflow on Kubernetes Vertex AI
- Vertex AI Feature Store (Managed Service )
- Feast
- Databricks Feature Store
MavenCode Confidential and Proprietary
27. What’s Feature Store All About
A Feature is a measurable observable attribute that is part of the input to a Machine Learning Model.
X1
X2
X3
Xn
Model Training
[Feature Vector]
Model
MavenCode Confidential and Proprietary
28. What’s Feature Store All About
X1
X2
X3
Xn
Model Training
[Feature Vector]
Model
Features are derived from
● Raw Datastore
● Streaming Datasource
● Aggregates of Raw Inputs
● Windows (mins, hourly, daily, weekly)
MavenCode Confidential and Proprietary
29. Features Change Over time!
X1
X2
X3
Xn
Model Training
X1
X2
X3
Xn
X1
X2
X3
Xn
Time
MavenCode Confidential and Proprietary
30. Feature Stores In MLOps
● Makes it easy to operationalize our ML workload, most importantly Data Management and Storage for
Model training
● Features can be shared easily among teams running different Model training pipelines
● We can get to version of datasets and track changes easily
● Consistency in Feature input attributes between Model Training and Serving
MavenCode Confidential and Proprietary
31. Getting Data into a Feature Store
import kfp
from kfp import components
KafkaDatastreamer_op =
kfp.components.create_component_from_func(KafkaDatastreamer,base_image="python:3.7.1”)
ValidatorOnSchema_op =
kfp.components.create_component_from_func(ValidatorOnSchema,base_image="python:3.7.1")
PreProcessor_op =
kfp.components.create_component_from_func(PreProcessor,base_image="python:3.7.1")
FeatureStoreWriter_op= kfp.components.create_component_from_func(FeatureStoreWriter,
base_image="mavencode.io/spark:v3.1.1")
MavenCode Confidential and Proprietary
33. MavenCode Confidential and Proprietary
Challenges In Enterprise ML
Reproducibility
● Not Easy to Reproduce ML Model Output
on each iterative runs
● Constantly Changing Training Data
● Consistent Environment Configuration
Issues
Reusability
● Training Pipelines are not
Componentized for Reusability
● No well defined way of doing Model
versioning and tagging
● Collaboration and sharing of source
code is not well defined
Manageability
● Managing model deployment and serving
between environments is difficult
● Versioning and Tracking model artifacts is
very difficult and complex
● No defined way to visually track updates
and changes
Automation
● A lot of deployment process is still
manual
● Steps needed to update model
parameters are not not automated
● Most data science teams are not
equipped with the right knowledge to
take models to production
34. Why Machine Learning with Kubeflow?
With Kubeflow out of the box on Kubernetes, we can easily have
Composability Portability
MavenCode Confidential and Proprietary
Scalability
35. What is Kubeflow
● Machine learning toolkit for Kubernetes.
● Platform to productionize ML models, making them simple, scalable and
reliable.
● Collection of Cloud native tools for all the stages of a model development
life cycle.
● Build integrated end-to-end pipelines which connect all the stages of a
model development life cycle.
MavenCode Confidential and Proprietary
36. Simply Put ...
Kubeflow Simplifies your Model Development Life Cycle (MDLC)
MavenCode Confidential and Proprietary
39. 3
1
2
Enterprise Machine Learning with Kubeflow
MLOps Training and Deployment Platform
In-Cluster Traffic Control By ISTIO -
RBAC, Access UI With SSO Identity
Compatible Proxy
Kubeflow Jupyter NoteBook Kubeflow Jupyter NoteBook Kubeflow Jupyter NoteBook Kubeflow Jupyter NoteBook
Kubeflow Managed Model
Infrastructure
Namespace - Bob Namespace - Dav Namespace - Chuck Namespace - Team
Data Scientist 1 Data Scientist 2 Data Scientist 3
Data Science Team
Authentication and
Authorization
Auto-Scalable CPU Node Pool Auto-Scalable GPU Node Pool
MavenCode Confidential and Proprietary
42. Airline Customer Prediction
● The Dataset is from Kaggle.
● The data is from an airline organization whose actual name is not given for
various reasons, therefore, the airline is given the pseudonym Invistico airlines.
● The dataset consists of (23 columns and 129880 entries) details of customers
who have already flown with them.
MavenCode Confidential and Proprietary
Data Scientists
Subject Matter
Experts
43. Problem Statement
Customer satisfaction is priority in the airline industry.
Unhappy or disengaged customers naturally mean fewer passengers and less revenue.
As satisfaction is rarely solely about the flight itself but also the experience from booking to landing, this scenario is aimed
at building a machine learning model using all salient features in the data to predict customer satisfaction.
45. Customers on business class seats were the most satisfied.
The dataset showed more satisfied customers than otherwise, with 54.7% of
the surveyed customers reporting satisfaction with their experiences
Exploratory Data Analysis
MavenCode Confidential and Proprietary
There were more female travelers than males and more females
reported satisfaction with their experiences.
Most customers travelled for business purposes and satisfaction was
higher in business travelers.
48. Feature Engineering
To make the data fit four our machine learning model, we performed the
following feature engineering steps:
1. Removing outliers
2. Dropping rows with null values
3. Dropping and combining columns with little or no correlation with our
variable
4. Converting Categorical features to numbers
MavenCode Confidential and Proprietary
Data Scientists
Data Engineers
49. Before Outlier Removal After Outlier Removal
MavenCode Confidential and Proprietary
Feature Engineering: Outlier Removal
50. Feature Engineering Data Pipeline
● Load data: reads data from source.
● Dataset Statistics: displays summary statistics of the data.
● Dataset Schema: automatically generates a schema by
inferring types, categories, and ranges from the data.
● Dataset Validation: uses the inferred schema to detect
anomalies in the data.
● Feature Engineering: performs necessary preprocessing
and feature engineering steps on the dataset.
MavenCode Confidential and Proprietary
52. ● An ML operator helps to deploy, monitor and manage the
lifecycle of a training job.
● Kubeflow Operators Include
○ Tf-operator
○ Pytorch-operator,
○ Xgboost-operator
○ MPI-operator and many more which can be found on
the official kubeflow account.
ML Operators - Overview
MavenCode Confidential and Proprietary
53. Model Training with Tensorflow Operator
● Tensorflow Operator is one of the operators offered by Kubeflow to make it easy to run and
monitor both distributed and non-distributed tensorflow jobs on Kubernetes.
● Training tensorflow models using tf-operator relies on centralized parameter servers for
coordination between workers. It supports the tensorflow framework only.
● After preprocessing our data, we built a tensorflow neural network model.
● Our tensorflow model had an accuracy of approximately 88%.
MavenCode Confidential and Proprietary
54. MavenCode Confidential and Proprietary
Hyperparameter Tuning
Model Risk
Managers/Auditors
ML Engineers
Data Scientists
55. Hyperparameters: Configuration and variable values that are external to the model, the values are always
set before model training process begin
Selecting the right Hyperparameters can significantly improve model performance in production
Hyperparameter Tuning: Is all about finding hyperparameter input values that optimizes the objective
function of the model training
What is Hyperparameter Tuning?
(a1, b1, c1,.....zN)
(a2, b2, c2,.....zN)
(a3, b3, c3,.....zN)
MavenCode Confidential and Proprietary
57. Manually tuning by Hand is very inefficient, error-prone and difficult to track
Capturing metrics across multiple jobs and comparing them is difficult!
Efficiently allocating resources and infrastructure on the Cluster to handle all the job runs is not an easy
task
As more Hyperparameters are added, the combinatorial search space of possible inputs to maximize the
training objective function grows exponentially!
Hyperparameter Tuning is Hard!
MavenCode Confidential and Proprietary
58. Hyperparameter Tuning with Katib on Kubeflow
Katib is the Hyperparameter tuning component of Kubeflow
It is Language and Framework Agnostic
- Tensorflow
- Pytorch
- MxNet
- XGBoost
Customizable Hyperparameter Search space Algorithm
- Random Search
- Grid search
- Bayesian Optimization
- Hyperband
MavenCode Confidential and Proprietary
59. 1. Experiment: An experiment is a single tuning run, also called an optimization run. You specify configuration
settings to define the experiment. The following are the main configurations:
● Objective: What you intend to optimize. This is the objective metric, also called the target variable.
● Search Space: The set of all possible hyperparameter values that the hyperparameter tuning job
should consider for optimization, and the constraints for each hyperparameter.
● Search Algorithm: The algorithm to use when searching for the optimal hyperparameter values.
Katib Concepts
MavenCode Confidential and Proprietary
60. Hyperparameter Tuning with Katib
Katib automates the Hyperparameter Tuning
process by running a pre-configured number of
training jobs (known as trials) in parallel.
MavenCode Confidential and Proprietary
61. Result of Katib Experiment
With katib hyperparameter tuning, accuracy increased from 88% to 92.1%
MavenCode Confidential and Proprietary
62. Model Serving with KFServing
● KFServing is Kubeflow’s model deployment
and serving toolkit
● To efficiently serve our model using
KfServing, we built a Kubeflow pipeline to
load data, preprocess, train the model, make
predictions, export and serve the model.
MavenCode Confidential and Proprietary
68. Model Development Life Cycle (Data Scientist View)
Data Information Knowledge Insight
Data Scientist workflow essentially follows this path ...
MavenCode Confidential and Proprietary
69. Machine Learning Development Life Cycle (Production Deployment)
Model Training
T
r
a
i
n
i
n
g
D
a
t
a
E
T
L
Tuning
Inferencing
S
e
r
v
i
n
g
M
o
n
i
t
o
r
i
n
g
Update
MavenCode Confidential and Proprietary