Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)

•

0 likes•193 views

The rise of DevOps and Agile over the past few years has led to a shift in responsibilities. Gone are the days when troubleshooting Kubernetes issues rested solely in the hands of DevOps teams. Today, developers are expected to own their code and will find themselves on-call to troubleshoot their apps if an issue arises. This poses a question, however: Are developers equipped with the right tools to effectively manage the troubleshooting process? After all, many of the tools they rely on to figure out root causes were initially built with DevOps teams in mind. Join Baruch Sadogursky, head of DevOps advocacy at JFrog and Itiel Shwartz, CTO and co-founder at Komodor as they discuss: The gap between DevOps teams and developers when it comes to troubleshooting K8s issues Why current DevOps tools do not speak “developer's language" Best practices and recommended tools to empower dev teams to troubleshoot independently

Software

Why DevOps Tools Do Not
Speak Developer Language
(And How To Overcome This)

Cloud native | March 2021
Itiel Shwartz
CTO & Co-Founder @ Komodor
● Leading a startup which is building the
first k8s-native troubleshooting platform.
● A big believer in dev empowerment and
moving fast.
Who Are We?
Baruch Sadogursky
Head of DevOps Advocacy @ JFrog
● Helping developers being more
productive with DevOps
● Co-author of #LiquidSoftware an
DevOps Tools for Java Developers
books

Cloud native | March 2021
https://www.cncf.io/wp-content/uploads/2020/11/CNCF_Survey_Report_2020.pdf
Today’s Reality:
Code Is Being
Released Faster
Than Ever

“Kubernetes has become the de
facto container orchestrator, with
over 88% having adopted K8s
already.”
Source: https://www.redhat.com/en/resources/kubernetes-adoption-security-market-trends-2021-overview
To Keep Up With The Pace, More & More
Organizations Are Turning to Kubernetes

…But It Comes
With It’s Own Set
of Challenges,
Especially Given
Its Complexity
https://dzone.com/articles/the-2018-state-of-devops-reports

Cloud native | March 2021
Question 1:
Who Should Own Troubleshooting K8s
Issues in the Age of DevOps?
Developers
DevOps
SREs/Platform
Observability
Teams

Cloud native | March 2021
Shifting Left is Essential

Cloud native | March 2021
Lack of K8s Knowledge/Expertise
Lack of Permissions/Access
Lack of Cultural Mindset
Lack of Developer-Friendly Tools
Question 2:
What is Blocking
Organizations
From Shifting Left?

Cloud native | March 2021
Question 3:
Why Is There a Gap in Cultural Mindset
When it Comes to Incident Response?

Cloud native | March 2021
Question 4:
Why Do Current DevOps
Tools Not Speak
“Developer Language”?

So How Can We Bridge
the Gap Between Dev &
Ops Teams?

Cloud native | March 2021
Best Practices to Empower
Dev Teams to Troubleshoot
Independently

Cloud native | March 2021
Recommended Tools to Empower Dev
Teams to Troubleshoot Independently

What's hot

Building Cloud Native Applications Using Azure Kubernetes Service

Dennis Moon

Is your kubernetes negative or positive

LibbySchulze

Autoscaling event driven apps

LibbySchulze

Sergio Seabra - Red Hat - OSL19

marketingsyone

Presented by: Veer Muchandi Presented at the All Things Open 2021 Raleigh, NC, USA Raleigh Convention Center Abstract: With infrastructure being within an enterprise datacenter and on cloud, with more and more applications getting containerized, with kubernetes being the primary orchestrator for workloads, considering security-first, configuration management and GitOps, infrastructure as code, visualization and problem solving from a single enterprise viewpoint, offering Application Developers and DevOps teams a consistent lifecycle, ability to deploy what enterprises want to run where they want, an Application Platform's needs are way more advanced than ever. In this quick session we will look at building blocks in the architecture of such a Modern Application Platform for enterprises.

What Makes up a Modern Application Platform?

All Things Open

Google Cloud Run’s use of Knative introduced a portable Serverless solution built on top of Kubernetes. In this talk, we’ll recap the basic guidelines, use cases, and benefits of a Serverless architecture. Getting up and started, you will learn to take advantage of containers and the Ambassador API Gateway to serve event-driven application workloads and save costs using your existing Kubernetes resources. https://confoo.ca/en/yul2020/session/build-your-own-serverless-with-knative

[Confoo Montreal 2020] Build Your Own Serverless with Knative - Alex Gervais

Ambassador Labs

Join Datadog for a webinar on monitoring Kubernetes with a focus on Amazon EKS. You'll learn how to get the most out of Datadog's intuitive platform and EKS's unique capabilities, including: How to monitor metrics, logs and traces from your EKS environment How to test the usability of your environment with features such as adaptive Browser Tests and globally available Real User Monitoring How to find and fix user-facing issues with synthetic monitoring features like adaptive Browser Tests and globally available Real User Monitoring

Monitoring Your AWS EKS Environment with Datadog

DevOps.com

Communication Ops (Co-Ops) is a collaboration model that helps connect people, machines, software and automation to a get-things-done pipeline. Encouraging transparency, continual improvement and speed, Co-Ops helps everyone work together. In this session, you'll learn how to build your own lightweight communication infrastructure based on Docker components, and what integrations matter most in getting your team humming along productively. We'll also use the latest in bot technology to augment reporting, support, and execution of a software delivery lifecycle.

Communication Operations: Lift off with Docker! - Jesse White

Docker, Inc.

Tectonic Summit 2016: Multitenant Data Architectures with Kubernetes

CoreOS

The Developer is the New CIO: How Vendors Adapt to the Changing Landscape

Lauren Cooney

Secure your container within 30 minutes

DevOps Indonesia

In this talk we are going to explore some patterns of evolution of legacy systems towards microservices, and how this transition can be oriented towards event-oriented architectures. We would like to talk about the trade-off between having a centralized database versus multiple microservices with their own database. In this scenario, how can we guarantee transaction consistency and when to use choreography versus orchestration. In addition, we are going to bring practical examples of when to use synchronous or asynchronous communication and when it makes sense to use something like Service Mesh or Event Sourcing.

Patterns of evolution from monolith to microservices

Karina Mora

Google deployment manager

Luillyfe Blanco

Cloud Native & Docker

Brian Christner

Automation can only be as good as the people who use it. High trust cultures which empower, minimize blame and connect people with intrinsic motivations will dominate the cloud native landscape. The cloud was not built with ITIL. Heavy change control processes attempted to minimize incidents but typically drain the enthusiasm of the people while resulting in brittle systems which fail catastrophically. Self service policy enforcing automation makes doing the right thing easy and expedient allowing cloud natives to focus on delivering for the business instead of managing politics and signatures.

Cloud Native Empowered Culture

VMware Tanzu

Introduction To Flink

Knoldus Inc.

Pivotal Cloud Foundry et Microsoft: Pourquoi? ... Et pourquoi pas?

VMware Tanzu

WSO2Con USA 2015: Planning Your Cloud Strategy

WSO2

Enterprise level cloud CI

SQUADEX

Cloud native

Ken Owens

What's hot (20)

Building Cloud Native Applications Using Azure Kubernetes Service

Is your kubernetes negative or positive

Autoscaling event driven apps

Sergio Seabra - Red Hat - OSL19

What Makes up a Modern Application Platform?

[Confoo Montreal 2020] Build Your Own Serverless with Knative - Alex Gervais

Monitoring Your AWS EKS Environment with Datadog

Communication Operations: Lift off with Docker! - Jesse White

Tectonic Summit 2016: Multitenant Data Architectures with Kubernetes

The Developer is the New CIO: How Vendors Adapt to the Changing Landscape

Secure your container within 30 minutes

Patterns of evolution from monolith to microservices

Google deployment manager

Cloud Native & Docker

Cloud Native Empowered Culture

Introduction To Flink

Pivotal Cloud Foundry et Microsoft: Pourquoi? ... Et pourquoi pas?

WSO2Con USA 2015: Planning Your Cloud Strategy

Enterprise level cloud CI

Cloud native

Similar to Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)

DevOps has evolved over these years and has presented us with a new set of challenges. We have more tools and complexity. This presentation will walk you thought - * How we tackle the explosion of tools and complexity and add more value to business * How platforms can help * What challenge we face when we build a platform from scratch * How a joint solution from Canonical MicroK8s and gopaddle can help.

Efficient platform engineering with Microk8s & gopaddle.pdf

Vinothini Raju

A hands-on workshop will go over the foundations of the containers platform, including an overview of the platform system components: images, containers, repositories, clustering, and orchestration. The strategy is to demonstrate through "live demo, and hands-on exercises." The reuse case of containers in building a portable distributed application cluster running a variety of workloads including HPC workload.

Bahrain ch9 introduction to docker 5th birthday

Walid Shaari

Find out what's going on in the world of #containers in 2018. Read #predictions from 16 of the industry's leading experts to learn more about Docker, Kubernetes, Microservices and more! Hear from industry thought leaders from companies like Datadog, Hedvig, ManageEngine, Mirantis, Red Hat, SUSE and more. Make sure to also read the more than 280+ other expert predictions from technologies across virtualization, cloud computing, hyperconverged, IoT, security, etc. here: http://bit.ly/2DQi2OT

VMblog - 2018 Containers Predictions from 16 Industry Experts

vmblog

KubeCon CloudNativeCon 2016 Seattle - a report

Krishna-Kumar

How to Contribute to Cloud Native Computing Foundation

CodeOps Technologies LLP

How to contribute to cloud native computing foundation (CNCF)

Krishna-Kumar

Epics is a decentralized crowdsourcing platform for incentivizing open source software development. Clients can set up a GitHub issue as a Quest (Smart Contract). Developers can receive token prizes by solving GitHub issues (Quests). This ecosystem makes developers focus on problems in their expertise. Also, clients will be able to solve more advanced problems. The more token prizes, the more attention to be solved issues. As a result, open source software grows faster in quality. Epics solves resource imbalances in software development.

Epics - Incentive Revolution for Open Source Development

Epics DAO

Cloud Foundry 2015 LInuxCon Keynote

Cloud Foundry Foundation

Imagine onboarding a next-generation developer with no mainframe experience who successfully debugs COBOL code on their first day. By equipping them with mainframe-specific extensions to common tools like Visual Studio Code combined with the Zowe framework, new talent can be productive immediately - all without disrupting colleagues using traditional tools. Join this session to learn how mainframe application development is merging with enterprise IT toolchains and processes, including CI/CD pipelines. The presentation will include a demonstration of a mainframe developer cockpit designed for productivity and ready for shift-left automation. Make “Day 1 Debug” a reality.

Enterprise DevOps Series: Using VS Code & Zowe

DevOps.com

Devops predictions for 2022

Serena Gray

Cloud to Edge

Wesley Reisz

Docker - A high level introduction to dockers and containers

Dr Ganesh Iyer

This deck provides an overview of Oracle's Cloud Native Application Development offerings. It covers developing and deploying cloud native applications like Microservices and Serverless functions using Continuous Integration and Delivery Pipelines. This will be followed by a workshop where you will get a hands-on experience of how to build and deploy simple Java and Node.js microservices using a CI/CD Pipelines and Kubernetes in Oracle Cloud.

Building and Deploying Cloud Native Applications

Manish Kapur

Cloud Native Meetup Santa Clara 07-11-2019 by Manish Kapur

Oracle Developers

Secrets of Successful Cloud Foundry Adopters

VMware Tanzu

StackEngine Problem Space Demo

Boyd Hemphill

Introducing .Net Core - What to do, When to do it, and How to do it.pptx

Onviqa Pvt. Ltd.

In the ever-evolving landscape of software development, staying ahead of the curve requires leveraging the latest tools and technologies. As we enter 2024, developers are faced with a plethora of options for streamlining their workflows, enhancing collaboration, and delivering high-quality software solutions. Integrated Development Environments (IDEs): IDEs are the cornerstone of modern software development, providing developers with a unified platform for writing, editing, and debugging code. With features like syntax highlighting, code completion, and version control integration, IDEs such as Visual Studio Code, IntelliJ IDEA, and Eclipse streamline the development process and improve productivity. Collaboration Tools: Collaboration tools play a crucial role in fostering communication and teamwork among developers. Platforms like Slack, Microsoft Teams, and Zoom provide instant messaging, video conferencing, and file sharing capabilities, enabling developers to collaborate effectively, regardless of their location or time zone. Code Editors: Code editors are lightweight yet powerful tools for writing and editing code. With features like customizable themes, plug-in support, and multi-language syntax highlighting, code editors like Sublime Text, Atom, and Vim enhance coding efficiency and readability, enabling developers to write clean and maintainable code. Testing Frameworks: Testing frameworks such as Selenium, JUnit, and PyTest enable developers to automate the testing process and ensure the quality of their code. By writing and running test cases, testing frameworks help identify and fix bugs early in the development cycle, reducing the risk of defects in production. Continuous Integration/Continuous Deployment (CI/CD) Tools: CI/CD tools like Jenkins, Travis CI, and CircleCI automate the process of building, testing, and deploying software updates. By integrating with version control systems and automating repetitive tasks, CI/CD tools enable developers to deliver software faster and more reliably, accelerating time-to-market and enhancing agility. Code Quality Analysis Tools: Code quality analysis tools like SonarQube, ESLint, and Checkstyle help developers identify and fix code smells and vulnerabilities. By analyzing code against coding standards and best practices, code quality analysis tools ensure the maintainability and security of software projects, enabling developers to write clean and robust code. Performance Monitoring Tools: Performance monitoring tools such as New Relic, Datadog, and Prometheus enable developers to monitor and optimize the performance of their applications in real-time. By collecting and analyzing metrics like response time, throughput, and error rate, performance monitoring tools help developers identify and address performance bottlenecks, ensuring optimal application performance and user experience.

Top 10 Must-Have Software Development Tools for 2024.pdf

Integrated IT Solutions

Watch this talk on YouTube: https://youtu.be/-3K74I7t7CQ Securing the Software Supply Chain has become a focus of cybersecurity efforts the world over. One aspect of this is with the generation and verification of a Software Bill of Materials (SBOM). But what is an SBOM and how would you go about setting this up for your cloud native container/applications/pipeline? The Flux team recently published a blog on this very topic and how they’ve gone about implementing these measures. During this session, Dan Luhring, OSS Engineering Manager at Anchore, will dive into SBOMs - what they are, why you need them, some common use cases and how to get your pipeline ready for SBOM generation and verification using the Flux SBOM as an example. Resources Anchore: A comprehensive, continuous security and compliance platform to protect your cloud-native applications. Anchore’s OSS tools featured during this session: - Syft: A CLI tool for generating a Software Bill of Materials (SBOM) from container images and file systems - Grype: An easy-to-integrate open source vulnerability scanning tool for container images and file systems. Speaker Bios: Dan Luhring heads up OSS at Anchore, where he leads the software engineering team that develops Syft and Grype. Dan is drawn deeply into the cloud native security space, where he focuses on container workflows and developer experience. Dan believes in making software more secure by making life better for software engineers and security practitioners. Dan is a maintainer of Sigstore’s Cosign project, and he loves partnering with other people to find solutions to daunting challenges. Priyanka (aka “Pinky”) is a Developer Experience Engineer at Weaveworks. She has worked on a multitude of topics including front end development, UI automation for testing and API development. Previously she was a software developer at State Farm where she was on the delivery engineering team working on GitOps enablement. She was instrumental in the multi-tenancy migration to utilize Flux for an internal Kubernetes offering. Outside of work, Priyanka enjoys hanging out with her husband and two rescue dogs as well as traveling around the globe.

Security: The Value of SBOMs

Weaveworks

The lessons I learned is that Open source quickly becomes the natural choice wherever commoditization is happening in the software stack. Thus we expect business-to-business open source, which is already a significant trend in recent history, to become an increasingly common form of open source collaboration. Companies who understand the ground rules of business-to-business open source will be better positioned to identify and take advantage of open source opportunities in the competitive spaces that they share with other companies. So I will share why open strategy is import for the enterprise. And how to do contributions for the open source projects n today’s topic.

OaaS:Open as a Strategy

OpenCity Community

Similar to Why DevOps Tools Do Not Speak Developer Language (and how to overcome this) (20)

Efficient platform engineering with Microk8s & gopaddle.pdf

Bahrain ch9 introduction to docker 5th birthday

VMblog - 2018 Containers Predictions from 16 Industry Experts

KubeCon CloudNativeCon 2016 Seattle - a report

How to Contribute to Cloud Native Computing Foundation

How to contribute to cloud native computing foundation (CNCF)

Epics - Incentive Revolution for Open Source Development

Cloud Foundry 2015 LInuxCon Keynote

Enterprise DevOps Series: Using VS Code & Zowe

Devops predictions for 2022

Cloud to Edge

Docker - A high level introduction to dockers and containers

Building and Deploying Cloud Native Applications

Cloud Native Meetup Santa Clara 07-11-2019 by Manish Kapur

Secrets of Successful Cloud Foundry Adopters

StackEngine Problem Space Demo

Introducing .Net Core - What to do, When to do it, and How to do it.pptx

Top 10 Must-Have Software Development Tools for 2024.pdf

Security: The Value of SBOMs

OaaS:Open as a Strategy

More from Komodor

CNCF JLM Meetup - Making Peace With the Grim Reaper

Komodor

Using Kubernetes for orchestration? Great—we hope things are running smoothly. The thing about Kubernetes, though, is that it tends to surprise you—throwing curveballs just when you think you've finally mastered the art of container management. And those curveballs usually come at you when you try to scale up. So, how can you scale K8s without striking out due to speed and reliability (not to mention sanity) issues? Join Guy Menahem, solution architect at Komodor, as he shares some of the hard lessons he learned from his own experiences with Kubernetes, including: - The “phantom” challenges of working with K8s - The five key factors you can’t ignore when scaling K8s - Best practices and tools for a smooth-running K8s system

5 things we learned not to ignore while scaling kubernetes webinar dev ops.co...

Komodor

You can’t build a CI/CD pipeline and support fast-paced development cycles without considering continuous reliability. On the one hand, this means being rehearsed and prepared for every scenario. On the other, this calls for a contingency plan for when (inevitably) something will go wrong. Join this live event and see how DevOps tools can help you plan for the best and prepare for the worst, as Julie from Gremlin injects chaos into the Bank of Anthos’ system and Rona from Komodor troubleshoots things back into order.

Chaos & Order: Breaking and Fixing Things in K8s Environments

Komodor

Preventing Kubernetes Misconfigurations

Komodor

Accelerated Troubleshooting with Komodor and Coralogix

Komodor

Troubleshooting in a distributed systems

Komodor

Building a Cloud-Native Company in 2020 [DevOps Days Buffalo]

Komodor

More from Komodor (7)

CNCF JLM Meetup - Making Peace With the Grim Reaper

5 things we learned not to ignore while scaling kubernetes webinar dev ops.co...

Chaos & Order: Breaking and Fixing Things in K8s Environments

Preventing Kubernetes Misconfigurations

Accelerated Troubleshooting with Komodor and Coralogix

Troubleshooting in a distributed systems

Building a Cloud-Native Company in 2020 [DevOps Days Buffalo]

Recently uploaded

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Eric Wang (Software Engineer, @Uber) Uber has numerous deep learning models, most of which are highly complex with many layers and a vast number of features. Understanding how these models work is challenging and demands significant resources to experiment with various training algorithms and feature sets. With ML explainability, the ML team aims to bring transparency to these models, helping to clarify their predictions and behavior. This transparency also assists the operations and legal teams in explaining the reasons behind specific prediction outcomes. In this talk, Eric Wang will discuss the methods Uber used for explaining deep learning models and how we integrated these methods into the Uber AI Michelangelo ecosystem to support offline explaining.

AI/ML Infra Meetup | ML explainability in Michelangelo

Alluxio, Inc.

Agnieszka Andrzejewska - BIM School Course in Kraków

bim.edu.pl

COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Globus

NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?

Understanding Globus Data Transfers with NetSage

Globus

Petr Matuska, Sales & Sales Engineering Lead, GraphAware Western Australia Police Force’s adoption of Neo4j and the GraphAware Hume graph analytics platform marks a significant advancement in data-driven policing. Facing the challenges of growing volumes of valuable data scattered in disconnected silos, the organisation successfully implemented Neo4j database and Hume, consolidating data from various sources into a dynamic knowledge graph. The result was a connected view of intelligence, making it easier for analysts to solve crime faster. The partnership between Neo4j and GraphAware in this project demonstrates the transformative impact of graph technology on law enforcement’s ability to leverage growing volumes of valuable data to prevent crime and protect communities.

GraphAware - Transforming policing with graph-based intelligence analysis

Neo4j

Into the Box 2024 - Keynote Day 2 Slides.pdf

Ortus Solutions, Corp

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Juraj Vysvader

As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.

Globus Compute wth IRI Workflows - GlobusWorld 2024

Globus

Breaking the Code : A Guide to WhatsApp Business API.pdf

Meon Technology

Unlocking Business Potential: Tailored Technology Solutions by Prosigns Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support. Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth. Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices. AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making. Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency. DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration. Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly. Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business. Join us on a journey of innovation and growth. Let's partner for success with Prosigns.

Prosigns: Transforming Business with Tailored Technology Solutions

Prosigns

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Lu Qiu (Data & AI Platform Tech Lead, @Alluxio) - Siyuan Sheng (Senior Software Engineer, @Alluxio) Speed and efficiency are two requirements for the underlying infrastructure for machine learning model development. Data access can bottleneck end-to-end machine learning pipelines as training data volume grows and when large model files are more commonly used for serving. For instance, data loading can constitute nearly 80% of the total model training time, resulting in less than 30% GPU utilization. Also, loading large model files for deployment to production can be slow because of slow network or storage read operations. These challenges are prevalent when using popular frameworks like PyTorch, Ray, or HuggingFace, paired with cloud object storage solutions like S3 or GCS, or downloading models from the HuggingFace model hub. In this presentation, Lu and Siyuan will offer comprehensive insights into improving speed and GPU utilization for model training and serving. You will learn: - The data loading challenges hindering GPU utilization - The reference architecture for running PyTorch and Ray jobs while reading data from S3, with benchmark results of training ResNet50 and BERT - Real-world examples of boosting model performance and GPU utilization through optimized data access

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...

Alluxio, Inc.

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

Large Language Models and the End of Programming

Matt Welsh

Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload. Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.

Designing for Privacy in Amazon Web Services

KrzysztofKkol1

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.

TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR

Tier1 app

Vitthal Shirke Microservices Resume Montevideo

Vitthal Shirke

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

Abortion Clinic

Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.

How Recreation Management Software Can Streamline Your Operations.pptx

wottaspaceseo

Key takeaways: Challenges of building platforms and the benefits of platformless. Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience. How Choreo enables the platformless experience. How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo. Demo of an end-to-end app built and deployed on Choreo.

Accelerate Enterprise Software Engineering with Platformless

WSO2

Recently uploaded (20)

AI/ML Infra Meetup | ML explainability in Michelangelo

Agnieszka Andrzejewska - BIM School Course in Kraków

Developing Distributed High-performance Computing Capabilities of an Open Sci...

Understanding Globus Data Transfers with NetSage

GraphAware - Transforming policing with graph-based intelligence analysis

Into the Box 2024 - Keynote Day 2 Slides.pdf

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Globus Compute wth IRI Workflows - GlobusWorld 2024

Breaking the Code : A Guide to WhatsApp Business API.pdf

Prosigns: Transforming Business with Tailored Technology Solutions

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...

Cyaniclab : Software Development Agency Portfolio.pdf

Large Language Models and the End of Programming

Designing for Privacy in Amazon Web Services

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR

Vitthal Shirke Microservices Resume Montevideo

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

How Recreation Management Software Can Streamline Your Operations.pptx

Accelerate Enterprise Software Engineering with Platformless

Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)

1. Why DevOps Tools Do Not Speak Developer Language (And How To Overcome This)

2. Cloud native | March 2021 Itiel Shwartz CTO & Co-Founder @ Komodor ● Leading a startup which is building the first k8s-native troubleshooting platform. ● A big believer in dev empowerment and moving fast. Who Are We? Baruch Sadogursky Head of DevOps Advocacy @ JFrog ● Helping developers being more productive with DevOps ● Co-author of #LiquidSoftware an DevOps Tools for Java Developers books

3. Cloud native | March 2021 https://www.cncf.io/wp-content/uploads/2020/11/CNCF_Survey_Report_2020.pdf Today’s Reality: Code Is Being Released Faster Than Ever

4. “Kubernetes has become the de facto container orchestrator, with over 88% having adopted K8s already.” Source: https://www.redhat.com/en/resources/kubernetes-adoption-security-market-trends-2021-overview To Keep Up With The Pace, More & More Organizations Are Turning to Kubernetes

5. …But It Comes With It’s Own Set of Challenges, Especially Given Its Complexity https://dzone.com/articles/the-2018-state-of-devops-reports

6. Cloud native | March 2021 Question 1: Who Should Own Troubleshooting K8s Issues in the Age of DevOps? Developers DevOps SREs/Platform Observability Teams

7. Cloud native | March 2021 Shifting Left is Essential

8. Cloud native | March 2021 Lack of K8s Knowledge/Expertise Lack of Permissions/Access Lack of Cultural Mindset Lack of Developer-Friendly Tools Question 2: What is Blocking Organizations From Shifting Left?

9. Cloud native | March 2021 Question 3: Why Is There a Gap in Cultural Mindset When it Comes to Incident Response?

10. Cloud native | March 2021 Question 4: Why Do Current DevOps Tools Not Speak “Developer Language”?

11. So How Can We Bridge the Gap Between Dev & Ops Teams?

12. Cloud native | March 2021 Best Practices to Empower Dev Teams to Troubleshoot Independently

13. Cloud native | March 2021 Recommended Tools to Empower Dev Teams to Troubleshoot Independently

14. Q&A

Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)

Similar to Why DevOps Tools Do Not Speak Developer Language (and how to overcome this) (20)

More from Komodor

More from Komodor (7)

Recently uploaded

Recently uploaded (20)

Why DevOps Tools Do Not Speak Developer Language (and how to overcome this)