NVIDIA DEEP LEARNING INFERENCE PLATFORM PERFORMANCE STUDY
| TECHNICAL OVERVIEW
| 1
Introduction
Artificial intelligence (AI), the dream of computer scientists for over half
a century, is no longer science fiction—it is already transforming every
industry. AI is the use of computers to simulate human intelligence. AI
amplifies our cognitive abilities—letting us solve problems where the
complexity is too great, the information is incomplete, or the details are
too subtle and require expert training.
While the machine learning field has been active for decades, deep
learning (DL) has boomed over the last five years. In 2012, Alex
Krizhevsky of the University of Toronto won the ImageNet image
recognition competition using a deep neural network trained on NVIDIA
GPUs—beating all the human expert algorithms that had been honed
for decades. That same year, recognizing that larger networks can learn
more, Stanford’s Andrew Ng and NVIDIA Research teamed up to develop
a method for training networks using large-scale GPU computing
systems. These seminal papers sparked the “big bang” of modern AI,
setting off a string of “superhuman” achievements. In 2015, Google and
Microsoft both beat the best human score in the ImageNet challenge. In
2016, DeepMind’s AlphaGo recorded its historic win over Go champion
Lee Sedol and Microsoft achieved human parity in speech recognition.
GPUs have proven to be incredibly effective at solving some of the most
complex problems in deep learning, and while the NVIDIA deep learning
platform is the standard industry solution for training, its inferencing
capability is not as widely understood. Some of the world’s leading
enterprises from the data center to the edge have built their inferencing
solution on NVIDIA GPUs. Some examples include:
NVIDIA DEEP LEARNING INFERENCE PLATFORM PERFORMANCE STUDY
| TECHNICAL OVERVIEW
| 1
Introduction
Artificial intelligence (AI), the dream of computer scientists for over half
a century, is no longer science fiction—it is already transforming every
industry. AI is the use of computers to simulate human intelligence. AI
amplifies our cognitive abilities—letting us solve problems where the
complexity is too great, the information is incomplete, or the details are
too subtle and require expert training.
While the machine learning field has been active for decades, deep
learning (DL) has boomed over the last five years. In 2012, Alex
Krizhevsky of the University of Toronto won the ImageNet image
recognition competition using a deep neural network trained on NVIDIA
GPUs—beating all the human expert algorithms that had been honed
for decades. That same year, recognizing that larger networks can learn
more, Stanford’s Andrew Ng and NVIDIA Research teamed up to develop
a method for training networks using large-scale GPU computing
systems. These seminal papers sparked the “big bang” of modern AI,
setting off a string of “superhuman” achievements. In 2015, Google and
Microsoft both beat the best human score in the ImageNet challenge. In
2016, DeepMind’s AlphaGo recorded its historic win over Go champion
Lee Sedol and Microsoft achieved human parity in speech recognition.
GPUs have proven to be incredibly effective at solving some of the most
complex problems in deep learning, and while the NVIDIA deep learning
platform is the standard industry solution for training, its inferencing
capability is not as widely understood. Some of the world’s leading
enterprises from the data center to the edge have built their inferencing
solution on NVIDIA GPUs. Some examples include:
Gary Paek from Intel presented this deck at the HPC User Forum in Tucson.
Learn more: https://software.intel.com/en-us/tags/18892
and
http://hpcuserforum.com
Watch the video presentation: http://wp.me/p3RLHQ-fdt
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
AWS & Intel Webinar Series - Accelerating AI ResearchIntel® Software
Scale your research workloads faster with Intel on AWS. Learn how the performance and productivity of Intel Hardware and Software help bridge the gap between ideation and results in Data Science. Get started on your AI Developer Journey @ software.intel.com/ai.
Supercomputing has swept rapidly from the far edges of science to the heart of our everyday lives. And propelling it forward – bringing it into the mobile phone already in your pocket and the car in your driveway – is GPU acceleration, NVIDIA CEO Jen-Hsun Huang told a packed house at a rollicking event kicking off this week’s SC15 annual supercomputing show in Austin. The event draws 10,000 researchers, national lab directors and others from around the world.
NVIDIA Volta Tensor Core GPU achieves new AI performance milestones in ResNet-50 for a single chip, single node, and single cloud instance. Explore the performance improvements.
NVIDIA CEO Jensen Huang Presentation at Supercomputing 2019NVIDIA
Broadening support for GPU-accelerated supercomputing to a fast-growing new platform, NVIDIA founder and CEO Jensen Huang introduced a reference design for building GPU-accelerated Arm servers, with wide industry backing.
Axel Koehler from Nvidia presented this deck at the 2016 HPC Advisory Council Switzerland Conference.
“Accelerated computing is transforming the data center that delivers unprecedented through- put, enabling new discoveries and services for end users. This talk will give an overview about the NVIDIA Tesla accelerated computing platform including the latest developments in hardware and software. In addition it will be shown how deep learning on GPUs is changing how we use computers to understand data.”
In related news, the GPU Technology Conference takes place April 4-7 in Silicon Valley.
Watch the video presentation: http://insidehpc.com/2016/03/tesla-accelerated-computing/
See more talks in the Swiss Conference Video Gallery:
http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter:
http://insidehpc.com/newsletter
Opening Keynote at GTC 2015: Leaps in Visual ComputingNVIDIA
NVIDIA CEO and co-founder Jen-Hsun Huang took the stage for the GPU Technology Conference in the San Jose Convention Center to present some major announcements on March 17, 2015. You'll find out how NVIDIA is innovating in the field of deep learning, what NVIDIA DRIVE PX can do for automakers, and where Pascal, the next-generation GPU architecture, fits in the new performance roadmap.
A Primer on FPGAs - Field Programmable Gate ArraysTaylor Riggan
A focus on the use of FPGAs by cloud service providers. Includes Microsoft Azure Catapult, Google Tensor Processors, and Amazon EC2 F1 instances. Also includes background info on how to get started with FPGAs
HPC DAY 2017 | Accelerating tomorrow's HPC and AI workflows with Intel Archit...HPC DAY
HPC DAY 2017 - http://www.hpcday.eu/
Accelerating tomorrow's HPC and AI workflows with Intel Architecture
Atanas Atanasov | HPC solution architect, EMEA region at Intel
NVIDIA CEO Jen-Hsun Huang introduces NVLink and shares a roadmap of the GPU. Primary topics also include an introduction of the GeForce GTX Titan Z, CUDA for machine learning, and Iray VCA.
Nvidia Deep Learning Solutions - Alex SabatierSri Ambati
Alex Sabatier from Nvidia talks about the future of Deep Learning from an chipmaker perspective
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Fórum E-Commerce Brasil | Tecnologias NVIDIA aplicadas ao e-commerce. Muito a...E-Commerce Brasil
Tecnologias NVIDIA aplicadas ao e-commerce. Muito além do hardware.
Jomar Silva
Gerente de relacionamento com desenvolvedores para a América Latina - NVIDIA
https://eventos.ecommercebrasil.com.br/forum/
Gary Paek from Intel presented this deck at the HPC User Forum in Tucson.
Learn more: https://software.intel.com/en-us/tags/18892
and
http://hpcuserforum.com
Watch the video presentation: http://wp.me/p3RLHQ-fdt
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
AWS & Intel Webinar Series - Accelerating AI ResearchIntel® Software
Scale your research workloads faster with Intel on AWS. Learn how the performance and productivity of Intel Hardware and Software help bridge the gap between ideation and results in Data Science. Get started on your AI Developer Journey @ software.intel.com/ai.
Supercomputing has swept rapidly from the far edges of science to the heart of our everyday lives. And propelling it forward – bringing it into the mobile phone already in your pocket and the car in your driveway – is GPU acceleration, NVIDIA CEO Jen-Hsun Huang told a packed house at a rollicking event kicking off this week’s SC15 annual supercomputing show in Austin. The event draws 10,000 researchers, national lab directors and others from around the world.
NVIDIA Volta Tensor Core GPU achieves new AI performance milestones in ResNet-50 for a single chip, single node, and single cloud instance. Explore the performance improvements.
NVIDIA CEO Jensen Huang Presentation at Supercomputing 2019NVIDIA
Broadening support for GPU-accelerated supercomputing to a fast-growing new platform, NVIDIA founder and CEO Jensen Huang introduced a reference design for building GPU-accelerated Arm servers, with wide industry backing.
Axel Koehler from Nvidia presented this deck at the 2016 HPC Advisory Council Switzerland Conference.
“Accelerated computing is transforming the data center that delivers unprecedented through- put, enabling new discoveries and services for end users. This talk will give an overview about the NVIDIA Tesla accelerated computing platform including the latest developments in hardware and software. In addition it will be shown how deep learning on GPUs is changing how we use computers to understand data.”
In related news, the GPU Technology Conference takes place April 4-7 in Silicon Valley.
Watch the video presentation: http://insidehpc.com/2016/03/tesla-accelerated-computing/
See more talks in the Swiss Conference Video Gallery:
http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter:
http://insidehpc.com/newsletter
Opening Keynote at GTC 2015: Leaps in Visual ComputingNVIDIA
NVIDIA CEO and co-founder Jen-Hsun Huang took the stage for the GPU Technology Conference in the San Jose Convention Center to present some major announcements on March 17, 2015. You'll find out how NVIDIA is innovating in the field of deep learning, what NVIDIA DRIVE PX can do for automakers, and where Pascal, the next-generation GPU architecture, fits in the new performance roadmap.
A Primer on FPGAs - Field Programmable Gate ArraysTaylor Riggan
A focus on the use of FPGAs by cloud service providers. Includes Microsoft Azure Catapult, Google Tensor Processors, and Amazon EC2 F1 instances. Also includes background info on how to get started with FPGAs
HPC DAY 2017 | Accelerating tomorrow's HPC and AI workflows with Intel Archit...HPC DAY
HPC DAY 2017 - http://www.hpcday.eu/
Accelerating tomorrow's HPC and AI workflows with Intel Architecture
Atanas Atanasov | HPC solution architect, EMEA region at Intel
NVIDIA CEO Jen-Hsun Huang introduces NVLink and shares a roadmap of the GPU. Primary topics also include an introduction of the GeForce GTX Titan Z, CUDA for machine learning, and Iray VCA.
Nvidia Deep Learning Solutions - Alex SabatierSri Ambati
Alex Sabatier from Nvidia talks about the future of Deep Learning from an chipmaker perspective
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Fórum E-Commerce Brasil | Tecnologias NVIDIA aplicadas ao e-commerce. Muito a...E-Commerce Brasil
Tecnologias NVIDIA aplicadas ao e-commerce. Muito além do hardware.
Jomar Silva
Gerente de relacionamento com desenvolvedores para a América Latina - NVIDIA
https://eventos.ecommercebrasil.com.br/forum/
Introduction to Software Defined Visualization (SDVis)Intel® Software
Software defined visualization (SDVis) is an open-source initiative from Intel and industry collaborators. Improve the visual fidelity, performance, and efficiency of prominent visualization solutions, while supporting the rapidly growing big data use on workstations through high-performance computing (HPC) on supercomputing clusters without memory limitations and cost of GPU-based solutions.
End to End Machine Learning Open Source Solution Presented in Cisco Developer...Manish Harsh
The RAPIDS suite of open source software libraries and APIs gives you the ability to execute end-to-end data science and analytics pipelines entirely on GPUs. Licensed under Apache 2.0, RAPIDS is incubated by NVIDIA® based on extensive hardware and data science science experience. RAPIDS utilizes NVIDIA CUDA® primitives for low-level compute optimization, and exposes GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.
Backend.AI Technical Introduction (19.09 / 2019 Autumn)Lablup Inc.
This slide introduces technical specs and details about Backend.AI 19.09.
* On-premise clustering / container orchestration / scaling on cloud
* Container-level fractional GPU technology to use one GPU as many GPUs on many containers at the same time.
* NVidia GPU Cloud integrations
* Enterprise features
Harnessing the virtual realm for successful real world artificial intelligenceAlison B. Lowndes
Artificial Intelligence is impacting all areas of society, from healthcare and transportation to smart cities and energy. How NVIDIA invests both in internal pure research and accelerated computation to enable its diverse customer base, across gaming & extended reality, graphics, AI, robotics, simulation, high performance scientific computing, healthcare & more. You will be introduced to the GPU computing platform & shown real world successfully deployed applications as well as a glimpse into the current state of the art across academia, enterprise and startups.
If you're like most of the world, you're on an aggressive race to implement machine learning applications and on a path to get to deep learning. If you can give better service at a lower cost, you will be the winners in 2030. But infrastructure is a key challenge to getting there. What does the technology infrastructure look like over the next decade as you move from Petabytes to Exabytes? How are you budgeting for more colossal data growth over the next decade? How do your data scientists share data today and will it scale for 5-10 years? Do you have the appropriate security, governance, back-up and archiving processes in place? This session will address these issues and discuss strategies for customers as they ramp up their AI journey with a long term view.
AI for an intelligent cloud and intelligent edge: Discover, deploy, and manag...James Serra
Discover, manage, deploy, monitor – rinse and repeat. In this session we show how Azure Machine Learning can be used to create the right AI model for your challenge and then easily customize it using your development tools while relying on Azure ML to optimize them to run in hardware accelerated environments for the cloud and the edge using FPGAs and Neural Network accelerators. We then show you how to deploy the model to highly scalable web services and nimble edge applications that Azure can manage and monitor for you. Finally, we illustrate how you can leverage the model telemetry to retrain and improve your content.
infoShare AI Roadshow 2018 - Tomasz Kopacz (Microsoft) - jakie możliwości daj...Infoshare
Podczas tej sesji przyjrzymy się, w jaki sposób można skorzystać z platformy Microsoft do budowy tzw. „inteligentnych” rozwiązań. W przykładach zobaczymy zarówno Cognitive Services, jak i wykorzystaniu GPU (a dokładniej – Batch AI) do uczenia sieci neuronowych. Zajmiemy się także skomplikowanym zagadnieniami związanymi z projektowaniem – tak by algorytmy rozszerzały ludzkie możliwości (a nie nas zastępowały). Sesja zakłada że słuchacze umieją programować.
NVidia and Kinetica presented together about the trends in GPU use cases across industries. The basics and GPU architecture was discussed and how it compares with ASIC and FPGA.
Kinetica presented their In-Memory Database Platform powered by GPU which provides capabilities for fast analytics, geospatial analytics and realtime ML/Deep Learning execution engine.
RAPIDS – Open GPU-accelerated Data ScienceData Works MD
RAPIDS – Open GPU-accelerated Data Science
RAPIDS is an initiative driven by NVIDIA to accelerate the complete end-to-end data science ecosystem with GPUs. It consists of several open source projects that expose familiar interfaces making it easy to accelerate the entire data science pipeline- from the ETL and data wrangling to feature engineering, statistical modeling, machine learning, and graph analysis.
Corey J. Nolet
Corey has a passion for understanding the world through the analysis of data. He is a developer on the RAPIDS open source project focused on accelerating machine learning algorithms with GPUs.
Adam Thompson
Adam Thompson is a Senior Solutions Architect at NVIDIA. With a background in signal processing, he has spent his career participating in and leading programs focused on deep learning for RF classification, data compression, high-performance computing, and managing and designing applications targeting large collection frameworks. His research interests include deep learning, high-performance computing, systems engineering, cloud architecture/integration, and statistical signal processing. He holds a Masters degree in Electrical & Computer Engineering from Georgia Tech and a Bachelors from Clemson University.
In this talk, Tong will start with the current landscape and typical use cases of Artificial Intelligence applications in the Telco domain. Then, she will introduce Intel’s strategy and products for Network AI, including our focus areas, our hardware portfolio, software stacks, roadmaps and some case studies.
Speaker: Tong Zhang, Principal Engineer and Chief Architect for AI and Analytics of the Network Platforms Group, Intel
Semiconductors are the driving force behind the AI evolution and enable its adoption across various application areas ranging from connected and automated driving to smart healthcare and wearables. Given that, electronics research, design and manufacturing communities around the world are increasingly investing in specialized AI chips providing less latency, greater processing power, higher bandwidth and faster performance. AI also attracts new technology players to invest in making their own specialized AI chips, changing the electronics manufacturing landscape and moving the AI technology towards machine learning, deep learning and neural networks.
Accelerate Machine Learning Software on Intel Architecture Intel® Software
This session presents performance data for deep learning training for image recognition that achieves greater than 24 times speedup performance with a single Intel® Xeon Phi™ processor 7250 when compared to Caffe*. In addition, we present performance data that shows training time is further reduced by 40 times the speedup with a 128-node Intel® Xeon Phi™ processor cluster over Intel® Omni-Path Architecture (Intel® OPA).
Similar to Introduction to PowerAI - The Enterprise AI Platform (20)
Build FAST Deep Learning Apps with Docker on OpenPOWER and GPUs Indrajit Poddar
GPU and NVLink accelerated training and inference with tensorflow and caffe on OpenPOWER systems. Presented at a meetup prior to DataWorks Summit Munich 2017.
Fast Scalable Easy Machine Learning with OpenPOWER, GPUs and DockerIndrajit Poddar
Transparently accelerated Deep Learning workloads on OpenPOWER systems and GPUs using easy to use open source frameworks such as Caffe, Torch, Tensorflow, Theano.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
8. Core concepts in Machine Learning:
Training Inference
Training
• Data intensive:
historical data sets
• Compute intensive:
100% accelerated
• Develop a model for
use on the edge as
inference
Inference
• Enables the computer
to act in real time
• Low Power
• Out at the edge
15. 15
To build a team with deep learning
expertise : 2 months ~ 1 year
To prepare massive training
data : ~ 10 man month(s)
To train a new
model : 1 hour ~
week
To give an AI
inference result :
< 1s
Challenges in creating an AI infrastructure
Time needed to:
• Find skills
• Handle large data-sets
• Hi-res images, video feed..
• Continuously train models
• Run inferencing at scale
• Handle rapidly evolving open
source components
CPUs are not getting faster as rapidly as before
- Moore’s law is dying
Resulting in unprecedented demand for :
• Offloaded computation, accelerators, and higher
memory bandwidth systems
• Easy to use software that works with open source and
scales
16. PowerAI: Enterprise Class, Ease of Use, Faster Training
Enterprise Software
Distribution
Binary Package of Major
Deep Learning Frameworks
with Enterprise Support
Tools for Ease of
Development
Graphical tools to Enhance
Data Scientist Developer
Experience
Faster Training Times
for Data Scientists
Performance Optimized for
Single Node & Distributed
Computing Scaling
17. 17
Data Lake
Transform & Prep
Data (ETL)
Trained Model
Images of
Damaged
Components
ModelTraining
Transform & Prep
Data (ETL)
Off-Line
Training
Production
LiveVideo
23. PowerAI
DL Frameworks + Libraries
(TensorFlow, Caffe, ..)
IBM Data Science
Experience (DSX)
Distributed Computing
with Spark & MPI
DL Developer Tools
Spectrum Scale High-Speed
File System via HDFS APIs
Cluster of NVLink Servers
PowerAI Enterprise (Coming soon)
IBM Enterprise
Support
Application Dev
Services
Enterprise Support & Services
to Augment Enterprise
Expertise
Packaged, Pre-Compiled Deep
Learning Frameworks
(TensorFlow, Caffe, Torch, ..)
Optimized for Scaling &
Fast Training Time
Data Scientists Productivity
Tools Targeted to DL
Developers
IBM Confidential
24. PowerAI: Making AI More Accessible to Developers
• AI Vision: Targeted at Application Developers
• Data Extraction, Transformation and Preparation tool
• DL Insight
• Distributed Deep Learning
Multi-tenant, Enterprise-ready Deep Learning Platform for Data Scientists
24
25. caffe-bvlc: install cuda,cuDNN, install openblas, install protobuf, clone, build and install opencv, install python, install python-dev, install
libgflags, install libgoogle-glog-dev, install liblmdb-dev, edit make file to enable CuDNN, make all, make distribution
Torch: complicated on Power as luaJIT has mixed support for OpenPOWER. We use a luaJIT fork to build.
caffe-nv: same dependencies as caffe-bvlc; separate upstream repo for caffe-nv, specific versions are needed for newer versions of Nvidia’s
DIGITS tool.
caffe-ibm: same dependencies as caffe-bvlc, separate build stream; versions; updates
Tensorflow: in PIP for x86, but it is often recommended to build from source: upgrade pip, install Bazel, install many dependencies including
java, configure the build, compile, pip install whl, upgrade protobuf
Theano: install python, numpy, scipy, openBLAS, python-dev, nose, Sphinx, cuda, pycuda, clone, build and install libgpuarray
DIGITS: clone digits from repo, install dependencies (PIP)
Life without PowerAI:
With PowerAI:
PowerAI: install cuda, cuDNN; sudo apt-get install power-mldl
26. DL Frameworks
(TF, Caffe, etc)
Data Prep & ETL via
Spectrum Conductor
with Spark
Input
Data
Deep Learning GUI
Data & Model
Management, ETLTools,
Monitor,Visualize,
Advise
DL Insight
Tuning Engine
AIVision
ComputerVisionApp
DevelopmentToolkit
IBM Spectrum Conductor with Spark
System mgmt, Distributed ETL, DistributedTraining, Hyper-Parameter Optimization
DistributedTraining
27. 27
Data Lake & Data Stores
Distributed Computing
Machine & Deep Learning
Libraries & Frameworks
CognitiveAPIs
(Eg:Watson)
In-House
CognitiveAPIs
Applications
Hadoop HDFS,
NoSQL DBs
Spark, MPI
TensorFlow, Caffe,
SparkML
Speech,Vision,
NLP, Sentiment
Segment Specific:
Finance, Retail,
Healthcare, etc.
Accelerated Servers Storage
Accelerated
Infrastructure
Transform & Prep
Data (ETL)
29. 29
Deep Learning Training + Inference
Accelerators
Clustering frameworks
Workload
Aware
Scheduling
Shared
Resource
Management
Emerging
Workloads
Dev Ops & Micro Services
High Performance
Computing
Design / Simulation / Modeling
‘New-gen
Workloads’
Hadoop, Spark, Containers
with Spark
IBM
Cloud
private
Ne
w
High Performance
Analytics
Trade / Risk Analytics
Containers and images
IBM Data
Science
Experience
31. 31
IBM OpenPOWER Moves on Deep Learning with a Vengeance
“In short, IBM kicked some butt today”
Rob Enderle
Industry Analyst
IBM brings Google's AI tools to its powerful computers
Google has cool technology to recognize images and speech, and IBM's
hardware can diagnose diseases and beat humans in Jeopardy.
Combine the two, and you get a powerful computer with serious brains.
32. OpenPOWER: Open Hardware for High Performance
32
Systems designed for
big data analytics
and superior cloud economics
Upto:
10 cores per cpu
96 hardware threads per cpu
1/2 TB RAM
7.6Tb/s combined I/O Bandwidth
OpenPOWER
Traditional
Intel x86
http://www.softlayer.com/POWER-SERVERS
https://power.jarvice.com/landing
33. Accelerated AI: Chip and Servers
POWER8 + coherent CAPI +
novel NVlink
for high BW coherent
CPU/GPU acceleration
S822LC-hpc:
• 2 POWER8 10 Core CPUs
• 4 NVIDIA P100 ”Pascal” GPUs
• 256 GB System Memory
• 2 SSD storage devices
• High-speed interconnect
(IB or Ethernet, depending on
infrastructure)
• Optional:
• Up to 1 TB System Memory
• PCIe attached NVMe storage
“POWER8 with NVLink”
S821LC:
High Density 2-Socket 1U
S822LC for Big Data
S822LC for High
Performance Computing
Power
Linux Servers
M.Gschwind, Bringing the Deep Learning Revolution into the Enterprise
Accelerated AI
Accelerator X
33
34. Introducing 822LC Power System for HPC:
First Custom-Built GPU Accelerator Server with NVLink and NVidia P100 GPUs
M.Gschwind, Bringing the Deep Learning Revolution into the Enterprise
▪ Custom-built GPU Accelerator Server
▪ High-Speed NVLink Connections between
CPUs & GPUs and among GPUs
▪ Features novel NVIDIA P100 Pascal GPU
accelerator
NVIDIA P100 Pascal GPU
2.5x Faster CPU-GPU Data Communication
via NVLink
NVLink
80 GB/s
GPU
P8
GPU GPU
P8
GPU
POWER8 NVLink Server
PCIe
32 GB/s
GPU GPU GPU GPU
No NVLink between CPU & GPU for x86
Servers: PCIe Bottleneck
x86 Servers with PCIe
x86 x86
34
35. Higher Performance with Power8 CPU-P100 GPU NVLink
P100
GPU
POWER8
CPU
GPU
Memory
System
Memory
P100
GPU
80 GB/s
GPU
Memory
NVLink
115 GB/s
P100
GPU
POWER8
CPU
GPU
Memory
System
Memory
P100
GPU
80 GB/s
GPU
Memory
NVLink
115 GB/s
37. PowerAI vs DGX-1: 1.6xTensorFlowThroughput / Dollar
(lower cost is better)
37
• TensorFlow 0.12 on the IBM PowerAI
platform takes advantage of the full
capabilities of NVLink
• For image classification and analysis this
means a 1.6X price performance advantage
relative to the NVIDIA DGX-1
System Images /
Second
List Price $ / Image /
Second
NVIDIA DGX-1
(8 P100 GPU,
512GB Mem)
330 $129,000 $390
PowerAI (4 P100
GPU, 512 GB
Mem)
273 $67,000 $241
39. PowerAI Trial Configurations in a public cloud:
• Docker container builds and comes up in minutes
• Single P100 GPUs
• 30 days with 60 hrs standard (120 for Sales referral)
• 128GB RAM, 32 CPU threads, 1TB shared storage
• Quad P100 GPUs
• 30 days with 120hrs standard (more by request)
• 512GB RAM, 128 CPU threads, 1TB shared storage
Contact: Michael Boros
Nimbix Cloud Advantages
• Easier to use
• Highest Performance
• Ultra Fast Launch Times
• Lower Cost
• Faster time to Value
• Bare-Metal Acceleration
• Enterprise Accounting
• Application Marketplace
• Private Apps
https://www.slideshare.net/IndrajitPoddar/fast-scalable-easy-machine-
learning-with-openpower-gpus-and-docker
Experience performance
with productivity
A superior integrated stack and
adequate hardware resources for
deep learning insights
40. 40
Launch deep learning
training by one-clickData labeling
Monitor the training
progress
Deploy the inference API to
data center
Generate and deploy the DL
inference accelerator onto FPGA
DL Engineer could get
optimized model parameters
DL Insight
DL Engineer
DSX
Inject the designed DL
network into AI Vision
AI Vision
Develop the DL
neural network via
the interactive GUI
Solution
developer
PowerAI
Inference Engine
Test
engineer
Error results will be looped back
to trigger new training task
"Easier Insights with Data Science Experience and PowerAI Deep Learning" -
https://ibm.box.com/s/m7ooeoi738rs7dq9l9v0i9iir79t4xmd
Analytics Signature Moment Event in Munich:
https://www.ibm.com/analytics/us/en/events/machine-learning/
41. • a 10x increase in
inspections/day
• a 90%decrease in
inspection time
• a Significant reduction
in worker accidents
Example value realized by an Asian
Utility company using PowerAI