SlideShare a Scribd company logo
Update: What’s Missing from
HPC?


       John West
       DoD HPC Modernization Program
If you’re in this room, you
probably think HPC is a great idea.
                                 HPCC Newport 2013
                                           Page-2
Peak Computational Capability by Country
And as we heard last year   1993
countries all over the
world are racing to close
the leadership gap in
capabilities once held by
just a small number of
nations.

                            2012




                            Data : top500.org                          HPCC Newport 2013
                                                                                 Page-3
Technical Computing

 Supercomputing                         Few users




High Performance                       Few users
       Computing
                    “Missing Middle”




       Individual                      Many users
      Computing


                                                    HPCC Newport 2013
                                                              Page-4
So…where are they?


                     HPCC Newport 2013
                               Page-5
HPCC Newport 2013
          Page-6
HPCC Newport 2013
          Page-7
HPCC Newport 2013
          Page-8
The HPC and
supercomputing
users we have
today didn’t have a
choice.




                      HPCC Newport 2013
                                Page-9
The choice was
obvious for
individual computer
users.


And the middle,
stayed at the
bottom.


                      HPCC Newport 2013
                               Page-10
The journey from the bottom.
                               HPCC Newport 2013
                                        Page-11
HPC User

                        • Incomplete toolchain

                         • Little expertise and no social support

                       • Primitive interfaces

                         • Complex management

                       • Expensive hardware and software
Individual Computing
User                                                        HPCC Newport 2013
                                                                     Page-12
Why do they stay at the bottom?

They already have something
that works, and it’s too hard to
just “take ‘er for a spin.”
                                  HPCC Newport 2013
                                           Page-13
How do we increase the reach of HPC?
Hardware has gotten cheaper and better
There are system management options that
 reduce deployment complexity
 Interfaces are primitive (some work needed)
 Incomplete toolchain (hand to hand combat)
Little expertise and no social support

“I don’t know how, and there’s no one around here
                                        I can ask!”

                                                 HPCC Newport 2013
                                                          Page-14
Makers and Takers
 HPC Consumer
  – Use high performance computers
  – Run applications
  – Understand computing principles
 HPC Provider
  – Run and design high performance computers
  – Write and extend applications
  – Master computing principles


In practice this is a continuous spectrum, and
workers may move in either direction during their
career.
                                                    HPCC Newport 2013
                                                             Page-15
An interagency (NITRD) position

 The NITRD High End
  Computing Interagency
  Working Group (HEC-IWG)
  position on education and
  workforce development
  (Mar 2013)
 Articulates foundational
  principles
 Starting place for coordinated
  agency programs that will
  build a workforce
 NITRD is not a funding
  agency

The Networking and Information Technology Research and Development
Program, www.nitrd.gov
                                                                     HPCC Newport 2013
                                                                              Page-16
NITRD Position Overview
 Affirms the importance of HPC/HEC in national
  security and competitiveness terms
 Reviews DOE/NNSA-funded survey on
  characteristics of the HEC workforce
  – Statistics remain a problem for this segment of the workforce
 Articulates foundational principles that must be
  addressed for success




                                                                    HPCC Newport 2013
                                                                             Page-17
DoE HPC Provider Study
IDC HPC User Forum: Special Study (July 2010). A Study of the Talent
                 and Skill Set Issues Impacting HPC Data Centers.
 Staffing is hard
  – 93% of HPC centers surveyed said that hiring qualified staff is
     “somewhat hard” or “very hard” with the majority reporting that it is “very
     hard” to find qualified staff.

 Where do staff come from?
  – STEM grads
  – Other HPC centers
  – HPC vendors
 What skills are needed on the provider side?
  – Combined understanding of a scientific discipline and computational
     science and/or computer science; parallel programming and code
     optimization, especially for scaling to large processor/core counts;
     algorithm development; HEC system administration; and understanding
     of parallel file systems.
                                                                              HPCC Newport 2013
                                                                                       Page-18
The NITRD Principles
 An effective program
  – Increases the impact of HPC/HEC
  – Must address the entire spectrum consumer  provider




                                                           HPCC Newport 2013
                                                                    Page-19
The NITRD Principles
 An effective program
  – Increases the impact of HPC/HEC
  – Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
  just as important as increasing STEM grads
  – Many of us came to HPC after practicing in a discipline that uses it
  – Steal whenever possible (executive MBAs, certificates, …)




                                                                           HPCC Newport 2013
                                                                                    Page-20
The NITRD Principles
 An effective program
  – Increases the impact of HPC/HEC
  – Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
  just as important as increasing STEM grads
 If we want to teach it we have to define it
  – Enumerate the skill vectors that span our space (steal when possible):
     admins, architects, developers, …
  – Then work with traditional and non-traditional education partners on
     curricula




                                                                             HPCC Newport 2013
                                                                                      Page-21
The NITRD Principles
 An effective program
  – Increases the impact of HPC/HEC
  – Must address the entire spectrum consumer  provider
 Career transition for those already in the workforce
  just as important as increasing STEM grads
 If we want to teach it we have to define it
 …and reinforce it
  – (Continue to) fund research that gives academic community experience
     with real world (ish) problems
  – Internships, fellowships, awards, etc.
  – Establishing and illuminating HPC career paths will help with recruitment
     and retention (certifications? Maybe eventually…)




                                                                           HPCC Newport 2013
                                                                                    Page-22
Next Steps for NITRD HEC Members
 Define a set of career paths and skillsets
 Map the union of current efforts, identify gaps
 With educators, describe and develop curricula that
  will produce new Providers and Consumers
 Pilot new, more flexible methods of education and
  workforce development that enable in-career
  transitions
 Continue to fund relevant academic research
  problems, internships, graduate and post-doctoral
  fellowships, and partnerships with industry and
  academia.


                             ...and share, share, share
                                                      HPCC Newport 2013
                                                               Page-23
Read it at goo.gl/e03fU
Comment at john.west@hpc.mil
                               HPCC Newport 2013
                                        Page-24

More Related Content

Similar to What’s Missing from HPC?

Uber cloud at ucc dresden dec 2013
Uber cloud at ucc dresden dec 2013Uber cloud at ucc dresden dec 2013
Uber cloud at ucc dresden dec 2013
Wolfgang Gentzsch
 
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAGCloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
Andrew Jones
 
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
Dana Gardner
 
OpenPOWER's ISC 2016 Recap
OpenPOWER's ISC 2016 RecapOpenPOWER's ISC 2016 Recap
OpenPOWER's ISC 2016 Recap
OpenPOWERorg
 
Interventions for scientific and enterprise applications
Interventions for scientific and enterprise applicationsInterventions for scientific and enterprise applications
Interventions for scientific and enterprise applications
eSAT Publishing House
 
Interventions for scientific and enterprise applications based on high perfor...
Interventions for scientific and enterprise applications based on high perfor...Interventions for scientific and enterprise applications based on high perfor...
Interventions for scientific and enterprise applications based on high perfor...
eSAT Journals
 
GridEngine Summit Keynote about Uber Cloud Experiment
GridEngine Summit Keynote about Uber Cloud ExperimentGridEngine Summit Keynote about Uber Cloud Experiment
GridEngine Summit Keynote about Uber Cloud Experiment
hpcexperiment
 
Gridcomputing
GridcomputingGridcomputing
Gridcomputing
pchengi
 
Introducing the Jisc National HPC Agreement
Introducing the Jisc National HPC AgreementIntroducing the Jisc National HPC Agreement
Introducing the Jisc National HPC Agreement
Martin Hamilton
 
Digital Railways presentation by Technology Strategy Board
Digital Railways presentation by Technology Strategy BoardDigital Railways presentation by Technology Strategy Board
Digital Railways presentation by Technology Strategy Board
KTN
 
UberCloud HPC Experiment Introduction for Beginners
UberCloud HPC Experiment Introduction for BeginnersUberCloud HPC Experiment Introduction for Beginners
UberCloud HPC Experiment Introduction for Beginners
hpcexperiment
 
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
JISC's Green ICT Programme
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance Computing
Nous Infosystems
 
Pioneering and Democratizing Scalable HPC+AI at PSC
Pioneering and Democratizing Scalable HPC+AI at PSCPioneering and Democratizing Scalable HPC+AI at PSC
Pioneering and Democratizing Scalable HPC+AI at PSC
inside-BigData.com
 
ISC 2016 Day 2 Recap
ISC 2016 Day 2 RecapISC 2016 Day 2 Recap
ISC 2016 Day 2 Recap
OpenPOWERorg
 
CloudLightning - Project and Architecture Overview
CloudLightning - Project and Architecture OverviewCloudLightning - Project and Architecture Overview
CloudLightning - Project and Architecture Overview
CloudLightning
 
North mobile data capture
North mobile data captureNorth mobile data capture
North mobile data capture
Stweeve
 
Engaging with HPC Midlands - Next Steps
Engaging with HPC Midlands - Next StepsEngaging with HPC Midlands - Next Steps
Engaging with HPC Midlands - Next Steps
Martin Hamilton
 
Bangladesh ic-design-program-rev4-1 bd
Bangladesh ic-design-program-rev4-1 bdBangladesh ic-design-program-rev4-1 bd
Bangladesh ic-design-program-rev4-1 bd
khalid noman husainy
 
Scaling the mirrorworld with knowledge graphs
Scaling the mirrorworld with knowledge graphsScaling the mirrorworld with knowledge graphs
Scaling the mirrorworld with knowledge graphs
Alan Morrison
 

Similar to What’s Missing from HPC? (20)

Uber cloud at ucc dresden dec 2013
Uber cloud at ucc dresden dec 2013Uber cloud at ucc dresden dec 2013
Uber cloud at ucc dresden dec 2013
 
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAGCloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
Cloud for HPC - UKRI Cloud Workshop 2019 - Andrew Jones NAG
 
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
Converged Infrastructure Approach Paves Way for Improved Data Center Producti...
 
OpenPOWER's ISC 2016 Recap
OpenPOWER's ISC 2016 RecapOpenPOWER's ISC 2016 Recap
OpenPOWER's ISC 2016 Recap
 
Interventions for scientific and enterprise applications
Interventions for scientific and enterprise applicationsInterventions for scientific and enterprise applications
Interventions for scientific and enterprise applications
 
Interventions for scientific and enterprise applications based on high perfor...
Interventions for scientific and enterprise applications based on high perfor...Interventions for scientific and enterprise applications based on high perfor...
Interventions for scientific and enterprise applications based on high perfor...
 
GridEngine Summit Keynote about Uber Cloud Experiment
GridEngine Summit Keynote about Uber Cloud ExperimentGridEngine Summit Keynote about Uber Cloud Experiment
GridEngine Summit Keynote about Uber Cloud Experiment
 
Gridcomputing
GridcomputingGridcomputing
Gridcomputing
 
Introducing the Jisc National HPC Agreement
Introducing the Jisc National HPC AgreementIntroducing the Jisc National HPC Agreement
Introducing the Jisc National HPC Agreement
 
Digital Railways presentation by Technology Strategy Board
Digital Railways presentation by Technology Strategy BoardDigital Railways presentation by Technology Strategy Board
Digital Railways presentation by Technology Strategy Board
 
UberCloud HPC Experiment Introduction for Beginners
UberCloud HPC Experiment Introduction for BeginnersUberCloud HPC Experiment Introduction for Beginners
UberCloud HPC Experiment Introduction for Beginners
 
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
Migrating from a physical to a hosted Data Centre - Experiences of a small Un...
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance Computing
 
Pioneering and Democratizing Scalable HPC+AI at PSC
Pioneering and Democratizing Scalable HPC+AI at PSCPioneering and Democratizing Scalable HPC+AI at PSC
Pioneering and Democratizing Scalable HPC+AI at PSC
 
ISC 2016 Day 2 Recap
ISC 2016 Day 2 RecapISC 2016 Day 2 Recap
ISC 2016 Day 2 Recap
 
CloudLightning - Project and Architecture Overview
CloudLightning - Project and Architecture OverviewCloudLightning - Project and Architecture Overview
CloudLightning - Project and Architecture Overview
 
North mobile data capture
North mobile data captureNorth mobile data capture
North mobile data capture
 
Engaging with HPC Midlands - Next Steps
Engaging with HPC Midlands - Next StepsEngaging with HPC Midlands - Next Steps
Engaging with HPC Midlands - Next Steps
 
Bangladesh ic-design-program-rev4-1 bd
Bangladesh ic-design-program-rev4-1 bdBangladesh ic-design-program-rev4-1 bd
Bangladesh ic-design-program-rev4-1 bd
 
Scaling the mirrorworld with knowledge graphs
Scaling the mirrorworld with knowledge graphsScaling the mirrorworld with knowledge graphs
Scaling the mirrorworld with knowledge graphs
 

Recently uploaded

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
FODUU
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 

Recently uploaded (20)

Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 

What’s Missing from HPC?

  • 1. Update: What’s Missing from HPC? John West DoD HPC Modernization Program
  • 2. If you’re in this room, you probably think HPC is a great idea. HPCC Newport 2013 Page-2
  • 3. Peak Computational Capability by Country And as we heard last year 1993 countries all over the world are racing to close the leadership gap in capabilities once held by just a small number of nations. 2012 Data : top500.org HPCC Newport 2013 Page-3
  • 4. Technical Computing Supercomputing Few users High Performance Few users Computing “Missing Middle” Individual Many users Computing HPCC Newport 2013 Page-4
  • 5. So…where are they? HPCC Newport 2013 Page-5
  • 9. The HPC and supercomputing users we have today didn’t have a choice. HPCC Newport 2013 Page-9
  • 10. The choice was obvious for individual computer users. And the middle, stayed at the bottom. HPCC Newport 2013 Page-10
  • 11. The journey from the bottom. HPCC Newport 2013 Page-11
  • 12. HPC User • Incomplete toolchain • Little expertise and no social support • Primitive interfaces • Complex management • Expensive hardware and software Individual Computing User HPCC Newport 2013 Page-12
  • 13. Why do they stay at the bottom? They already have something that works, and it’s too hard to just “take ‘er for a spin.” HPCC Newport 2013 Page-13
  • 14. How do we increase the reach of HPC? Hardware has gotten cheaper and better There are system management options that reduce deployment complexity  Interfaces are primitive (some work needed)  Incomplete toolchain (hand to hand combat) Little expertise and no social support “I don’t know how, and there’s no one around here I can ask!” HPCC Newport 2013 Page-14
  • 15. Makers and Takers  HPC Consumer – Use high performance computers – Run applications – Understand computing principles  HPC Provider – Run and design high performance computers – Write and extend applications – Master computing principles In practice this is a continuous spectrum, and workers may move in either direction during their career. HPCC Newport 2013 Page-15
  • 16. An interagency (NITRD) position  The NITRD High End Computing Interagency Working Group (HEC-IWG) position on education and workforce development (Mar 2013)  Articulates foundational principles  Starting place for coordinated agency programs that will build a workforce  NITRD is not a funding agency The Networking and Information Technology Research and Development Program, www.nitrd.gov HPCC Newport 2013 Page-16
  • 17. NITRD Position Overview  Affirms the importance of HPC/HEC in national security and competitiveness terms  Reviews DOE/NNSA-funded survey on characteristics of the HEC workforce – Statistics remain a problem for this segment of the workforce  Articulates foundational principles that must be addressed for success HPCC Newport 2013 Page-17
  • 18. DoE HPC Provider Study IDC HPC User Forum: Special Study (July 2010). A Study of the Talent and Skill Set Issues Impacting HPC Data Centers.  Staffing is hard – 93% of HPC centers surveyed said that hiring qualified staff is “somewhat hard” or “very hard” with the majority reporting that it is “very hard” to find qualified staff.  Where do staff come from? – STEM grads – Other HPC centers – HPC vendors  What skills are needed on the provider side? – Combined understanding of a scientific discipline and computational science and/or computer science; parallel programming and code optimization, especially for scaling to large processor/core counts; algorithm development; HEC system administration; and understanding of parallel file systems. HPCC Newport 2013 Page-18
  • 19. The NITRD Principles  An effective program – Increases the impact of HPC/HEC – Must address the entire spectrum consumer  provider HPCC Newport 2013 Page-19
  • 20. The NITRD Principles  An effective program – Increases the impact of HPC/HEC – Must address the entire spectrum consumer  provider  Career transition for those already in the workforce just as important as increasing STEM grads – Many of us came to HPC after practicing in a discipline that uses it – Steal whenever possible (executive MBAs, certificates, …) HPCC Newport 2013 Page-20
  • 21. The NITRD Principles  An effective program – Increases the impact of HPC/HEC – Must address the entire spectrum consumer  provider  Career transition for those already in the workforce just as important as increasing STEM grads  If we want to teach it we have to define it – Enumerate the skill vectors that span our space (steal when possible): admins, architects, developers, … – Then work with traditional and non-traditional education partners on curricula HPCC Newport 2013 Page-21
  • 22. The NITRD Principles  An effective program – Increases the impact of HPC/HEC – Must address the entire spectrum consumer  provider  Career transition for those already in the workforce just as important as increasing STEM grads  If we want to teach it we have to define it  …and reinforce it – (Continue to) fund research that gives academic community experience with real world (ish) problems – Internships, fellowships, awards, etc. – Establishing and illuminating HPC career paths will help with recruitment and retention (certifications? Maybe eventually…) HPCC Newport 2013 Page-22
  • 23. Next Steps for NITRD HEC Members  Define a set of career paths and skillsets  Map the union of current efforts, identify gaps  With educators, describe and develop curricula that will produce new Providers and Consumers  Pilot new, more flexible methods of education and workforce development that enable in-career transitions  Continue to fund relevant academic research problems, internships, graduate and post-doctoral fellowships, and partnerships with industry and academia. ...and share, share, share HPCC Newport 2013 Page-23
  • 24. Read it at goo.gl/e03fU Comment at john.west@hpc.mil HPCC Newport 2013 Page-24