Successfully reported this slideshow.
Your SlideShare is downloading. ×

PEARC17: The Community Software Repository from XSEDE: A Resource for the National Research Community

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 31 Ad

PEARC17: The Community Software Repository from XSEDE: A Resource for the National Research Community

Download to read offline

The Extreme Science and Engineering Discovery Environment (XSEDE) connects cyberinfrastructure (CI) resources, software, and services. One of XSEDE’s primary goals in supporting US research generally is to “advance the ecosystem” - making use of XSEDE’s leadership position to create software, tools, and services that lead to an e ective and e cient national cyberinfrastructure. Software enables this endeavor in two very distinct ways: enabling the oper- ation of XSEDE as a distributed yet integrated cyberinfrastructure resource; and by providing access to a wide variety of software of value to end user researchers and students, operators of campus cyberinfrastructure resources, and to those considering to propose new cyberinfrastructure resources to the National Science Founda- tion (NSF). The Community Software Repository (CSR) provides transparency about how XSEDE operates and provides access to software of use and value to the US research community generally. The CSR provides access to use cases that describe needs expressed by the research community, capability delivery plans that describe how XSEDE meets those needs, and the actual software that meets those needs. Software is delivered in a variety of forms and formats. The CSR also includes mechanisms for interaction between XSEDE sta , software developers, and the end user community to acceler- ate meeting of community needs and aid software developers in nding audiences for their software. XCI’s long term goal is that the XSEDE Community Software Repository will be widely used and valuable to the national research community.

The Extreme Science and Engineering Discovery Environment (XSEDE) connects cyberinfrastructure (CI) resources, software, and services. One of XSEDE’s primary goals in supporting US research generally is to “advance the ecosystem” - making use of XSEDE’s leadership position to create software, tools, and services that lead to an e ective and e cient national cyberinfrastructure. Software enables this endeavor in two very distinct ways: enabling the oper- ation of XSEDE as a distributed yet integrated cyberinfrastructure resource; and by providing access to a wide variety of software of value to end user researchers and students, operators of campus cyberinfrastructure resources, and to those considering to propose new cyberinfrastructure resources to the National Science Founda- tion (NSF). The Community Software Repository (CSR) provides transparency about how XSEDE operates and provides access to software of use and value to the US research community generally. The CSR provides access to use cases that describe needs expressed by the research community, capability delivery plans that describe how XSEDE meets those needs, and the actual software that meets those needs. Software is delivered in a variety of forms and formats. The CSR also includes mechanisms for interaction between XSEDE sta , software developers, and the end user community to acceler- ate meeting of community needs and aid software developers in nding audiences for their software. XCI’s long term goal is that the XSEDE Community Software Repository will be widely used and valuable to the national research community.

Advertisement
Advertisement

More Related Content

Similar to PEARC17: The Community Software Repository from XSEDE: A Resource for the National Research Community (20)

Recently uploaded (20)

Advertisement

PEARC17: The Community Software Repository from XSEDE: A Resource for the National Research Community

  1. 1. August 15, 2016 The Community Software Repository from XSEDE: A Resource for the National Research Community JP Navarro, Craig Stewart, Rich Knepper, Lee Liming, David Lifka, Maytal Dahan July 13, 2017
  2. 2. The Community Software Repository (CSR) from XSEDE July 13 / 12:00 pm / Strand 12 Enabling Software Sharing in the XSEDE community https://software.xsede.org/ For end users, service operators, software operators and integrators, and developers Sharing software requirements, priorities, gaps, designs, installable packages, ready-to-use software, delivery plans, documentation, and community discussions July 9-13, 2017 — New Orleans — pearc17.pearc.org 2
  3. 3. 3 XSEDE Proposal XSEDE aims to be a connector of cyberinfrastructure (CI) resources, software, and services that enable collaborations to seamlessly access advanced computing resources and share data. XSEDE Cyberinfrastructure Integration (XCI) Team Mission The mission of the XCI team is to facilitate interaction, sharing and compatibility of all relevant software and related services across the national CI community building on and improving on the foundational efforts of XSEDE. … including seamlessly integrating new CI resources supported by XSEDE and CI resources located on campuses throughout the US.
  4. 4. XSEDE Stakeholders Users Funding Sources Service Providers Software Providers, Vendors, and Integrators
  5. 5. 5 Software and services must be well designed and well documented in a clear, transparent, publicly accessible fashion. Raises two important and difficult questions? 1. How can XSEDE promote this when we are just one of many stakeholders? 2. How can XSEDE make what we do sustainable beyond our current award? Important Observation #1
  6. 6. 6 NSF Advisory Committee on Cyberinfrastructure (2011). Finding: “The current state of cyberinfrastructure software and current levels of expert support for use of cyberinfrastructure create barriers in use of the many and varied campus and national cyberinfrastructure facilities. These barriers prevent the US open science and engineering research community from using the existing, open US cyberinfrastructure as effectively and efficiently as possible.” Recommendation: “The NSF should fund activities that support the evolution and maturation of cyberinfrastructure through careful analyses of needs (in advance of creating new cyberinfrastructure facilities) and outcomes (during and after the use of cyberinfrastructure facilities). The NSF should establish and fund processes for collecting disciplinary community requirements and planning long-term cyberinfrastructure software road maps to support disciplinary community research objectives.” Important Observation #2
  7. 7. The XSEDE 1 Past – We Built Elaborate Engineering Process Specific Architecture Specific Software Partners
  8. 8. Why Change? • Make XSEDE’s software activities more streamlined and transparent • Enable the community to use XSEDE’s software and service delivery best practices • Enable more software and services to integrate into XSEDE • Enable more infrastructure to integrate into XSEDE • Expand capabilities available to XSEDE users 8
  9. 9. The XSEDE 2 Present – We Facilitate Streamlined Delivery Process Pragmatic Architecture New Tool: CSR Increased Focus on Community Software Provider Increased Focus on Un-Allocated Service Provider
  10. 10. XSEDE 2 – Areas of Change Streamlined Integration Process 1. Engineering Process (software, services, and infrastructure) 2. Pragmatic Architecture (community driven) Greater Community Focus 3. Campuses and un-allocated service providers 4. Community Software Providers New Community Enabling Tool 5. The Community Software Repository (CSR)
  11. 11. Change - Streamlined Integration Process 11 CI Users CI Providers Software Partners CI Providers Use Cases Transparent Process Software Catalogs Docs
  12. 12. Greater Focus on: 1) Unallocated (infrastructure) Service Providers Integrating campuses and provide special software services for campuses Examples: SSO Hub, working with L3s to advertise their HPC resources 1) Community Software Providers Integrating pre-existing software and services from the community and from vendors Examples: Globus Services, Kepler Change - Greater Community Focus
  13. 13. 13 VISION: To enabling transparent and traceable software requirements, priorities, delivery, sharing, and discovery. Supports the entire software delivery process (not just a software catalog): 1. Understanding Requirements 2. Preparing Software Solutions 3. Sharing Software Solutions 4. Discovering Software Solutions 5. Return on Investment (ROI) Community Software Repository
  14. 14. Key roles: • Front door to all XCI software activities • “Home base” for the extended XSEDE software ecosystem where developers, integrators, operators, users, and funding agencies collaborate to accelerate resource, software, and software based service integration into the national CI XSEDE must participate and enable: • XSEDE software activities are discoverable thru the CSR • Community can pick and choose CSR elements to share and discover 14 CSR Roles
  15. 15. 15 • XSEDE organization • End Users • Campus CI Operators • Service Providers (SPs) • Software Developers and Integrators • Organizations competing for XSEDE related software awards • Funding agencies Each stakeholder has a different set of needs! CSR Customers / Stakeholders
  16. 16. 1. Understanding Requirements 2. Preparing Software Solutions 3. Sharing Software Solutions 4. Discovering Software Solutions CSR Focus Areas
  17. 17. 17 Supporting capabilities: • Requirement discussions • A Use Case Registry tracks the driving requirements • Capability Delivery Plans detail use case support components and gaps • Stakeholder use case priorities • Proposed and planned Engineering activities needed to fill capability caps CSR - 1. Understanding Requirements
  18. 18. 18 Supporting capabilities: • Track active engineering effort delivering capabilities • Sharing engineering and security designs • Managing engineering reviews • Engineering discussions CSR - 2. Preparing Software Solutions
  19. 19. 19 Supporting capabilities: • Repositories: –RPM Repository –Tar Repository –Future VM, container, and build recipe repositories • Software metadata catalogs –From repositories –Operational NOTE: our expectation is that the vast majority of software described in the CSR will be hosted externally in other repositories. CSR - 3. Sharing Software Solutions
  20. 20. 20 Supporting capabilities: • Use case discovery interfaces • Repository and operational software and service discovery interfaces • Discussion Forums: –Deployment support discussions –Use discussions CSR - 4. Discovering Software Solutions
  21. 21. Campus examples XNIT “Pull the lever, build a cluster from scratch”XCBC Add XSEDE-like functionality to a existing and functioning cluster Install community software (packaged by XCI) via Community Software Repository (by hand or by subscription from YUM repo) At present XNIT toolkits (YUM repo) contains a total of 37 software titles packaged as relocatable RPMs. Counting across different packaging methods (different MPI implementations, different libraries, development versions vs. production versions) this is a total of 142 separate packages
  22. 22. 22 End Users examples • What can users do (use cases)? • What software is available on SP resources? • XSEDE User Portal
  23. 23. 23 Developers examples • API information • Capability gaps
  24. 24. 24 Service Provider examples • Software packages: • Integration status: • Operations status:
  25. 25. 25 https://software.xsede.org/
  26. 26. 26 Future CSR capabilities • Science Gateway software • Community Software Area (CSA) software • Campus and L3 Service Provider software • XSEDE enterprise services • VMs, containers, and build recipe sharing • Community software and service ratings • Community discussion forums and … what will help you? https://software.xsede.org/feedback
  27. 27. END OF PRESENTATION 27
  28. 28. Supplemental material 28
  29. 29. XCI Provided Software - operating XSEDE Service Provider Resources XSEDE Federation Accounting & Allocations Description & Status Login Data Movement Execution Common User Environment User Portal Information Services Security Services Accounting Service Login Services Data Services Monitoring Services Enterprise Services Blue boxes are XCI provided
  30. 30. 30 Architecture Design, Decomposition Requirements: Use Cases, Qualities Detailed Implementation Plans Design, Integration, (Development) Coordinated Deployment Production Support Architecture Active Design Review Integration Test Team(s): A&D, S&SE, TIS XSEDE Software Engineering Flow Acceptance Test Team(s): SD&I Area(s): Operations, User Services Users Pilots Service Providers Team(s): TEOS, ECSS, … User & Operator experience Operations enhancements Bug fixes, routine enhancements Need for new capabilities Implementation-ready architecture Deployment-ready software and service Operational infrastructure

×