Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

eInfraCentral e-infrastructure performance and KPIs presentation to e-IRG Malta workshop 8 June 2017


Published on

The European e-Infrastructure Services Gateway ( project aims to:
Structure an open and guided discussion between e-infrastructures to consensually define a common catalogue for their services.
Develop a single entry point (one-stop shop) - the eInfraCentral portal - for end users to browse the service catalogue, and enhance the monitoring of key performance indicators (KPIs) that focus on availability and quality of services and user satisfaction.
Draw policy and sustainability lessons for the future development of a European e-infrastructure ‘market place’ as an extension of the common service catalogue and portal so that it includes a much broader range of e-infrastructures and services.
As part of the process of developing a Catalogue of Services (CoS) and the eInfraCentral portal, the project team will collect and structure key performance indicators (KPIs) and related data from the e-infrastructure service providers. In doing so, the project will take account of both the current state of the art of monitoring performance by the e-infrastructure project and build on the work of, and collaborate with, the e-IRG. We are also taking account of the views of service providers and users currently being collected through an online survey (open until 19 June):

The KPI dissemination and visualisation on the portal will be based on objective criteria that serve the goal to disseminate e-infrastructures services to related stakeholders and to the public-at-large in a seamless and impartial way. Different classes of KPIs will be identified in order to group individual KPIs in a more comprehensible and user-friendly manner, such as the ones related to the infrastructure capacity, the usage, the users, the investments made, and others.

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

eInfraCentral e-infrastructure performance and KPIs presentation to e-IRG Malta workshop 8 June 2017

  1. 1. e-infrastructure assessment strategy Malta, 8 June 2017 Alasdair Reid, 1
  2. 2. eInfraCentral project Project overview • 30 month H2020 CSA project – one of two ‘foundation stones’ of EOSC • Developing a harmonised service catalogue and a ’one-stop shop’ portal – Co-design with e-infrastructure service providers & users • Monitoring service performance & quality – interface with e-IRG project work on KPIs 2
  3. 3. eInfraCentral Contribution to monitoring e-infrastructure performance • During service catalogue alignment process ensure a common approach to Service Performance Monitoring – KPIs on usage, users, capacity, etc. – Services catalogued will be expected to list available KPIs • During portal development: – Providing a tool for seamlessly monitoring service performance and quality across multiple service providers • KPI choice informed by user needs elicitation survey 3
  4. 4. Catalogue of services Key Performance Indicators (KPIs) • For whom – user/funder/policy maker? • For what purpose – market visibility, choice of service, internal/external evaluation? • Building on past examples: – e-IRG, e-nventory, etc. • Input on KPI selection will be informed by the results of the user needs questionnaire 4
  5. 5. Service Catalogue Alignment Detailed timeline 01. Gather flagship European e- Infrastructures’ current catalogue/list of services 02. Draft classification of available service catalogues and validation of state-of- play 03. eInfraCentral Service Catalogue Requirements fully documented 04. Global e-Infra Initiatives provide their perspectives on service catalogue alignment 05. Service classification completed & linking KPIs to Services 06. Consensus on service classification/taxonomy achieved 07. Finalised service typologies communicated to all flagship e-Infras 09. Service representation schema defined and delivered 08. Common set of KPIs selected for adoption by e-Infrastructures 5
  6. 6. eInfraCentral Portal Service providers Functionalities 6 • Easy Registration of services • Prefilled fields and suggestions • Uniform service description & comparable service offerings across e-Infrastructures • Management of service access requests – Customer feedback gathering • Harvesting and Recording of service KPIs over time – Monitoring and Advanced Visualization of service KPIs
  7. 7. eInfraCentral Portal Service consumers Functionalities 7 • Search services using keywords • Explore/browse a catalogue of services • Browse through specific filters (faceted search) • View services organized per category • View details for a specific service • View key performance indicators (KPIs) for a specific service • Compare service characteristics and KPIs • Visually explore service stats & KPIs through different types of graphs • Different views based on user group & interest (researcher, funder, etc) • Different filters on Service types and offerings • Different highlighted KPIs • Different rankings and characteristics for comparison
  8. 8. Survey on requirement elicitation Aim and state of play • Targeted questions to service providers and users/customers, as well as potential users/customers and interested stakeholders • Survey is open until 19 June 2017 via following link – • 209 replies received by 5 June 2017: – 48% service providers – 25% current users – 26% others 8
  9. 9. Survey on requirement elicitation KPI questions to service providers 1/2 • Select from the following list those KPIs that you are currently providing to a) service users/consumers, b) policy makers/funders and c) wider public: – Service Level Agreements in place – User representatives in governing bodies – Number of up- and downtime – Number of users and its increase over time – Availability (24/7) – Number of service requests – Number of incidents/ mean time to repair – None of the above 9
  10. 10. Survey on requirement elicitation KPI questions to service providers 2/2 • List additional KPIs that you have for internal use or you are currently providing to service users/consumers, policy makers/funders and wider public and that have not been included in the above question: [open ended response] • How can your service KPIs be accessed and updated? – Download URL – Web UI – Rest API – Dedicated application – SPARQL endpoint • List KPIs for internal use or for users/consumers, policy makers/funders and wider public that you are planning to collect in the future? [open ended response] 10
  11. 11. Answers to KPI questions 1/4 KPIs that are currently provided [45 answers from 101] 0 5 10 15 20 25 30 35 40 None of the above Number of incidents/ mean time to repair Number of up- and downtime User representatives in governing bodies Number of CPU, bandwidth, storage Number of users and its increase over time Number of service requests Availability (24/7) Service Level Agreements in place 11
  12. 12. Answers to KPI questions 2/4 Additional KPIs that are not on the dropdown list [14 answers from 101] • Usage/consumption of CPU cycles • Storage per storage class per scientific domain • Number of implemented projects (facilitated use cases) • Growth in data resources • Volume of data downloaded • Increase in data published (by country, by node/partner organisation) • Publications citing the used service and increase in data citation (by dataset/ by data node/ by each country) • Different type of flagged data quality issues • Number of providers supporting open standard interfaces • Web statistics (hits, visitors) • Graphical user interface (for users) 12
  13. 13. Answers to KPI questions 3/4 Access to service KPIs [42 answers from 101] Other ways to access and update KPIs: •Quarterly reports (PDF files) •Direct link on a website •KPI info available only internally •Mix of the mentioned means + direct requests of specific KPI information 13
  14. 14. Answers to KPI questions 4/4 KPIs to be collected in the future [13 answers from 101] • Performance monitoring and benchmarking • Dependencies by other resources • Availability of helpdesk • Researchers decide, if they are willing to spend their grant money for any given e-infrastructure • Usage parameter characteristics, not only service features • Reviewing indicators that provide better insight on impact of research 14
  15. 15. @einfracentral Survey: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731049 15