2005 EPRI PowerPoint Template
Upcoming SlideShare
Loading in...5
×
 

2005 EPRI PowerPoint Template

on

  • 1,184 views

 

Statistics

Views

Total Views
1,184
Views on SlideShare
1,184
Embed Views
0

Actions

Likes
1
Downloads
8
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2005 EPRI PowerPoint Template 2005 EPRI PowerPoint Template Presentation Transcript

  • Developing Software in EPRI Software Engineering Team (SET) 2009 © 2009 Electric Power Research Institute, Inc. All rights reserved. 1
  • Software Engineering Team Quality Manager and Sector Contacts Mary McKenna Manuel Morales Tien Luong Rachel Ostraat PDU NUC ENV, GEN © 2009 Electric Power Research Institute, Inc. All rights reserved. 2
  • EPRI Software Engineering Team (SET) Venkat Natarajan Mary McKenna Manuel Morales Tien Luong Rachel Ostraat Oliver Carcallas Adam Wiseman Vu Nguyen Colette Handy Dinah Carson Dixie Herd © 2009 Electric Power Research Institute, Inc. All rights reserved. 3
  • Agenda • EPRI Software Distribution Center • Developer’s QA Process • Frequently Asked Questions (FAQs) and Common Problems • Software Development Requirements Process Table • SET Guide for Testing Your Software © 2009 Electric Power Research Institute, Inc. All rights reserved. 4
  • EPRI Software Distribution Charlotte, NC © 2009 Electric Power Research Institute, Inc. All rights reserved. 5
  • EPRI Software Distribution Center Distribute over 95% of EPRI Software and emedia products to EPRI members and non-members Library houses over 400 software and 850 emedia products We activate product records in APOLLO which captures that deliverable date has been met • Majority of orders received are processed via EPRI.com and are shipped within 24-48 hours • Prior to order being shipped, it has been screened for export control, program/project eligibility and licensing Customers can receive hard copies (CDR) of our products, or download from EPRI.com for immediate use © 2009 Electric Power Research Institute, Inc. All rights reserved. 6
  • EPRI Software Distribution Center Maintains a Nuclear Project Quality Plan and set of Project Quality Instructions Responsible for distribution of EPRI products developed under an Appendix B program. Houses an archive library of more than 550 legacy software products dating back to the early 80’s Retains source code for all active products For legacy products being updated, certain steps must be followed to allow access to source code Since 1997, all documentation for software (e.g. User Manuals) has been maintained The EPRI Software Distribution Center is extremely proud of our dedication to our customers and our #1 goal is to achieve the highest level of customer satisfaction © 2009 Electric Power Research Institute, Inc. All rights reserved. 7
  • Developer’s QA Process © 2009 Electric Power Research Institute, Inc. All rights reserved. 8
  • Developer’s QA Process • Corporate software quality assurance plan is: – A high level plan – A description of processes, procedures, and guidelines to ensure the production of high-quality, error-free software – Adaptable to multiple software projects • Each of the subject points may be handled differently by each project, depending on client’s requirements © 2009 Electric Power Research Institute, Inc. All rights reserved. 9
  • Corporate Software QA Plan Contents • Software standards • Test Plans – Unit, system, and integration testing – Beta testing • Bug Reporting and Tracking • Verification and Validation of software See the minimum list at: http://mydocs.epri.com/docs/SDRWeb/processguide/csqap.html Developers are expected to have a current software quality assurance plan within the past 18 months © 2009 Electric Power Research Institute, Inc. All rights reserved. 10
  • Frequently Asked Questions (FAQs) and Common Problems © 2009 Electric Power Research Institute, Inc. All rights reserved. 11
  • FAQs • What do I have to do for software quality approval when first submitting a contract? – Assemble the Software Contract Package – The developer and the EPRI project manager should also review the Process Table • How do I know if my software needs to follow a different process from that outlined in the basic Process Table? – The key milestones, dates, and document approvals for your software project will be listed on the Software Deliverable Requirements form in your project's Contract Package. © 2009 Electric Power Research Institute, Inc. All rights reserved. 12
  • FAQs (cont.) • What forms are required for EPRI software? – http://mydocs.epri.com/docs/SDRWeb/processguide/forms.html – Software Deliverable Requirements Form (SDRF) – Software Encryption Functions Checklist – Software Life Cycle Management Document – Software Product Description Template © 2009 Electric Power Research Institute, Inc. All rights reserved. 13
  • FAQs (cont.) • When is beta testing a requirement? – Beta testing is expected for all EPRI software. Exceptions may be made by the Sector VP. • What test services does the SET Team provide? – Alpha, Beta, Prescreen, and Final Acceptance tests © 2009 Electric Power Research Institute, Inc. All rights reserved. 14
  • FAQs (cont.) • Why do I need my application tested?--my funders don't want to spend the extra money – All EPRI software must deliver the level of usability and reliability that customers and EPRI management expect. The testing process ensures this. Beta testing and final acceptance testing are EPRI requirements. © 2009 Electric Power Research Institute, Inc. All rights reserved. 15
  • Common Problems • Problem: The application crashed because an incorrect input was entered into a data input field. – Solution: Implement range checking functions in all relevant input fields. • Problem: An anomaly was fixed in one area of the application, but not in other similar areas of the application – Solution: Review the SET test report for the use of the phrase, "check for similar occurrences" by the tester. © 2009 Electric Power Research Institute, Inc. All rights reserved. 16
  • Common Problems (cont.) • Problem: The user's manual does not follow EPRI's software manual guidelines. – Solution: Before submitting for testing, visit the Software Manual Preparations Guidelines webpage for guidelines and requirements. • Problem: The tutorial/solved example problem results did not match exactly with the actual results from the application. – Solution: Before submitting for testing, the developer needs to check the results in the documentation compared to the actual results generated by the application. © 2009 Electric Power Research Institute, Inc. All rights reserved. 17
  • Software Development Requirements Resource © 2009 Electric Power Research Institute, Inc. All rights reserved. 18
  • Where Is The Software Development Website? © 2009 Electric Power Research Institute, Inc. All rights reserved. 19
  • Where Is The Software Development Website? Via EPRI.com © 2009 Electric Power Research Institute, Inc. All rights reserved. 20
  • Where Is The Software Development Website? Location: http://mydocs.epri.com/docs/SDRWeb/processguide/index.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 21
  • Software Development Website • Features of the Software Development Requirements Website – Outlines EPRI software requirements – Provides guidance and assistance for EPRI software developers, project managers, and software quality managers on the software development life cycle Please Note: For the Nuclear Sector, these software development requirements are not applicable to software developed under, the EPRI Quality Assurance Program, which complies with 10CFR50 Appendix B, 10CFR21 and ISO 9000-1994. Software developed under the EPRI Quality Assurance Program shall follow the requirements documented in that Program. Your EPRI contract will clearly state if this separate program applies to you. © 2009 Electric Power Research Institute, Inc. All rights reserved. 22
  • Software Development Requirements Process Table © 2009 Electric Power Research Institute, Inc. All rights reserved. 23
  • Process Table: Homepage & Step 1 Click on the Requirements Process link (circled above) in the websites left-side navigation bar to go directly to the process table. Step 1. Concept Development © 2009 Electric Power Research Institute, Inc. All rights reserved. 24
  • Process Table: Step 2 to Step 7 Step 2. Defining Requirements Step 3. Design Step 4. Implementation Step 5. Alpha & Beta Test Step 6. Final Acceptance Test Step 7. Support & Maintenance © 2009 Electric Power Research Institute, Inc. All rights reserved. 25
  • EPRI Project Manager Process Table: Roadmap Major Responsibility Software Developer Software Engineering Team Step 1: Define User RFP SW Contract Step 2: SW Planning Requirements If Needed Package Documents Step 5: Beta Step 5: SW Step 3 and Step 4: Testing Prototype/Alpha Start SW & User Manual & Review Development (for PM & SET) Step 6: SW Customer Review Beta Testers’ Prescreen or Beta Testing & Feedback & Complete SW Development Final Acceptance User Feedback Bug Report Testing Step 7: Support Announcement Archive / Retire & & Distribution Maintenance © 2009 Electric Power Research Institute, Inc. All rights reserved. 26
  • Step 1: Concept Development Concept development begins when an EPRI customer specifies or confirms the need for a new software system or for modifications to an existing software product. What must be included in a Software Contract Package? • Software Deliverable Requirements Form (SDRF) • Software Life Cycle Management Document • Software Encryption Functions Checklist • Software Product Description Template • Developer Qualifications Summary • Developer Corporate Software Quality Assurance Document For additional contract package details, see: http://mydocs.epri.com/docs/SDRWeb/processguide/swdrf.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 27
  • Step 1 (cont’d) Budgeting for Testing For typical software, SET has observed that on average: • Beta test requires 14 testing hours • Final Acceptance test (usually two tests) requires 34 testing hours Therefore, planning about 48 testing hours is a safe estimate. Depending on the complexity of the software, some software will require more or less testing time. © 2009 Electric Power Research Institute, Inc. All rights reserved. 28
  • Step 2: Defining Requirements Software planning documents are developed, which address EPRI software product and process requirements according to the developer’s Corporate Software Quality Assurance Document. What planning documents does EPRI require? Example planning documents: • Software Requirements Document (SRD) • Software Development Plan (SDP) • Functional Specification (FS) For planning document information, sample content, and requirements, see: http://mydocs.epri.com/docs/SDRWeb/processguide/reqdes.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 29
  • Step 3: Design Documentation The design step begins when the software requirements have been defined. The design document provides precise directions to software programmers about how basic control and data structures will be organized. See the EPRI Software Types chart for requirements specific to the type of software being produced: http://mydocs.epri.com/docs/SDRWeb/processguide/soft_typ.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 30
  • Step 4: Implementation Software programming begins after defining the problem and designing the solution. The software is built and tested according to the planning and design documentation. Implementation Tasks are performed by developer: • Project Plan Status Reviews • Code Software • Create Solved Examples • Unit, System, and Integration Tests • Draft Documentation • Verification & Validation (V&V) © 2009 Electric Power Research Institute, Inc. All rights reserved. 31
  • Step 5: Alpha & Beta Testing The main objective of Alpha and Beta testing is to show with a high level of confidence that the software application meets the following acceptance criteria: function, performance, usability, features, and capabilities. ALPHA Testing: • Performed in the developer's environment. • Software contains most core functions, but will not contain all the intended functionality. BETA Testing: • Performed in customer environments. • Allows users to find errors and provide functionality feedback before product release. © 2009 Electric Power Research Institute, Inc. All rights reserved. 32
  • Step 5 (cont’d) - Beta Testing BETA Testing Reminders: Testers - EPRI requires at least one customer beta tester. Three or more beta testers are recommended. Approval – Before a beta software is distributed to users, SET must perform a Beta review. The acceptance for beta distribution is given within 24-Hours (72-Hours in Nov. and Dec.) of physically receiving the software. Distribution – All beta software must be distributed to users via EPRI.com. Distribution requires a beta splash Screen. For splash screen information and all other requirements, go here: http://mydocs.epri.com/docs/SDRWeb/processg uide/betapre.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 33
  • Step 6: Final Acceptance Test In the Final Acceptance Step, the software is submitted to EPRI and completed. Software is released for distribution after successfully passing Final Acceptance Testing. When must I submit my software if it is due on December 31, 2009? For software due on 12/31/09: – Beta must be submitted by 10/01/09 – Final must be submitted by 11/02/09 © 2009 Electric Power Research Institute, Inc. All rights reserved. 34
  • Step 6 (cont’d) - Final Acceptance Testing What must I submit for the final acceptance test? Final Acceptance Test submittal package: • Application CD-ROM • Source Code CD-ROM • Source Code Transmittal Letter • Software Acceptance Form (SAF) • Certificate of Conformance • Developer response to previous SET report For package details, go: http://mydocs.epri.com/docs/SDRWeb/proc essguide/achk.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 35
  • Step 7: Support & Maintenance After the final software release, support and maintenance (archiving, retiring, and bug-fixing) of the software become important as customers use the software. Software Support The EPRI Customer Assistance Center (CAC) provides first-line support. Contact CAC at (800) 313-3774 or email askepri@epri.com. Maintenance – Archiving and Retiring Software archival and retirement are handled by the EPSC after Project Manager initiation, the archival process has been followed, and approval is obtained. © 2009 Electric Power Research Institute, Inc. All rights reserved. 36
  • Step 7 (cont’d) - Maintenance Maintenance – Bug Fix Process What is the bug fix process? 1. Project Manager notifies SET of the bug 2. Developer fixes software and submits for testing 3. SET tests the updated software 4. EPSC sends software to tester for real world data test. 5. After “OK” from tester, software is distributed. 6. Customers are notified by EPSC of update. Reminder - Always budget for support and maintenance needs. © 2009 Electric Power Research Institute, Inc. All rights reserved. 37
  • Process Table: SET’s Role In Process Step 1 (Approve) - Software Contract Package Step 3 (Review) - Software Planning Documents Step 5 (Approve) – Beta Testing Step 6 (Approve) – Final Acceptance Testing (Responsible) – Distribution by EPSC Step 7 (Responsible) - Archive/Retire by EPSC © 2009 Electric Power Research Institute, Inc. All rights reserved. 38
  • SET Guide for Testing Your Software © 2009 Electric Power Research Institute, Inc. All rights reserved. 39
  • Usability Testing Sections • Installation • Solved Example Problems (or Tutorial) • User Documentation • Graphical User Interface (GUI) • Stress Testing • Security Vulnerability Testing © 2009 Electric Power Research Institute, Inc. All rights reserved. 40
  • Installation 1. Run a Virus Scan 2. Verify: – User Manual – Instructions for installation 3. If applicable, provide network installation instructions 4. Default setting installation © 2009 Electric Power Research Institute, Inc. All rights reserved. 41
  • Installation (cont’d) 5. Uninstall, then reinstall in non- default directory/drive 6. Click on Cancel button(s) during installation process 7. If serial numbers or security keys are required, enter invalid entries to make sure the security works © 2009 Electric Power Research Institute, Inc. All rights reserved. 42
  • Installation (cont’d) 8. Change the Program Folder where the shortcut in the Windows Start menu is located. 9. Applications that do not require installation, such as Spreadsheets, still require installation instructions. © 2009 Electric Power Research Institute, Inc. All rights reserved. 43
  • Solved Example Problems (or Tutorial) Reminder: Three solved example problems (or one tutorial) are required • Run solved example problems (or tutorial) to make sure all inputs and results (i.e., calculations, graphs, screen captures, etc.) in the application match exactly with the inputs and results in the user documentation Note: If any inputs or results do not match, the software can not be approved to send to customers © 2009 Electric Power Research Institute, Inc. All rights reserved. 44
  • Solved Example Problems (or Tutorial) Additional SET information and Solved Example Problems (or Tutorial): http://mydocs.epri.com/docs/SDRWeb/processguide/test case.html © 2009 Electric Power Research Institute, Inc. All rights reserved. 45
  • User Documentation 1. Check that EPRI Technical Publications User Manual template was used (or followed) Note: This ensures title page, disclaimer page, contacts page, copyright and ordering information are all current and that EPRI style guides are used 2. Check headers and footers 3. Check for system requirements: a. Hardware and Software specifications b. Permissions such as Administrator rights 4. Check application feature descriptions 5. Check spelling and grammar © 2009 Electric Power Research Institute, Inc. All rights reserved. 46
  • User Documentation • SET has a Manual template for the required documentation. • Below is a link for the documentation template: http://mydocs.epri.com/docs/SD RWeb/processguide/swurr.html #WebReq © 2009 Electric Power Research Institute, Inc. All rights reserved. 47
  • Graphical User Interface (GUI) 1. Windows fit in the main application screen and nothing is cut-off if windows are resized 2. Make sure all data/information is accessible 3. Internationalization, check multiple regions 4. Change appearance settings 5. Controls on pages must respond properly to Tab order and hot-keys (alt-keys) 6. Check online Help feature, including buttons to open the Help feature © 2009 Electric Power Research Institute, Inc. All rights reserved. 48
  • Stress Testing 1. Range checking – Look for input fields and enter invalid values 2. Make sure that numeric- only fields accept only numeric values 3. Follow the solved example problems, but then skip a step or do them in a different sequence © 2009 Electric Power Research Institute, Inc. All rights reserved. 49
  • Stress Testing (cont’d) 4. Check print feature 5. If there are logins, enter invalid login information 6. Check error messages for clarity. Error messages should appear when the error occurs. 7. Check for spelling within the application © 2009 Electric Power Research Institute, Inc. All rights reserved. 50
  • Stress Testing (cont’d) 8. For databases: a. ensure all connections through the application are valid when accessing data b. ensure single quotes and double quotes are tested to verify they do not corrupt the database c. add duplicate records d. delete all records to make sure it does not crash the application 9. Modify data files (such as adding an extra comma) to make sure the application gives a correct error message © 2009 Electric Power Research Institute, Inc. All rights reserved. 51
  • Stress Testing (cont’d) 10. For application administrative With administrative feature features, make sure only administrators of the application may access those features 11. Check for compatibility with Microsoft Office applications if Without administrative feature applicable (such as copy and paste features) 12. Click all buttons to make sure they work 13. Check save feature (does not overwrite existing file without permission, saves to correct directory, creates correct extension, etc.) © 2009 Electric Power Research Institute, Inc. All rights reserved. 52
  • Stress Testing (cont’d) 14. Check open file feature (correct file extensions, choosing incorrect file type The International Standard brings up error message, etc.) date notation 15. If there are graphs, check graph features and settings DD-MM-YYYY 16. Check options/settings not covered in the sample problems. United States Standard 17. Check to make sure international units date Notation are converted correctly MM-DD-YYYY © 2009 Electric Power Research Institute, Inc. All rights reserved. 53
  • Stress Testing (cont’d) 18. Maximize, minimize, and resize windows to make sure the application responds correctly. 19. Check keyboard shortcuts 20. Check all menu items, including the pop-up menus that come up when the user right-mouse clicks an item 21. If there are hardware/software keys, check to see if the application responds when executed with the key(s), then without the key(s) © 2009 Electric Power Research Institute, Inc. All rights reserved. 54
  • Security Vulnerability Testing © 2009 Electric Power Research Institute, Inc. All rights reserved. 55
  • Security Vulnerability Testing • OWASP Top Ten Web Application Vulnerabilities – http://www.owasp.org/index.php/OWASP_Top_Ten_Project 1. Cross Site Scripting (XSS) 2. Injection Flaws 3. Malicious File Execution 4. Insecure Direct Object Reference 5. Cross Site Request Forgery (CSRF) 6. Information Leakage and Improper Error Handling 7. Broken Authentication and Session Management 8. Insecure Cryptographic Storage 9. Insecure Communications 10. Failure to Restrict URL Access © 2009 Electric Power Research Institute, Inc. All rights reserved. 56
  • Security Vulnerability Testing • 2 examples of vulnerabilities SET will test for: – Cross-Site Scripting – Structured Query Language (SQL) Injection • The developer is expected to address security vulnerabilities when developing an application © 2009 Electric Power Research Institute, Inc. All rights reserved. 57
  • Security Vulnerability Testing (cont.) • Cross-Site Scripting - Harmful scripts are entered into web sites via querystring or form field • Example: – Enter in "<script type="text/javascript"> alert(‘hello’); </script>" into a form field to check whether the form field is validated • Allows the user to execute scripts that are harmful • See the following for more information: http://www.owasp.org/index.php/Cross-site-scripting © 2009 Electric Power Research Institute, Inc. All rights reserved. 58
  • Security Vulnerability Testing (cont.) • SQL Injection – Injection of a SQL Query through input data, such as a querystring or form • Examples: – In the querystring, enter a SQL Statement, such as " ‘; Delete from users --’ ", into a querystring variable – Enter in " ' OR 1=1 " into a form field or querystring variable • See the following for more information and testing examples: http://www.owasp.org/index.php/SQL_Injection © 2009 Electric Power Research Institute, Inc. All rights reserved. 59
  • Security Vulnerability Testing (cont.) • Testing tools: – OWASP’s Web Scarab – Acunetix Web Security Scanner – IBM Rational AppScan • Reference: – Open Web Application Security Project (OWASP) http://www.owasp.org/index.php/Main_Page © 2009 Electric Power Research Institute, Inc. All rights reserved. 60
  • What SET Does Not Do SET software usability testing does not do: 1. V&V (Verification and Validation) testing 2. test or validate real world data (this should be done by beta testers) 3. exhaustive testing or “white box” (source code) testing SET usability testing will not find all errors and is not intended to All errors are expected to be found by developers © 2009 Electric Power Research Institute, Inc. All rights reserved. 61
  • Together…Shaping the Future of Electricity © 2009 Electric Power Research Institute, Inc. All rights reserved. 62