Best Practices for Rigorous Evaluation
of RFID Software Components


 RFID-SysTech 2010, 06/15/2010

 Matthieu-P. Schapran...
Outline

2

     ■  Introduction
     ■  Use Case: The European Pharmaceutical Supply Chain
     ■  Generating Realistic T...
Introduction

3

     ■  Conducting rigorous evaluation of software artifacts is hard
     ■  This is especially true for ...
Outline

4

     ■  Introduction
     ■  Use Case: The European Pharmaceutical Supply Chain
     ■  Generating Realistic T...
Use Case: The European
    Pharmaceutical Supply Chain

5




    Best Practices for Rigorous Evaluation of RFID Software ...
Use Case: The European
    Pharmaceutical Supply Chain

6




    Best Practices for Rigorous Evaluation of RFID Software ...
Use Case: The European
    Pharmaceutical Supply Chain

7




    Best Practices for Rigorous Evaluation of RFID Software ...
Outline

8

     ■  Introduction
     ■  Use Case: The European Pharmaceutical Supply Chain
     ■  Generating Realistic T...
Generating Realistic Test Data

9

     ■  Two possibilities
          □  Data Generators
          □  Simulation
     ■  ...
Outline

10

      ■  Introduction
      ■  Use Case: The European Pharmaceutical Supply Chain
      ■  Generating Realist...
Selected Metrics to Consider

11

      ■  Application Performance Index (Apdex)
           □  satisfied:          0≤t<x
 ...
Selected Metrics to Consider

12

      ■  Throughput
           □  A = Amount of completed tasks
           □  T = Observ...
Outline

13

      ■  Introduction
      ■  Use Case: The European Pharmaceutical Supply Chain
      ■  Generating Realist...
Benchmarking EPC Network Components

14

      ■  System in test
           □  EPCIS server and/ or
           □  EPC Disc...
Single System in Test

15




     Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
Multiple Systems in Test

16

      ■  Define system URLs in JMeter “user.properties”,




      ■  Store EPCIS server add...
Multiple Systems in Test

17




     Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
Multiple Systems in Test

18

        ■  and send events to respective EPCIS servers.




        ■  Files and Scripts for...
Outline

19

      ■  Introduction
      ■  Use Case: The European Pharmaceutical Supply Chain
      ■  Generating Realist...
Summary

20

      ■  Evaluating software systems is important
      ■  How to create realistic data  simulation
      ■ ...
21




                                Thank you for
                                your attention!


                   ...
Upcoming SlideShare
Loading in …5
×

Best Practices for Rigorous Evaluation of RFID Software Components

773 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
773
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Best Practices for Rigorous Evaluation of RFID Software Components

  1. 1. Best Practices for Rigorous Evaluation of RFID Software Components RFID-SysTech 2010, 06/15/2010 Matthieu-P. Schapranow
  2. 2. Outline 2 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  3. 3. Introduction 3 ■  Conducting rigorous evaluation of software artifacts is hard ■  This is especially true for RFID software components □  high throughput and □  large data volume ■  We want to share our insights into evaluating RFID components □  EPCIS servers and □  EPC Discovery Services Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  4. 4. Outline 4 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  5. 5. Use Case: The European Pharmaceutical Supply Chain 5 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  6. 6. Use Case: The European Pharmaceutical Supply Chain 6 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  7. 7. Use Case: The European Pharmaceutical Supply Chain 7 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  8. 8. Outline 8 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  9. 9. Generating Realistic Test Data 9 ■  Two possibilities □  Data Generators □  Simulation ■  Our conclusion: only simulation can reflect the complex dependencies of RFID-aided supply chains ■  We implemented a special-purpose simulator ■  Four scenarios of the pharmaceutical supply chain Interference No interference Scale 1:2000 Scenario 1 Scenario 3 Scale 1:1000 Scenario 2 Scenario 4 ■  Download at epic.hpi.uni-potsdam.de/Home/RFIDSySTech2010 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  10. 10. Outline 10 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  11. 11. Selected Metrics to Consider 11 ■  Application Performance Index (Apdex) □  satisfied: 0≤t<x □  tolerating: x ≤ t < 4x □  frustrating: t ≥ 4x □  RS = number of satisfied response time samples □  RT = number of tolerating response time samples □  N = number of all samples in the report group Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  12. 12. Selected Metrics to Consider 12 ■  Throughput □  A = Amount of completed tasks □  T = Observation time Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  13. 13. Outline 13 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  14. 14. Benchmarking EPC Network Components 14 ■  System in test □  EPCIS server and/ or □  EPC Discovery Service ■  Apache JMeter as a test tool □  Apache Project □  Widely accepted performance test tool for web applications □  Property files for variables, e.g. server URLs □  Very flexible through BeanShell scripting Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  15. 15. Single System in Test 15 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  16. 16. Multiple Systems in Test 16 ■  Define system URLs in JMeter “user.properties”, ■  Store EPCIS server addresses in JMeter variables Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  17. 17. Multiple Systems in Test 17 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  18. 18. Multiple Systems in Test 18 ■  and send events to respective EPCIS servers. ■  Files and Scripts for download at epic.hpi.uni-potsdam.de/Home/ RFIDSySTech2010 Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  19. 19. Outline 19 ■  Introduction ■  Use Case: The European Pharmaceutical Supply Chain ■  Generating Realistic Test Data ■  Selected Metrics to Consider ■  Benchmarking EPC Network Components ■  Summary Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  20. 20. Summary 20 ■  Evaluating software systems is important ■  How to create realistic data  simulation ■  What to measure  see “metrics to consider” (Apdex, throughput) ■  How to put load on the software system  Apache JMeter ■  How to test multiple systems at once  JMeter & BeanShell Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010
  21. 21. 21 Thank you for your attention! Questions? Best Practices for Rigorous Evaluation of RFID Software Components, RFID-SysTech 2010

×