MobileBench Consortium 2013

2,876 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,876
On SlideShare
0
From Embeds
0
Number of Embeds
47
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MobileBench Consortium 2013

  1. 1. Copyright© MobileBench Consortium, 2013 MobileBenchMobileBench ConsortiumConsortium MobileBench Standard performance metric for Mobile 2013.07
  2. 2. Copyright© MobileBench Consortium, 2013 Lack of credible Benchmark tools • The biggest motivation of developing a new benchmark tool was the lack of credibility, fairness, and accuracy among conventional benchmark tools in the market.  No detailed information for deviceNo detailed information for device comparisoncomparison  Small Test items and coverageSmall Test items and coverage  HardHard to get feedbackto get feedback  Difficult to interpret the resultsDifficult to interpret the results  Wide fluctuationWide fluctuation  User experiences reflected insufficientlyUser experiences reflected insufficiently 1 2 3 4
  3. 3. Copyright© MobileBench Consortium, 2013 Goal for MobileBench consortium  FairnessFairness Ensure the fairness through opening source-code  ConsistencyConsistency Minimize the standard deviation of results + User Experience+ User Experience the performance evaluation based-on user scenario + Flexibility+ Flexibility Support various evaluation functions for Analysis  DistinctionDistinction Maximize device performance
  4. 4. Copyright© MobileBench Consortium, 2013 MobileBenchTM Strength (1/5) Avg: 0.7 Avg: 4.4 Avg: 12.0 Avg: 0.6  Consistency Minimize the standard deviation of results
  5. 5. Copyright© MobileBench Consortium, 2013 MobileBenchTM Strength (2/5)  Distinction Maximize device performance 108% 73% 17% 12% Spec.
  6. 6. Copyright© MobileBench Consortium, 2013 MobileBenchTM Strength (3/5)  Flexibility Support various evaluation functions for analysis
  7. 7. Copyright© MobileBench Consortium, 2013 MobileBenchTM Strength (4/5) MobileBenchMobileBenchTMTM ResultResult  User Experience the performance evaluation based-on user scenario Other ToolOther Tool ResultResult Difference of storage performance MobileBenchMobileBenchTMTM --UXUX ResultResult
  8. 8. Copyright© MobileBench Consortium, 2013 MobileBenchTM Strength (5/5)  System Monitoring Monitor the behavior of each device in the background Processor Usage & Freq. Memory Usage Storage I/O MFLOPS/MIPS/Memory/StorageMFLOPS/MIPS/Memory/Storage Java-scriptJava-script Web-loading / HTML5Web-loading / HTML5 UI-stressUI-stress * Time resolution 1sec* Time resolution 1sec
  9. 9. Copyright© MobileBench Consortium, 2013 MobileBench Suite MobileBench MobileBench-UX MobileBenchMobileBench SuiteSuite  Mobile ApplicationMobile Application  Device Benchmark onDevice Benchmark on PlatformPlatform  Device Benchmark onDevice Benchmark on KernelKernel  Synthetic WorkloadSynthetic Workload  System MonitorSystem Monitor  Mobile App. +Mobile App. + PC AgentPC Agent  Mobile Device Control via USBMobile Device Control via USB  Real Workload/ApplicationReal Workload/Application  ScenarioScenario--basedbased  System MonitorSystem Monitor
  10. 10. Copyright© MobileBench Consortium, 2013 MobileBench Features • MobileBench is a total benchmark solution for mobile devices. It can measure the performance of each H/W component as well as web- related and UI performance. 1. System Performance: - CPU/Memory/Storage 2. Web browsing: - Javascript, Web Loading, HTML5 compatibility 3. User Experience: - U/I Stress, Battery Eater 4. Special Feature: - System Monitor, Email Reporting MobileBenchTM Features
  11. 11. Copyright© MobileBench Consortium, 2013 MobileBench Features Features AP Memory Storage Function in Advanced Mode Processor Core #, Time control Memory Thread #, Request size Storage Chunk size, Pre-condition, Native access Java Script None Web Loading None HTML5 Compatible None Graphic 3D Graphic (TBD) MobileBenchTM
  12. 12. Copyright© MobileBench Consortium, 2013 MobileBench-UX Features • MobileBench-UX is a PC software to measure UX performance controlling mobile devices with PC via USB connection. It can simulate all the different user scenario, changing the combination of scenario items. Since all the test items are what users actually do every day, this tool can actually represent and score the real user experience. One Click! UX based bench Multitasking bench Multimedia bench Automatic reporting CameraVideo Playback Video Creating Application Installation Application Loading Media Scanning I/O Operation Boot Test
  13. 13. Copyright© MobileBench Consortium, 2013 MobileBench-UX Features Category Item AP Memory Storage Function Utility  Reboot Rebooting time  Media Scanning Android media scan service time  External File I/O PC to phone file copy time  MTP Test Android media transfer protocol  App. Install Application install time Multi- media  Video Playback-S Movie frame drop/delay time - Background storage file copy  Video Playback-M Movie frame drop/delay time - Background memory access  Video Creating MPEG-1 video encoding time  Video Trans-coding Movie, MPEG-1 video encoding time  Camera Taking pictures and saving time Multi- tasking  App. Switching 20-dummy application switching time  Multi-Popup Dual-movie play  Web Browsing Web page loading & zoom in-out MobileBenchTM-UX
  14. 14. Copyright© MobileBench Consortium, 2013 MobileBench Consortium • Organization Structure Board of Director (Chair & Directors) Special Committees Work Committees CFO Secretary President Tool Maintainer Board of Director Work Committee Spec. release Approval Spec. Approval Specification & Tools Developer Yes Yes Yes Yes Yes Adopter No Yes No Yes Yes Honorary* No Yes No No Yes Non-member No No No No No (Lite version only) * Non-Profit organization, educational institutes may be participated as “Honorary” membership with limited right for FREE • Membership types
  15. 15. Copyright© MobileBench Consortium, 2013 Consortium Roadmap • First General Assembly is set for 26th of September 2013 at Shenzhen, China • Open for anyone for 1st General Assembly and launching event of MobileBench suit 2013 for FREE 2013 2014 2Q 3Q 4Q 1Q Jun. Consortium Development Sep. Mar. Milestone  2nd Consortium Meeting  Milestone setup for MobileBench 2014 Sep.Jun. Mar. MobileBench 2013 rev.1.0 1st MobileBench General Assembly Release of MobileBench suit 2013 1st BoD @ Beijing Release of beta MobileBench Suit
  16. 16. Copyright© MobileBench Consortium, 2013 MobileBench Consoritium Register in advance to secure your spot for 1st MobileBench Consoritum General Assembly for FREE September 26th 2013, Shenzhen, China (25th for Press & Board of Director meeting) Hotels and more information will be informed later
  17. 17. Copyright© MobileBench Consortium, 2013 Thank youThank you

×