Your SlideShare is downloading. ×
  • Like
2014 icws research bipinv2
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

2014 icws research bipinv2

  • 74 views
Published

 

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to like this
No Downloads

Views

Total Views
74
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
1
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Different sources
    Different formats of reviews
    Written in natural language, difficult for machine to understand

Transcript

  • 1. Quality of Experience: What End-users Say About Web Services? Bipin Upadhyaya
  • 2. Introduction • Web services provided by different vendors have different qualities • Quality of Service(QoS) refers to non- functional requirements, such as performance or availability requirements • QoS deals with how a service behave at run- time 2
  • 3. Service Selection Approaches QoS aware architecture filters the low quality web services from a set of functionally equivalent services 3
  • 4. Limitation of QoS • Static Release • Not frequently updated • Environment & platform specific • Runtime Monitoring • Evaluation of QoS at client-side • resource intensive, time consuming & expensive End users barely understand QoS attributes 4
  • 5. User Experience On average, users tell 15 people about good experiences and 24 people about bad experiences A word-of-mouth recommendation is the primary factor behind 20% to 50% of all purchasing decisions Ref: http://www.mckinsey.com/insights/marketing_sales/a_new_way_to_measure_word-of-mouth_marketing
  • 6. Reviews • Web 2.0 has enabled people to broadcast their experience to the mass • Online user reviews are examples of such phenomenon • Credible source of information 6
  • 7. Heterogeneity in Reviews 7
  • 8. Extract Relevant Information from Reviews • Extract service features such as • Parts and Properties • Opinions about the features are expressed as • Adjectives and Adverbs #Hotel Pool ElevatorFood Breakfast Staff Location Room 8
  • 9. Overview of Our Approach 9 POS Tagged Sentence Extract Sentence & Tag POS QoE Attributes & Opinions Extract QoE & Opinion Aggregated QoE Attributes Group Similar QoE Attributes
  • 10. POS Tag for Review Sentences Dropbox has great synchronization and folder sharing capability. Dropbox/NNP has/VBZ great/JJ synchronization/NN &/CC folder/NN sharing/NN capability/NN ./. POS Tagger 10
  • 11. Extract QoE Attributes and Opinion (great, synchronization) (great, Folder sharing capability) Dropbox/NNP has/VBZ great/JJ synchronization/NN &/CC folder/NN sharing/NN capability/NN ./. Detect negation and reverse adjectives Detect POS patterns 11
  • 12. Polarity of Opinion • SentiWordNet is used to assign a score of opinion • Each word in WordNet is assigned a score • Allocates a sentiment score of positive, negative or objective to words 12 (great, Synchronization) (great, Folder sharing capability) Great –> 0.75
  • 13. Cluster QoE Attributes Performance (6) Download (3) Install (2)Ease to Use(4) Operation (4) Sync (3) Synchronization(4) Back-up (5) Store (3) Fast (4) Upload (3) Download (3) Install (2)Ease to Use (4) Setup(2) Sync (3) Synchronization(4) Back-up (5) Store (3) Performance (6) Operation (4) Fast (4)Upload (3) Setup(2) Clustering 13
  • 14. Select Candidate Element • Rank each word in a cluster based on frequency and semantic similarity with other words • Candidate elements can be used by users to specify preferences Performance (6) Operation (4) Fast (4) Upload (3) QoE Attribute Rank Performance 7.04 Operation 5.168 Upload 3.66 Fast 5.982 14
  • 15. Research Questions RQ1. Can our approach extract QoE from online reviews? RQ2. Does user sentiment about QoE attributes changes over time? RQ3. How do QoE attributes relate with QoS attributes? 15
  • 16. Case Study Setup 448 440 422 419 6980 5866 6611 4110 0 1000 2000 3000 4000 5000 6000 7000 8000 Trip Shopping Storage Mapping Service Sentences containing QoE attributes and opinions Sentences without QoE attributes and opinions • Collect reviews for a periods of 13 months • Collect QoS attributes such as execution time and availability for all services 16
  • 17. RQ1. Can our approach extract QoE from online reviews? 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Travel Shopping Storage Mapping Service Recall Precision Extract QoE attributes with high precision and recall 17 90% 72%
  • 18. RQ2. Does user sentiment about QoE attributes change over time? 18 User’s sentiment about QoE attributes changes over time
  • 19. RQ3. How do QoE attributes relate with QoS attributes? High correlation between QoS and QoE Attributes indicates the possibility of using QoE attributes for service selection. 19
  • 20. 20