Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Step-By-Step Debugging Technique To Facilitate Mashup Development and Maintenance


Published on

by Waldemar Hummer, Philipp Leitner, Schahram Dustdar

  • Be the first to comment

  • Be the first to like this

A Step-By-Step Debugging Technique To Facilitate Mashup Development and Maintenance

  1. 1. A Step-By-Step Debugging TechniqueTo Facilitate Mashup Development and MaintenanceWaldemar Hummer, Philipp Leitner, Schahram Dustdar Distributed Systems Group, Vienna University of Technology 4th International Workshop on Web APIs and Services Mashups 8th IEEE European Conference on Web Services December 1, 2010
  2. 2. Outline Motivation Mashups in WS-Aggregation • Query Language • Declarative Mashup Definition Integrated Mashup Debugging • Goals • Techniques • Benefits Conclusion 2
  3. 3. Motivation Mashups combine data from heterogeneous Web sources Clear vision of the desired output Trade-off between abstraction and flexibility [10] Proposed Simplifications • Domain-Specific Languages (e.g., [7,15]) • Approaches for non-programmers (e.g., table-based [13,23]) In any case, arbitrarily querying/combining data is complex • Complexity of mashup programming can pose a barrier [24] • „The biggest problem of mashup is the data“ [14] • Further challenge: non-integrity of data, evolution of data sources Focus on development and debugging process • SW-Engineering techniques not well-supported for Mashups [10] • Important aspect: lack of support for interactive debugging [17] Demand for integrated mashup debugging facilities 3
  4. 4. WS-Aggregation Aggregation of heterogeneous Web services data Loose coupling • Data sources stored in registry [16] Scalable, distributed execution [11] • Configurable distribution strategies • Ad-hoc aggregator topologies Web services Aggregation Query Language (WAQL) • Based on XQuery • Integration of non-XML data sources (e.g., CSV, JSON, HTML, ...) • Expressing data dependencies • Possibility to generate sub-requests from templates Mashup Definition 1. Set of inputs for sub-requests {s1, s2, ..., sn} 2. WAQL Queries for transforming results 3. Data dependencies between sub-requests 4
  5. 5. Mashup Scenario Tourist information mashup • Input: city, date • Output: visa information, hotel rooms, wheather forecast 5
  6. 6. Mashup Scenario (2) Request Input (HTTP GET): /getVisaInfo?c=${//country} Data Source Response: ..<div id="visas">...</div>.. Preparation Query: <visaInfo>  {//div[@id=’visas’]/node()} </visaInfo> Prepared Data Source Result: <visaInfo>...</visaInfo>  Query TypesQuery Type Applied when? Applied to what?Preparation Query immediately data source responseIntermediate Query before passing data to parent aggregator all prepared DS resultsFinalization Query before returning data to client all prepared DS results 6
  7. 7. Data Dependencies Request Input (HTTP GET): /getVisaInfo?c=${//country} Data Dependency resolved at runtimeDataDependencyGraph Explicit Dependency: /getVisaInfo? c=$1{//country} 7
  8. 8. Generated Inputs Request Input (HTTP GET): /getRooms? h=$(${//hotelName/text()}) &d=${date}  Data Dependency: ${...} Generated Inputs: $(...) resolved at runtime 8
  9. 9. Mashup Debugging Debugging considered a core domain of SW-Engineering Different goals and purposes [25], e.g.: • Asserting expectations • Detecting anomalies • Tracking origins Ultimate Goal: Fixing a Defect • Defects in mashups: − Data source(s) unavailable − Data mistakenly dropped − Data falsely added − Data improperly transformed Different stages • During development • When underlying data sources change (→ maintenance) 9
  10. 10. Asserting expectations  Specify XQuery (or XPath) expressions that need to evaluate to true. Apply assertion Evaluate assertion to response or before or after total result? applying query? When should the assertion be evaluated?# Assertion Expression [E]nd / Data [R]esponse / [B]efore / Source ID [T]otal Result [A]fter1 count(//div[@id=’hotels’]/div)<=5 E T A2 //div[@id=’visaInfo’]/b/text() E T A3 //div[@id=’wheather’][a[1]/text()][a[2]/text()] E T A4 every $h in //hotelName/text() satisfies E T B //hotel[name=$h]/rooms5 string-length(//country/name)>0 1 R B6 every $r in //row satisfies count($r/col)>=3 4 R A 10
  11. 11. Detecting Anomalies  Different Types of Anomalies  Type implies Required Action(s)Anomaly Required ActionFault Response from DS Correct Request / Change EndpointPreprocessing Error Fix Invalid WAQL QueryUnresolvable Dependency Add Data SourceAmbiguous Dependency Refactor WAQL Preparation QueryCircular Dependency Revise Explicit DependenciesFailed Assertion (Before Q.) Correct Request / Change EndpointFailed Assertion (After Query) Correct WAQL Query 11
  12. 12. Tracking Origins Dependency Graph 12
  13. 13. GUI Integration 4 views: Design, Debug, Result (source), Preview (HTML) GUI indicates anomalies and possible origins Integrated support for top-down development 13
  14. 14. Conclusion Mashup development remains complex • Demand for enhanced testing and debugging techniques WS-Aggregation • Framework for mashup definition and distributed execution • Query language WAQL • Integration of non-XML data sources • Data dependencies, generated inputs Debugging aspects • Asserting expectations • Detecting anomalies • Tracking origins Benefits of explicit debugging support • Detects and locates anomalies • Enables top-down development • Facilitates maintenance 14
  15. 15. Discussion 15
  16. 16. Top-Down Development Define assertions first Successively add data source requests Similarity to Test-Driven-Development (TDD)# Defined DS Assertions Unresolved Requests Defined Violated Dependencies1 - 1,2,3,4 1,2,3 -2 3 1,2,3,4 2,3,4 -3 2,3 1,2,3,4 2,3,4 ${//country}4 1,2,3 1,2,3,4,5 3,4 -5 1,2,3,4 1,2,3,4,5,6 3 -6 1,2,3,4,6 1,2,3,4,5,6 3 ${//coords}7 1,2,3,4,5,6 1,2,3,4,5,6 - - 16
  17. 17. Scenario Details# WAQL DS Request DS Response (Ex) WAQL Prep. Query Prepared Result (Ex)1 <getCountry><city>${city} <country <country>{country/ <country>Austria </city></getCountry> name=“Austria“/> @name}</country> </country>2 /getVisaInfo?c=${//country} ..<div id="visa"> <visaInfo>{//div[@id=’visa’ <visaInfo>..</visaInfo> ...</div>.. ]/node()}</visaInfo>3 <getHotels><city>${city} ..<hotel><name> <hotelNames>{for $h in <hotelNames> </city></getHotels> Sacher</name> //hotel[position()<6] return <hotelName>Sacher <stars>5</stars>... <hotelName> </hotelName> </hotel>.. {$h/name/text()} ... </hotelName> </hotelNames> }</hotelNames>4 /getRooms? {"hotel":{"name":{" jsonToXML(/) <hotel><name>Sacher h=$(${//hotelName/text()}) $":"Sacher"}, "rooms": </name><rooms> &d=${date} {"room":[{"beds":{" <room><beds>2</beds> $":"2"}, "price":{" <price>350</price> $":"350"}},{..},..]}}} </room>..</rooms> </hotel>5 /cityCoords? Innsbruck,47N,11E let $c=csvToXML(/)// <coords><lat>48N</lat> c=${//country} Salzburg,47N,13E row[col[1]=’${city}’]/col <long>16E</long> Wien,48N,16E return <coords><lat>{$c[2] </coords> ... /text()}</lat><long>{$c[3] /text()}</long></coords>6 <getWheather>${//coords} <wheather date=".." - <wheather date=".." </getWheather> temperature="23" temperature="23" humidity="81%"/>.. humidity="81%"/>.. 17
  18. 18. References [7] F. Curbera, M. Duftler, R. Khalaf, and D. Lovell. Bite: Workflow composition for the web. In International Conference on Service-Oriented Computing, pages 94–106, Berlin, Heidelberg, 2007. Springer-Verlag. [10] L. Grammel and M.-A. Storey. An end user perspective on mashup makers. Technical Report DCS-324-IR, University of Victoria, 2008. [11] W. Hummer, P. Leitner, and S. Dustdar. WS-Aggregation: Distributed Aggregation of Web Services Data. In 26th Symposium On Applied Computing (SAC), March 21-25, 2011. To appear. [14] X. Liu, Y. Hui, W. Sun, and H. Liang. Towards Service Composition Based on Mashup. In IEEE Congress on Services, pages 332 –339, July 2007. [15] E. M. Maximilien, H. Wilkinson, N. Desai, and S. Tai. A Domain-Specific Language for Web APIs and Services Mashups. In Int. Conference on Service-Oriented Computing, pages 13–26. Springer, 2007. [16] A. Michlmayr, F. Rosenberg, P. Leitner, and S. Dustdar. End-to-End Support for QoS- Aware Service Selection, Binding and Mediation in VRESCo. IEEE Transactions on Services Computing, 3(3):193–205, 2010. [23] G. Wang, S. Yang, and Y. Han. Mashroom: end-user mashup programming using nested tables. In 18th International Conference on World Wide Web, pages 861–870, New York, USA. ACM, 2009. 18