The document discusses the evolution of programming and data handling, emphasizing the rise of end-user programming and the importance of flexible APIs and standard data models. It highlights the need for effective provenance documentation and trust in data sources, as well as the challenges and innovations associated with workflow systems for end-user development. Additionally, it touches on the significance of remixing as a form of combinatorial innovation and the role of community tools in enhancing programming practices.

1. Paul Groth | Vrije Universiteit Amsterdam | pgroth@few.vu.nl Image: http://www.flickr.com/photos/tomk32/2988993409 / All images are under a creative commons license

2. Image: http://www.flickr.com/photos/lyza/2487848260/sizes/l/

3. Image: http://www.flickr.com/photos/gigi_murru/2757085392/sizes/l

5. Set of component technologies that can be combined and recombined to create new innovations From Hal Varian http://people.ischool.berkeley.edu/~hal/ Chief Economist at Google 3. http://www.flickr.com/photos/oskay/1364146497/sizes/m/ 2. http://www.flickr.com/photos/cwalker71/1041784395/sizes/l / 1. http://www.flickr.com/photos/restlessglobetrotter/448362507/sizes/m/ 1 2 3

6. TCP/IP, XML, HTTP, Standard Libraries 1469 Web APIs http://www.programmableweb.com/

7. Image: http://www.flickr.com/photos/davestfu/2157396025/sizes/l/ Image: http://www.flickr.com/photos/danielleblue/170497153/sizes/o/

8. By 2012, 90M end user programmers in the US alone 13M would describe themselves as programmers 55M will use spreadsheets and databases [Scaffidi et al 06]

9. We have gone from dozens of markets of millions of users to millions of markets of dozens of users [Adams 08] The “long tail of programming” [Anderson 08]

10. 1. Records 2. Turntables and mixers 3. Recording equipment

11. Image: http://www.flickr.com/photos/melodramababs/2446537799/sizes/l/

12. Remixing ++ Common, flexible and usable APIs Standard data models Emergence of nice tools Convergence of the Web and the Semantic Web RDFa, OpenCalais, more “other structured data”

13. Remixing ++ Common, flexible and usable APIs Standard data models Emergence of nice tools Convergence of the Web and the Semantic Web RDFa, OpenCalais, more “other structured data” No more conversion components

14. Shared Techniques [WIKIAI’09 @IJCAI]

15. Open Task Repository

17. http://www.like.nu 57 endpoints Over 1 billion triples Billion Triple Challenge Dataset We made it available on Amazon EC2 See http://bit.ly/13FOWT Built on eRDF evolutionary algorithm for searching over triples Christophe Guéret and Stefan Schlobach

18. Image: http://www.flickr.com/photos/danielleblue/170497153/sizes/o/

19. Declaratively capture analysis steps and their dependencies Steps are represented by components Software programs (codes), Web services, … Workflow systems enable the creation, editing and management of workflows and their executions Wings/Pegasus, Taverna, Yahoo Pipes, VizTrails, Kepler, …

20. 7/10/09 SWF 2009

21. Visual programming not inherently superior Green, Nardi, Moher End-user development See Nardi, Repenning and Loannidou, Myers, Lieberman,... Workflows are end-user development environments

22. Title: BLASTP with simplified results returned Description: This workflow Performs a blastp search on protein sequence, extracts sequence id within the blast report and retrives the corresponding seuqences.[sic] ≅

23. - myexperiment.org - 2300 users - 750 workflows - 160 groups

25. [IUI’09] [AAAI SS 09] [SWF 2009]

26. High level templates Adapt to availability of components and data Use rich descriptions [e-science 09]

27. Data (triples) How were they produced? Which ones should I trust? Who’s responsible? From Chris Bizer From pipes.deri.org

28. Enterprises need to know where and how their data was produced. Uptime Compliance to regulations Quality assurance Performance improvements … . http://www.ifixit.com/Teardown/iPod-touch-3rd-Generation/1158/1

29. Image: http://www.flickr.com/photos/seidsvag/122718624/sizes/l/

30. Oxford English Dictionary: the fact of coming from some particular source or quarter; origin , derivation the history or pedigree of a work of art, manuscript, rare book, etc.; concretely, a record of the ultimate derivation and passage of an item through its various owners.

31. Computer representation of provenance Provenance is represented by documentation Provenance is a query answered by searching over documentation

32. Instrument & Collect Collate Query Use

33. Ensure high-quality characteristics Protocol for recording documentation Formalised as an abstract state machine Proofs to ensure these characteristics [IEEE TPDS Groth 08]

34. Common logical structure shared by all creating and querying actors Enables the autonomous, asynchronous production of documentation by different application components Open, extensible model XML + OWL serializations Tools can operate on it (e.g. visualisation, reasoning) [ACM Toit 08: Groth, Moreau, Miles]

35. [e-Science 08]

36. from esaw09

37. http://www.flickr.com/photos/newbirth/2834643961/

39. Reputation

40. http://www.flickr.com/photos/el_ramon/3804532661/

41. Content http://www.flickr.com/photos/ogcodes/2095054686/

42. Content Nice Letterhead

43. Content Nice Letterhead Official Seal

44. Content Nice Letterhead Official Seal A particular statement is present

45. Content Nice Letterhead Official Seal ≈ A particular statement is present

46. Works well in open systems Deals with unknown agents Make a decision without reputation information Deals with new agents No need for a system designer to analyse new entrants Deals with agents behaving unexpectedly

47. System for gaining experience about contracts ( provenance ) Algorithms for assessment of contract proposals based on prior experience

48. Use prior experience based on provenance to ascertain trust [esaw 09]

49. Use prior experience based on provenance to ascertain trust Trust of new workflow components

50. The Community http://www.flickr.com/photos/dunechaser/142079357/sizes/o/

51. Provenance Challenges Testing the interoperability of provenance systems 14 Teams – 3 Challenges http://twiki.ipaw.info/bin/view/Challenge/ Open Provenance Model Interoperability model for exchanging provenance http://twiki.ipaw.info/bin/view/OPM/ The W3C Provenance Incubator Group http://www.w3.org/2005/Incubator/prov/

52. Remixing = combinatorial innovation Make remixing easier 1. Data and Data Discovery 2. Component exposure and composition 3. Process capture and organization

53. Contact: pgroth@few.vu.nl Follow: http://twitter.com/pgroth Read: http://www.pgroth.com

54. Make syntactic errors hard Make syntactic errors impossible Use objects as language elements Make domain-oriented languages Meta-domain orientation Support incremental development Facilitate decomposable test units Multiple views and incremental disclosure Integrate with the web Encourage syntonicity Allow immersion Scaffold typical designs Community tools