0
Pipelines: plumbing for the next web




Ian Forrester
backstage.bbc.co.uk
cubicgarden.com
Overview


    In depth discussion of the abstract concept of flow *
•



    Not discussing the merits of UNIX or XML pip...
Imagine. . .


    Processing data from your local machine with data
•

    on-line
    Chaining together services you use...
The Landscape
Hot on the Horizon


    REST APIs are dotted all over the landscape
•

    Web applications are opening up
•

    Useful ...
REST XML API Everywhere
REST XML API Everywhere
Opening Up the Silos
Opening Up the Silos
Web Services with Revenue Models
Ubiquitous Feeds
Widgets and Gadgets
Desktop Search
Semantic Desktop projects
Semantic Desktop projects
File System Meta-data
How People are Building Things


    Particls
•

    Automator
•

    Yahoo Pipes
•
Touchstone / Particls


        Particls: An alerts/updates and attention
•

        management platform
        Ingests m...
Particls
A Pipeline Application
XML Configurations
Outline Processing Markup Language (OPML)
Attention Profiling Markup Language (APML)
Particls Internals
Learning from Particls


        Particls Limitations:
•

          Main purpose is an alert and attention engine
    •

 ...
Automator


        A beautiful graphical user interface for Applescript
•



          “Accomplish all of your time-consu...
Apple Automator
Automator
Applescript
Automator Internals
Learning from Automator


        Automator Limitations
•

         Proprietary
    •

         Requires Applescript
    •...
Yahoo Pipes


    New beta service that enables users to create
•

    mash-ups with a graphical user interface
    Mash-u...
Y! Pipes
Y! Pipes




   s
Y! Pipes
Y! Pipes Internals
Learning from Y! Pipes


        Yahoo! Pipes Limitations
•

         Only has access to data and content on-line
    •

 ...
Fitting the Puzzle Together


    Combine:
•

    Underlying XML definitions of Particls
•

    Slimline and smooth look o...
flow *
So what is flow*?
flow *
flow *
The flow * System
Adding Widgets to the Mix
Separate the Desktop API
But. . .there is no flow *


    flow * currently does not exist
•

    flow * is not just an application
•

    flow * is...
My Current flow * Setup
More Configuration Ideas
More Configuration Ideas
More Configuration Ideas
Flow* is. . .


    The glue that allows you to break out of the silos
•

    on-line and off-line
    The ability to pipe...
Core User-generated Pipeline Principles


        Definable
•

         Core pipeline defined as XML
    •

        Graphi...
Three Levels
Three Levels of Abstraction
Three Levels of Users, One Purpose
XPROC Level
Pipe GUI Level
Task Level
Pipelines Options
The Current Landscape
Particls: An Attention Platform
Automator: Strictly for OSX Only
Y! Pipes: No Access to the Local System
Y! Pipes: Could Access Local System
Yahoo widgets authenticator
Securing the Desktop and Cloud
Y! Pipes: All the Right Elements
Microsoft just around the corner?
Pipelines
User-generated Pipelines


        Pipelines generation is only limited by our own
•

        creativity and the APIs avai...
Pipeline Example
Pipeline Example
Complex Pipeline Example
Complex Pipeline Example
Complex Pipeline Example
The Ecosystem of the API
No fully functional API?


    Forget it now... your dead
•
Described in XPROC (part 1)
<?xml version=quot;1.0quot; encoding=quot;UTF-8quot;?>
<p:pipeline name=quot;encoder examplequ...
Described in XPROC (part 2)

<p:choose name=quot;stoarage-optionsquot;>
<p:when test=quot;form:///storage.selector='amazon...
What happens next?
To do


    Build flow* using a Rich Internet Application
•

    language like XUL or Apollo
    Write an Authentication l...
Summary


        User generated pipelines are viable for:
•

         People wanting automation
    •

         Non-progr...
Final thought


    Mashing-up, remixing, sharing is part of our culture
•
Thanks for listening, questions?


        Ian Forrester - pipelines@cubicgarden.com
•

        http://www.cubicgarden.com...
Upcoming SlideShare
Loading in...5
×

plumbing for the next web

5,297

Published on

The next web will be about flow, this flow will be user generated pipelines through applications and services. Unlike before these Pipelines will be definable, non-proprietary and shareable by anyone

Published in: Economy & Finance, Technology
2 Comments
11 Likes
Statistics
Notes
  • Great demonstration about the require to innovate company models; tips on how to represent them succinctly; along with the need to make development initiatives actionable. Superb use of photographs along with easy to understand illustrative examples.
    Sharika
    http://financeadded.com http://traveltreble.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • fantastic! More hotwebcam girls in live without registration on http://BestCamX.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
5,297
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
260
Comments
2
Likes
11
Embeds 0
No embeds

No notes for slide

Transcript of "plumbing for the next web"

  1. 1. Pipelines: plumbing for the next web Ian Forrester backstage.bbc.co.uk cubicgarden.com
  2. 2. Overview In depth discussion of the abstract concept of flow * • Not discussing the merits of UNIX or XML pipelines • Assumes an understanding of a pipe and pipeline •
  3. 3. Imagine. . . Processing data from your local machine with data • on-line Chaining together services you use regularly • Automating the process so it ran at convenient • times Not requiring programming knowledge • A process that is open, extensible and sharable •
  4. 4. The Landscape
  5. 5. Hot on the Horizon REST APIs are dotted all over the landscape • Web applications are opening up • Useful web services with revenue models • RSS and ATOM feeds are becoming ubiquitous • Desktop search is standard in Vista and OSX • Widgets and Gadgets are becoming useful • New projects to organise desktop meta-data •
  6. 6. REST XML API Everywhere
  7. 7. REST XML API Everywhere
  8. 8. Opening Up the Silos
  9. 9. Opening Up the Silos
  10. 10. Web Services with Revenue Models
  11. 11. Ubiquitous Feeds
  12. 12. Widgets and Gadgets
  13. 13. Desktop Search
  14. 14. Semantic Desktop projects
  15. 15. Semantic Desktop projects
  16. 16. File System Meta-data
  17. 17. How People are Building Things Particls • Automator • Yahoo Pipes •
  18. 18. Touchstone / Particls Particls: An alerts/updates and attention • management platform Ingests many different input sources via input • adapters Works out importance of new information to the • user and displays a proportional alert Alerts take many forms based on urgency and relevancy • levels Alerts are controlled with output adapters. •
  19. 19. Particls
  20. 20. A Pipeline Application
  21. 21. XML Configurations
  22. 22. Outline Processing Markup Language (OPML)
  23. 23. Attention Profiling Markup Language (APML)
  24. 24. Particls Internals
  25. 25. Learning from Particls Particls Limitations: • Main purpose is an alert and attention engine • No standard underline definable language • Not sharable outside of the pipes application • Choice of inputs and outputs limited • No automation • Semi-proprietary • Particls Highlights • Breakthrough web application • GUI has all the right elements • Form level abstract is great • Cloning pipes is very smart • Pushing new attention markup standard •
  26. 26. Automator A beautiful graphical user interface for Applescript • “Accomplish all of your time-consuming, repetitive manual • tasks quickly, efficiently and effortlessly with Automator Workflows. It’s simple to create custom Workflows just by dragging items, pointing and clicking. Perform the task once using your Automator Workflow or many times — even share Workflows with friends.” Apple.com •
  27. 27. Apple Automator
  28. 28. Automator
  29. 29. Applescript
  30. 30. Automator Internals
  31. 31. Learning from Automator Automator Limitations • Proprietary • Requires Applescript • Underlying definable language non-XML based • Works better as automation software • Not natively web enabled • Automator Highlights • Break through desktop application • Powerful Gui • Form level abstract is great • Great automation • Sharing of scripts very simple • Choice of inputs and outputs extensible via code •
  32. 32. Yahoo Pipes New beta service that enables users to create • mash-ups with a graphical user interface Mash-ups can be saved so other people can edit • and re-use them Still geeky but easier than creating mash-ups by • coding “a milestone in the history of the Internetquot; - Tim O'Reilly •
  33. 33. Y! Pipes
  34. 34. Y! Pipes s
  35. 35. Y! Pipes
  36. 36. Y! Pipes Internals
  37. 37. Learning from Y! Pipes Yahoo! Pipes Limitations • Only has access to data and content on-line • No XSL transformations • No standard underlying definable language • Not sharable outside of the Y! Pipes application • Choice of inputs and outputs limited • No automation • Proprietary • Yahoo! Pipes Highlights • Breakthrough web application • GUI has all the right elements • Form level abstract is great • Cloning pipes is very smart •
  38. 38. Fitting the Puzzle Together Combine: • Underlying XML definitions of Particls • Slimline and smooth look of Particls • Form control of Automator Power and control of Automator • Visual pipeline GUI of Yahoo! Pipes • Automation of Automator • Sharing of Automator and Yahoo! Pipes •
  39. 39. flow *
  40. 40. So what is flow*?
  41. 41. flow *
  42. 42. flow *
  43. 43. The flow * System
  44. 44. Adding Widgets to the Mix
  45. 45. Separate the Desktop API
  46. 46. But. . .there is no flow * flow * currently does not exist • flow * is not just an application • flow * is not just a service • flow * is a combination of good practices served up • for the power users
  47. 47. My Current flow * Setup
  48. 48. More Configuration Ideas
  49. 49. More Configuration Ideas
  50. 50. More Configuration Ideas
  51. 51. Flow* is. . . The glue that allows you to break out of the silos • on-line and off-line The ability to pipe services together using their • APIs and feeds as though programming code or interacting with the site directly XPROC documents with namespaces elements (User-generated Pipelines) Three levels of abstraction in one file •
  52. 52. Core User-generated Pipeline Principles Definable • Core pipeline defined as XML • Graphical • Levels of abstraction from the XML • Standard • Using the new W3C standard XPROC • Shareable • As easy to share as a OPML file • Open • Can be extended and plugged into widget engines • Non-proprietary • No lock-in, uses standard web and XML technologies •
  53. 53. Three Levels
  54. 54. Three Levels of Abstraction
  55. 55. Three Levels of Users, One Purpose
  56. 56. XPROC Level
  57. 57. Pipe GUI Level
  58. 58. Task Level
  59. 59. Pipelines Options
  60. 60. The Current Landscape
  61. 61. Particls: An Attention Platform
  62. 62. Automator: Strictly for OSX Only
  63. 63. Y! Pipes: No Access to the Local System
  64. 64. Y! Pipes: Could Access Local System
  65. 65. Yahoo widgets authenticator
  66. 66. Securing the Desktop and Cloud
  67. 67. Y! Pipes: All the Right Elements
  68. 68. Microsoft just around the corner?
  69. 69. Pipelines
  70. 70. User-generated Pipelines Pipelines generation is only limited by our own • creativity and the APIs available XPROC feeds Flow* like Javascript feeds • Greasemonkey User-generated Pipelines could be... • XPROC+SVG+XFORMS • XPROC+XUL • XPROC+RDF+XFORMS • XPROC+FlexML • XPROC+XHTML •
  71. 71. Pipeline Example
  72. 72. Pipeline Example
  73. 73. Complex Pipeline Example
  74. 74. Complex Pipeline Example
  75. 75. Complex Pipeline Example
  76. 76. The Ecosystem of the API
  77. 77. No fully functional API? Forget it now... your dead •
  78. 78. Described in XPROC (part 1) <?xml version=quot;1.0quot; encoding=quot;UTF-8quot;?> <p:pipeline name=quot;encoder examplequot; xmlns:p=quot;http://www.w3.org/2007/03/xprocquot;> <p:input port=quot;videoquot; href=quot;file://///127.0.0.1/my videos/quot;/> <p:choose name=quot;free-or-payquot;> <p:when test=quot;form:///encoder.dropdown='yes'quot;> <p:output port=quot;http://www.amazon.com/ws/ec2quot;> <p:parameter name=quot;amazon/ec2imagequot; value=quot;video-encodingquot;/> <p:parameter name=quot;amazon/encodetoquot; value=quot;flvquot;/> <p:parameter name=quot;amazon/encodetoquot; value=quot;h.264quot;/> <p:parameter name=quot;amazon/encodetoquot; value=quot;mpeg4quot;/> <p:parameter name=quot;amazon/accountquot; value=quot;form:///amazon.accountquot;/> <p:parameter name=quot;amazon/passwordquot; value=quot;form:///amazon.passwordquot;/> <p:parameter name=quot;amazon/accesskeyquot; value=quot;form:///amazon.accesskeyquot;/> <p:parameter name=quot;amazon/piorityquot; value=quot;1quot;/> </p:output> </p:when> <p:when test=quot;form:///encoder.dropdown='no'quot;> <p:output port=quot;http://www.heywatch.com/quot;> <p:parameter name=quot;encodequot; value=quot;flvquot;/> <p:parameter name=quot;encodequot; value=quot;h.264quot;/> <p:parameter name=quot;encodequot; value=quot;mpeg4quot;/> <p:parameter name=quot;loginquot; value=quot;form:///accountquot;/> <p:parameter name=quot;passwordquot; value=quot;form:///passwordquot;/> </p:output> </p:when> <p:otherwise> <p:output port=quot;local://start.loocal.applicationquot; href=quot;file://///127.0.0.1/C/program files/mediacleaner/mediaclean.exequot;/> </p:otherwise> </p:choose>
  79. 79. Described in XPROC (part 2) <p:choose name=quot;stoarage-optionsquot;> <p:when test=quot;form:///storage.selector='amazon'quot;> <p:output port=quot;http://www.amazon.com/ws/s3quot;> <p:parameter name=quot;amazon/accountquot; value=quot;form:///amazon.accountquot;/> <p:parameter name=quot;amazon/passwordquot; value=quot;form:///amazon.passwordquot;/> <p:parameter name=quot;amazon/accesskeyquot; value=quot;form:///amazon.accesskeyquot;/> <p:parameter name=quot;amazon/bucketquot; value=quot;form:///amazon.bucketquot;/> </p:output> </p:when> <p:when test=quot;form:///stroage.selector='bliptv'quot;> <p:output port=quot;http://www.blip.tvquot;> <p:parameter name=quot;loginquot; value=quot;form:///bliptv.accountquot;/> <p:parameter name=quot;passwordquot; value=quot;form:///bliptv.passwordquot;/> </p:output> </p:when> <p:when test=quot;form:///stroage.selector='youtube'quot;> <p:output port=quot;http://www.youtube.comquot;> <p:parameter name=quot;loginquot; value=quot;form:///youtube.accountquot;/> <p:parameter name=quot;passwordquot; value=quot;form:///youtube.passwordquot;/> </p:output> </p:when> <p:otherwise> <p:output port=quot;resultquot; href=quot;ftp://user:pass@ftp.cubicgarden.com/myvideo/quot;/> </p:otherwise> </p:choose> </p:pipeline>
  80. 80. What happens next?
  81. 81. To do Build flow* using a Rich Internet Application • language like XUL or Apollo Write an Authentication layer for the APIs – Open • ID based broker system for APIs? Decide on one way to write User-generated • Pipelines which works for flow* Write schema for namespaced extensions into • user-generated pipelines (XPROC document) Investigate the semantic desktop projects • Explorer W3Cs draft Widget specification •
  82. 82. Summary User generated pipelines are viable for: • People wanting automation • Non-programmers • Power users who can't be bothered to load up their IDE • Flow* (although only in early development) is very • powerful All the elements are in place, it just needs a developer or • company with a some vision
  83. 83. Final thought Mashing-up, remixing, sharing is part of our culture •
  84. 84. Thanks for listening, questions? Ian Forrester - pipelines@cubicgarden.com • http://www.cubicgarden.com/blojsom/blog/pipelines/ • Flickr pictures : Creative Commons Attribution-NonCommercial-NoDerivs License • http://www.flickr.com/photos/andidfl/ • http://www.flickr.com/photos/_f1guy68_/ • http://www.flickr.com/photos/coolinbox/ • http://www.flickr.com/photos/belljar/ • http://www.flickr.com/photos/dgbalancesrocks/ • http://www.flickr.com/photos/whinger/ • http://www.flickr.com/photos/wasabicube/ • http://www.flickr.com/photos/jacy •
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×