Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Loading in …3
1 of 25

SeedMe Platform for Sharing Research Results



Download to read offline

Most scientific computation and analyses create important transient data and preliminary results. Quick and effective assessments of these data are necessary for efficient use of researchers time and computation resources, but this process is complicated when a large collaborating team is geographically dispersed and/or some team members do not have direct access to the computation resource and output data. Current methods for sharing and assessing transient data and preliminary results are cumbersome, labor intensive, and largely unsupported by useful tools and procedures. Each research team is forced to create their own ad hoc procedures to push results from system to system, and user to user, to guide the next step in their research. The SeedMe platform provides a web-based cyberinfrastructure to enable easy sharing and streaming of transient data and preliminary results directly from computing resources to a variety of platforms, from mobile devices to workstations, and make it possible to quickly and conveniently view and assess results and provide an essential yet missing component in computing infrastructure.

Related Books

Free with a 30 day trial from Scribd

See all

SeedMe Platform for Sharing Research Results

  1. 1. Your results from disk to device Stream Encode Explore and Disseminate My Experiments SeedMe name inspired by Seed : proliferate and grow
  2. 2. National Science Foundation This material is based upon work supported by the National Science Foundation under Grant No. OCI-1235505 Acknowledgements Kristen Levy, Mahidhar Tatineni, Michael Dwyer, Tom Hutton, & Doug Weimer, SDSC Apple Inc. : Provided test hardware/software on loan during conceptualization phase Team Amit Chourasia Mona Wong David Nadeau Andrew Ferbert Michael Norman
  3. 3. Presentation Overview • Situate context & define problem • Introduce SeedMe • Sample use cases • Sample end user interaction • Q & A
  4. 4. Computation Cycle Compute/Ingest Post Process Analyze Monitor ? Validate ? Abstract ?
  5. 5. Monitoring Validation Derived Products Abstraction
  6. 6. Direct Access Direct Web Sharing Not All Members Have Access ---- Need Exact Location + Permissions Inflexible Security Policy Derived Products (Preliminary Results) (Transient Content) Many customers
  7. 7. WebpageEmail Download + Send Scattered Results Can’t share larger content Manual Download + Upload Handle Privacy Significant time commitment Manual Cloud Drive Download + Upload Cannot describe content Manual Sharing Derived Products (Many customers)
  8. 8. Compute Anticipate React Waterworld (1995)
  9. 9. Accessibility Irony 1993 “NCSA Mosaic, or simply Mosaic, is the web browser credited with popularizing the World Wide Web.” Mosaic (Web browser), Wikipedia eb_browser) 1990 “The web was originally conceived and developed to meet the demand for automatic information- sharing between scientists in universities and institutes around the world.” The birth of the web. CERN Mission Dashboard at NASA – JPL Voyager space crafts have been sharing information since 1977 2007 Mobile devices can share content instantly 2014 Computation simulations cannot share information with us • No place to push information (no infrastructure) • Lack of easy tools
  10. 10. Pitfalls in sharing derived content Download Upload Download (Round Trip + 1) Video Encoding Complexity Missing Easy Automation How To Describe & Discuss Content Replication & Scalability
  11. 11. Why not use existing tools?
  12. 12. Your results from disk to device SeedMe offers you to • Seamlessly seed (upload) your research content • Share and access your content • Generate dashboards for your simulation or expt. • Describe your content (everything is not a file) • Discuss your work, right next to your content Features and Benefits Ubiquitous Access your results anywhere on any device Collaborate Discuss & keep your colleagues updated with latest results at one spot Near Real Time Track progress from your simulation or instrument Share and Reuse Share your research results and reusable content widely Sequence to Video Convert sequence of images to videos at desired frame rate Automate Easily integrate your scripts, workflow or app using SeedMe tools Security All your communications with SeedMe are encrypted Privacy Control access to your content with private, group or public privileges
  13. 13. SeedMe Collection Ticker (Text) Ephemeral Sequences (Image set) Videos (video) Files (Catch all) Meta Data (Title, Description, Tags, Key Value Pairs, Credits, License) Privacy + Collaborators (Public, Group, Private) + (emails) Plots (Image)
  14. 14. SeedMe: How it Works Create Collection SignUp 1 2 Create Collection Web Browser Command Line or REST (Automate scripts, workflow or app) SignUp 1 2 Set Metadata Create Collection Set Privacy & Sharing Upload Content Web Browser Command Line or REST (Automate scripts, workflow or app) SignUp 1 52 3 4 Set Metadata Create Collection Set Privacy & Sharing Upload Content Update as desired Web Browser Command Line or REST (Automate scripts, workflow or app) SignUp 1 52 3 4 Set Metadata Create Collection Set Privacy & Sharing Upload Content View or Download on Web Update as desired Web Browser Command Line or REST (Automate scripts, workflow or app) SignUp Phone or Tablet or Computer (Web Browser) 1 52 3 4
  15. 15. SeedMe User Interactions • Create Collection ( CLI, API, Web Browser) • Update Collection ( CLI, API, Web Browser) • View Collection (Web Browser only)
  16. 16. Hardware Software MySQL Drupal Apache Gearman Scheduler Web nodes Storage Video encoding nodes Responsive Web Output Desktops Phones Tablets Ingestion HTTPS Web Browser REST SeedMe Architecture Network Switch 20 gbe20 gbe
  17. 17. SeedMe Processing Ingestion (Atomic or Recurring) Instant Processing (Text, Images) Notification Responsive Content Access Batch Processing (Videos)
  18. 18. Use Case: Sequence to Video Sequences of images are generated by Visualizations Confocal scans Time lapse recording Requirements Frame rate High Quality Encoding Universal playback Automation seedme.cmd -t “Seq Collection” # Collection title -sp "sample/sequences/steam" # Seq Path -st "sequence title" # Seq Title -sd "Desc of sequence" # Seq Description -sr "5" # Seq frame Rate -se # Trigger Seq Encode Command line example, but you can upload content simply via Web Browser
  19. 19. Use Case: Create Dashboards Results from simulations, instruments, analysis Files Plots Sequences Videos Requirements Managed Sharing Universal access Automation seedme.cmd -t “Quick Start” # Collection title -privacy group # Privacy -email # Share -notify # Notify (not automatic) -pp "sample/plots/node.png" # Plot Path -sp "sample/sequences/steam" # Seq Path -vp "sample/videos/air.mp4" # Video Path -fp "sample/files/doc.pdf” # File Path Command line example, but you can upload content simply via Web Browser
  20. 20. Use Case: Monitor Computation state Progress % Progress parameters Last file created Requirements Sharing Universal access Automation seedme.cmd -t “Progress Tracking” # Create new Collection seedme.cmd -update 29643 # Update collection ID -tic "step 1" # Ticker Text -tic "step 2" # Ticker Text -fp "sample/files/doc.pdf” # File Path Command line example, but you can upload content simply via Web Browser
  21. 21. Use Case: Share & Reuse Disseminate Share and reuse content IPython notebook State files from vis tools R scripts Matlab scripts Applications Exploring : ‘Save to SeedMe’ Kepler Workflow VisIt software YT software Integrate SeedMe Python Module or write your own REST client
  22. 22. Demo: Web Browser Types of Collections: My Shared Public You may Add, Edit and Notify collaborators Add a new collection View a collection with monitoring info View a collection with variety of content
  23. 23. seedme.cmd -title "SeedMe Quick Start" seedme.cmd -update 29643 # Update -fp "sample/files/doc.pdf" # File Path -pp "sample/plots/node.png" # Plot Path -sp "sample/sequences/steam" # Seq Dir Path -vp "sample/videos/air.mp4" # Video Path seedme.cmd -update 29643 -privacy group -email -email seedme.cmd -update 29643 -notify Demo: Command Line Interaction Create a new collection Add image seq. + create video from it Setup sharing Notify Group
  24. 24. Getting Started Interaction • Web Browser • Standalone executable (command line) • Python module with API ( • Curl Information • Quick Start Guide • Use Cases • Documentation • Tips for C, Fortran codes • Blog
  25. 25. Signup / Share Contact Us amit Twitter @SeedmeOrg

Editor's Notes

  • As must our research knowledge and content
  • Each iteration in Computation cycle requires creation of succinct and consumable derived products
  • Derived products are created by a single user?
    But usually we have more than one person interested in assessing the derived products
  • Problems
  • Problems
  • What are pitfalls in sharing derived products currently
    Download/Upload: Manual operation (baby sit), Slow Network, Large Content
    Movie Encoding: Complex & Cumbersome
    Web Setup: Laborious and requires management
    Replication: For each instance and each group
    Scalable: from content perspective and ubiquity (desktop to smart phones)
    All this requires too much plumbing
  • Existing tools are not really suitable for HPC research oriented needs
  • New content in SeedMe is called Collection
    Metadata: All text, key/value pairs, tags, credits, expiration
    Ticker: ephemeral text stream like stocks or tweets. Example: simulation progress, simulation state, convergence
    Plots: Images, charts, documents
    Image sequences: set of images
  • ×