Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
G04-Misc-Debianmed
G04-Misc-Debianmed
Loading in …3
×
1 of 24

B03-GenomeContent-Intermine

0

Share

Download to read offline

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

B03-GenomeContent-Intermine

  1. 1. Using RESTful Webservices for Interoperability <ul><li>What we are
  2. 2. Why webservices matter to us
  3. 3. Some webservice examples
  4. 4. Some lessons learned </li></ul>
  5. 5. What is InterMine? <ul><li>FlyMine
  6. 6. ModMine [modEncode]
  7. 7. YeastMine [SGD]
  8. 8. RatMine [RGD]
  9. 9. ZfinMine [ZFIN] </li></ul><ul><li>TargetMine [NIBIO]
  10. 10. metabolicMine
  11. 11. ?? [MGI]
  12. 12. ?? [Wormbase]
  13. 13. ? </li></ul>
  14. 14. Flybase Ensembl Biogrid RedFly NCBI Wormbase OMIM Reactome anoEST PDB ArrayExpress FlyAtlas
  15. 15. Storage and Organisation ObjectStore DB
  16. 16. Storage and Organisation Heavily denormalised – read-only Rebuilt for each release Read-write – persists across releases UserProfile DB ObjectStore DB Lists refers to
  17. 17. Retrieval Internal API ObjectStore DB Lists SQL IQL PathQuery Template
  18. 18. Webapp Interface
  19. 19. Webapp Interface
  20. 20. The need to communicate <ul><li>MOD ↔ Mine
  21. 21. Mine ↔ Mine
  22. 22. User ↔ Mine </li></ul>
  23. 23. History of Webservices <ul><li>SOAP
  24. 24. PRO: enterprise, XML
  25. 25. CON: enterprise, XML </li></ul>
  26. 26. History of Webservices <ul><li>REST + clients </li><ul><li>Java client library
  27. 27. Perl client
  28. 28. JavaScript client
  29. 29. Python client
  30. 30. Ruby client (soon...) </li></ul></ul>
  31. 31. History of Webservices <ul><li>XML
  32. 32. Flat files (CSV/TSV)
  33. 33. JSON
  34. 34. Biological formats (GFF3, Fasta, BED) </li></ul>
  35. 35. Benefits of REST <ul><li>Very low threshold
  36. 36. Rapid development
  37. 37. Easy to debug
  38. 38. Can do what you need
  39. 39. Where you need to do it (the browser) </li></ul>
  40. 40. Embeddable Table Widgets
  41. 41. For not much code IMBedding . loadTemplate ( { name : &quot;Gene_Identifiers&quot; , constraint1 : &quot;Gene.secondaryIdentifier&quot; , op1 : &quot;<&quot; , value1 : &quot;CG1046&quot; , }, &quot;#placeholder&quot; );
  42. 42. For not much code use Webservice::InterMine 'flymine'; my $query = Webservice::InterMine ->new_query(class => 'Gene'); $query->add_views(qw/symbol pathways.name/); $query->add_constraint('Gene.symbol', 'ONE OF', ['zen', 'h']); while (<$query>) { print @$_; }
  43. 43. List Management <ul><li>Other side of InterMine data management: </li><ul><li>Creation
  44. 44. Retrieval
  45. 45. Updates
  46. 46. Deletion </li></ul></ul>
  47. 47. Concise use Webservice::InterMine; <ul>my $service = Webservice::InterMine->get_service($url, $token); my $list = $service->new_list( <ul>name => 'A list', type => 'Gene', content => '/some/file/somewhere.txt', </ul>); $list->show; $list->append([“zen”, “bib”, “h”]); $list->delete; </ul>Has necessitated better authentication
  48. 48. Automated Workflows List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $list = $flymine->new_list(type => 'Gene', content => “some/file.txt”); my $query = $list->to_query; $query->add_views('homologues.homologue.symbol'); $query->add_constraint('homologues.homologue.organism.shortName', '=', 'R norvegicus'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $query->results(''hashrefs”)]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); my $rat_query = $ratmine->new_query(class => 'Gene'); $rat_query->add_views(qw/symbol primaryIdentifier proteins.proteinDomains.name/); $rat_query->add_constraint('symbol', 'ONE OF', $rat_symbols); $rat_query->show;
  49. 49. Automated Workflows with Sugar List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $fly_results = $flymine ->new_list(type => 'Gene', content => “some/file.txt”) ->select('homologues.homologue.symbol') ->where('homologues.homologue.organism.shortName', '=', 'R norvegicus') ->results('hashrefs'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $fly_results]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); $ratmine->new_query(class => 'Gene') ->select(qw/symbol proteins.proteinDomains.name/) ->where('symbol', 'ONE OF', $rat_symbols) ->show;
  50. 50. Consequences: <ul><li>MODs can better integrate services
  51. 51. Users can manage their workflows
  52. 52. Mines can interact
  53. 53. Integration with other software projects (Perl client is BioPerl aware) </li></ul>
  54. 54. Write your own front end! <ul><li>Already have begun alternative front-end projects for smaller groups
  55. 55. Nearly provides all the services the webapp does (see intermine.org/wiki/WebService) </li></ul>
  56. 56. Lessons learned <ul><li>JSON is the one true data format
  57. 57. But the more formats the better (within reason)
  58. 58. Fail loudly
  59. 59. REST doesn't mean you have to follow CRUD method mappings.
  60. 60. Every language has its quirks, especially with HTTP requests; simple is better.
  61. 61. Tokens for authentication </li></ul>
  62. 62. Acknowledgements: <ul><li>University of Cambridge
  63. 63. NIH
  64. 64. Wellcome Trust </li></ul>Gos Micklem (PI), Richard Smith, Julie Sullivan, Sergio Contrino, Daniela Butano, Radek Stepan, Fengyuan Hu, Adrian Carr, Mike Lyne, Alex Kalderimis

Editor's Notes

  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • More snippets – show on on the backend: - what requests are made to perform this.
  • More snippets – show on on the backend: - what requests are made to perform this.
  • ×