Successfully reported this slideshow.
Using RESTful Webservices for Interoperability <ul><li>What we are
Why webservices matter to us
Some webservice examples
Some lessons learned </li></ul>
What is InterMine? <ul><li>FlyMine
ModMine [modEncode]
YeastMine [SGD]
RatMine [RGD]
ZfinMine [ZFIN] </li></ul><ul><li>TargetMine [NIBIO]
metabolicMine
?? [MGI]
?? [Wormbase]
? </li></ul>
Flybase Ensembl Biogrid RedFly NCBI Wormbase OMIM Reactome anoEST PDB ArrayExpress FlyAtlas
Storage and Organisation  ObjectStore DB
Storage and Organisation  Heavily denormalised – read-only Rebuilt for each release Read-write – persists across releases ...
Retrieval Internal API ObjectStore DB Lists SQL IQL PathQuery Template
Webapp Interface
Webapp Interface
The need to communicate <ul><li>MOD ↔ Mine
Mine ↔ Mine
User ↔ Mine </li></ul>
History of Webservices <ul><li>SOAP
PRO: enterprise, XML
Upcoming SlideShare
Loading in …5
×

B03-GenomeContent-Intermine

737 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

B03-GenomeContent-Intermine

  1. 1. Using RESTful Webservices for Interoperability <ul><li>What we are
  2. 2. Why webservices matter to us
  3. 3. Some webservice examples
  4. 4. Some lessons learned </li></ul>
  5. 5. What is InterMine? <ul><li>FlyMine
  6. 6. ModMine [modEncode]
  7. 7. YeastMine [SGD]
  8. 8. RatMine [RGD]
  9. 9. ZfinMine [ZFIN] </li></ul><ul><li>TargetMine [NIBIO]
  10. 10. metabolicMine
  11. 11. ?? [MGI]
  12. 12. ?? [Wormbase]
  13. 13. ? </li></ul>
  14. 14. Flybase Ensembl Biogrid RedFly NCBI Wormbase OMIM Reactome anoEST PDB ArrayExpress FlyAtlas
  15. 15. Storage and Organisation ObjectStore DB
  16. 16. Storage and Organisation Heavily denormalised – read-only Rebuilt for each release Read-write – persists across releases UserProfile DB ObjectStore DB Lists refers to
  17. 17. Retrieval Internal API ObjectStore DB Lists SQL IQL PathQuery Template
  18. 18. Webapp Interface
  19. 19. Webapp Interface
  20. 20. The need to communicate <ul><li>MOD ↔ Mine
  21. 21. Mine ↔ Mine
  22. 22. User ↔ Mine </li></ul>
  23. 23. History of Webservices <ul><li>SOAP
  24. 24. PRO: enterprise, XML
  25. 25. CON: enterprise, XML </li></ul>
  26. 26. History of Webservices <ul><li>REST + clients </li><ul><li>Java client library
  27. 27. Perl client
  28. 28. JavaScript client
  29. 29. Python client
  30. 30. Ruby client (soon...) </li></ul></ul>
  31. 31. History of Webservices <ul><li>XML
  32. 32. Flat files (CSV/TSV)
  33. 33. JSON
  34. 34. Biological formats (GFF3, Fasta, BED) </li></ul>
  35. 35. Benefits of REST <ul><li>Very low threshold
  36. 36. Rapid development
  37. 37. Easy to debug
  38. 38. Can do what you need
  39. 39. Where you need to do it (the browser) </li></ul>
  40. 40. Embeddable Table Widgets
  41. 41. For not much code IMBedding . loadTemplate ( { name : &quot;Gene_Identifiers&quot; , constraint1 : &quot;Gene.secondaryIdentifier&quot; , op1 : &quot;<&quot; , value1 : &quot;CG1046&quot; , }, &quot;#placeholder&quot; );
  42. 42. For not much code use Webservice::InterMine 'flymine'; my $query = Webservice::InterMine ->new_query(class => 'Gene'); $query->add_views(qw/symbol pathways.name/); $query->add_constraint('Gene.symbol', 'ONE OF', ['zen', 'h']); while (<$query>) { print @$_; }
  43. 43. List Management <ul><li>Other side of InterMine data management: </li><ul><li>Creation
  44. 44. Retrieval
  45. 45. Updates
  46. 46. Deletion </li></ul></ul>
  47. 47. Concise use Webservice::InterMine; <ul>my $service = Webservice::InterMine->get_service($url, $token); my $list = $service->new_list( <ul>name => 'A list', type => 'Gene', content => '/some/file/somewhere.txt', </ul>); $list->show; $list->append([“zen”, “bib”, “h”]); $list->delete; </ul>Has necessitated better authentication
  48. 48. Automated Workflows List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $list = $flymine->new_list(type => 'Gene', content => “some/file.txt”); my $query = $list->to_query; $query->add_views('homologues.homologue.symbol'); $query->add_constraint('homologues.homologue.organism.shortName', '=', 'R norvegicus'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $query->results(''hashrefs”)]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); my $rat_query = $ratmine->new_query(class => 'Gene'); $rat_query->add_views(qw/symbol primaryIdentifier proteins.proteinDomains.name/); $rat_query->add_constraint('symbol', 'ONE OF', $rat_symbols); $rat_query->show;
  49. 49. Automated Workflows with Sugar List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $fly_results = $flymine ->new_list(type => 'Gene', content => “some/file.txt”) ->select('homologues.homologue.symbol') ->where('homologues.homologue.organism.shortName', '=', 'R norvegicus') ->results('hashrefs'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $fly_results]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); $ratmine->new_query(class => 'Gene') ->select(qw/symbol proteins.proteinDomains.name/) ->where('symbol', 'ONE OF', $rat_symbols) ->show;
  50. 50. Consequences: <ul><li>MODs can better integrate services
  51. 51. Users can manage their workflows
  52. 52. Mines can interact
  53. 53. Integration with other software projects (Perl client is BioPerl aware) </li></ul>
  54. 54. Write your own front end! <ul><li>Already have begun alternative front-end projects for smaller groups
  55. 55. Nearly provides all the services the webapp does (see intermine.org/wiki/WebService) </li></ul>
  56. 56. Lessons learned <ul><li>JSON is the one true data format
  57. 57. But the more formats the better (within reason)
  58. 58. Fail loudly
  59. 59. REST doesn't mean you have to follow CRUD method mappings.
  60. 60. Every language has its quirks, especially with HTTP requests; simple is better.
  61. 61. Tokens for authentication </li></ul>
  62. 62. Acknowledgements: <ul><li>University of Cambridge
  63. 63. NIH
  64. 64. Wellcome Trust </li></ul>Gos Micklem (PI), Richard Smith, Julie Sullivan, Sergio Contrino, Daniela Butano, Radek Stepan, Fengyuan Hu, Adrian Carr, Mike Lyne, Alex Kalderimis

×