B03-GenomeContent-Intermine

707 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
707
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • Unpack “widgets” terminology - statistical/graphical analysis of lists
  • More snippets – show on on the backend: - what requests are made to perform this.
  • More snippets – show on on the backend: - what requests are made to perform this.
  • B03-GenomeContent-Intermine

    1. 1. Using RESTful Webservices for Interoperability <ul><li>What we are
    2. 2. Why webservices matter to us
    3. 3. Some webservice examples
    4. 4. Some lessons learned </li></ul>
    5. 5. What is InterMine? <ul><li>FlyMine
    6. 6. ModMine [modEncode]
    7. 7. YeastMine [SGD]
    8. 8. RatMine [RGD]
    9. 9. ZfinMine [ZFIN] </li></ul><ul><li>TargetMine [NIBIO]
    10. 10. metabolicMine
    11. 11. ?? [MGI]
    12. 12. ?? [Wormbase]
    13. 13. ? </li></ul>
    14. 14. Flybase Ensembl Biogrid RedFly NCBI Wormbase OMIM Reactome anoEST PDB ArrayExpress FlyAtlas
    15. 15. Storage and Organisation ObjectStore DB
    16. 16. Storage and Organisation Heavily denormalised – read-only Rebuilt for each release Read-write – persists across releases UserProfile DB ObjectStore DB Lists refers to
    17. 17. Retrieval Internal API ObjectStore DB Lists SQL IQL PathQuery Template
    18. 18. Webapp Interface
    19. 19. Webapp Interface
    20. 20. The need to communicate <ul><li>MOD ↔ Mine
    21. 21. Mine ↔ Mine
    22. 22. User ↔ Mine </li></ul>
    23. 23. History of Webservices <ul><li>SOAP
    24. 24. PRO: enterprise, XML
    25. 25. CON: enterprise, XML </li></ul>
    26. 26. History of Webservices <ul><li>REST + clients </li><ul><li>Java client library
    27. 27. Perl client
    28. 28. JavaScript client
    29. 29. Python client
    30. 30. Ruby client (soon...) </li></ul></ul>
    31. 31. History of Webservices <ul><li>XML
    32. 32. Flat files (CSV/TSV)
    33. 33. JSON
    34. 34. Biological formats (GFF3, Fasta, BED) </li></ul>
    35. 35. Benefits of REST <ul><li>Very low threshold
    36. 36. Rapid development
    37. 37. Easy to debug
    38. 38. Can do what you need
    39. 39. Where you need to do it (the browser) </li></ul>
    40. 40. Embeddable Table Widgets
    41. 41. For not much code IMBedding . loadTemplate ( { name : &quot;Gene_Identifiers&quot; , constraint1 : &quot;Gene.secondaryIdentifier&quot; , op1 : &quot;<&quot; , value1 : &quot;CG1046&quot; , }, &quot;#placeholder&quot; );
    42. 42. For not much code use Webservice::InterMine 'flymine'; my $query = Webservice::InterMine ->new_query(class => 'Gene'); $query->add_views(qw/symbol pathways.name/); $query->add_constraint('Gene.symbol', 'ONE OF', ['zen', 'h']); while (<$query>) { print @$_; }
    43. 43. List Management <ul><li>Other side of InterMine data management: </li><ul><li>Creation
    44. 44. Retrieval
    45. 45. Updates
    46. 46. Deletion </li></ul></ul>
    47. 47. Concise use Webservice::InterMine; <ul>my $service = Webservice::InterMine->get_service($url, $token); my $list = $service->new_list( <ul>name => 'A list', type => 'Gene', content => '/some/file/somewhere.txt', </ul>); $list->show; $list->append([“zen”, “bib”, “h”]); $list->delete; </ul>Has necessitated better authentication
    48. 48. Automated Workflows List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $list = $flymine->new_list(type => 'Gene', content => “some/file.txt”); my $query = $list->to_query; $query->add_views('homologues.homologue.symbol'); $query->add_constraint('homologues.homologue.organism.shortName', '=', 'R norvegicus'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $query->results(''hashrefs”)]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); my $rat_query = $ratmine->new_query(class => 'Gene'); $rat_query->add_views(qw/symbol primaryIdentifier proteins.proteinDomains.name/); $rat_query->add_constraint('symbol', 'ONE OF', $rat_symbols); $rat_query->show;
    49. 49. Automated Workflows with Sugar List of Identifiers -> List of database records -> Homologues in X -> Their protein domains use InterMine::Webservice; my $flymine = Webservice::InterMine->get_service(' www.flymine.org/query ', 'TOKEN'); my $fly_results = $flymine ->new_list(type => 'Gene', content => “some/file.txt”) ->select('homologues.homologue.symbol') ->where('homologues.homologue.organism.shortName', '=', 'R norvegicus') ->results('hashrefs'); my $rat_symbols = [map {$_->{ homologues.homologue.symbol}} $fly_results]; my $ratmine = Webservice::InterMine->get_service('ratmine.mcw.edu/ratmine'); $ratmine->new_query(class => 'Gene') ->select(qw/symbol proteins.proteinDomains.name/) ->where('symbol', 'ONE OF', $rat_symbols) ->show;
    50. 50. Consequences: <ul><li>MODs can better integrate services
    51. 51. Users can manage their workflows
    52. 52. Mines can interact
    53. 53. Integration with other software projects (Perl client is BioPerl aware) </li></ul>
    54. 54. Write your own front end! <ul><li>Already have begun alternative front-end projects for smaller groups
    55. 55. Nearly provides all the services the webapp does (see intermine.org/wiki/WebService) </li></ul>
    56. 56. Lessons learned <ul><li>JSON is the one true data format
    57. 57. But the more formats the better (within reason)
    58. 58. Fail loudly
    59. 59. REST doesn't mean you have to follow CRUD method mappings.
    60. 60. Every language has its quirks, especially with HTTP requests; simple is better.
    61. 61. Tokens for authentication </li></ul>
    62. 62. Acknowledgements: <ul><li>University of Cambridge
    63. 63. NIH
    64. 64. Wellcome Trust </li></ul>Gos Micklem (PI), Richard Smith, Julie Sullivan, Sergio Contrino, Daniela Butano, Radek Stepan, Fengyuan Hu, Adrian Carr, Mike Lyne, Alex Kalderimis

    ×