Canadian Space Science Data Portal
       Technical Architecture
        CANARIE Synergy Workshop
           April 21 – 22...
Technical Architecture

PROBLEM DOMAIN


                         CSSDP - NEP 21   2
Space Weather




    CSSDP - NEP 21   3
CSSDP - NEP 21   4
Key Domain Characteristics
Governance-related
• Many independent Data Providers
• Data usage restrictions and auditing
Pro...
Heterogeneity




    CSSDP - NEP 21   6
Technical Architecture

SOLUTION DOMAIN


                         CSSDP - NEP 21   7
Virtual
Observatory




              CSSDP - NEP 21   8
Portal




CSSDP - NEP 21            9
Primary Design Elements




         CSSDP - NEP 21   10
Federating Resources
       Local Site                                                                                    ...
Block Diagram
                             UI

Presentation   Forms     Reports       Charts         Visualizations




  ...
Workflow
CSSDP - NEP 21              13
Pipes                    Filters




        CSSDP - NEP 21             14
Scientific Workflows
    D a ta L o a d e r


                                                    G e n e ra te
          ...
Technology




CSSDP - NEP 21                16
Data
                                    Layer
                                                                           ...
SPASE XML




CSSDP - NEP 21               18
Registration: JiBX and Java




           CSSDP - NEP 21     19
Wicket



CSSDP - NEP 21            20
class Class Mo...


                                                   «<Presentati...
                                   ...
Some of the Lessons Learned (So Far)
MySQL vs Postgres

Human-readable vs automated URI formation

Wicket vs Struts

Hiber...
More Lessons Learned (So Far)
Scrum and kanban (BVC)

UI prototyping

Unit vs integration testing

Administration and oper...
Outstanding Questions
Web service file transfer
  – MTOM? Redirection?


REST vs WS-* (or both)
  – SPASE vs IVOA, OGSA


...
Upcoming SlideShare
Loading in...5
×

CSSDP Presentation April 09

159

Published on

CSSDP Presentation from the NEP Synergy Workshop in April 09

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
159
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CSSDP Presentation April 09

  1. 1. Canadian Space Science Data Portal Technical Architecture CANARIE Synergy Workshop April 21 – 22, 2009
  2. 2. Technical Architecture PROBLEM DOMAIN CSSDP - NEP 21 2
  3. 3. Space Weather CSSDP - NEP 21 3
  4. 4. CSSDP - NEP 21 4
  5. 5. Key Domain Characteristics Governance-related • Many independent Data Providers • Data usage restrictions and auditing Process-related • Research groups / teams • Resource search & discovery • Routine steps to prepare data for research Data-related • Large size and volume of data sets • Heterogeneity: instruments, data formats Technology-related • Heterogeneity: repository technologies, analytic / visualization technologies • Cost of technology vs funding models CSSDP - NEP 21 5
  6. 6. Heterogeneity CSSDP - NEP 21 6
  7. 7. Technical Architecture SOLUTION DOMAIN CSSDP - NEP 21 7
  8. 8. Virtual Observatory CSSDP - NEP 21 8
  9. 9. Portal CSSDP - NEP 21 9
  10. 10. Primary Design Elements CSSDP - NEP 21 10
  11. 11. Federating Resources Local Site Overall System Local Site Local Site Federated Local Site System Local Data Repository Federated Instrument System Instrument Data Site Data Data Copied to Portal Local Site Enterprise Service Bus Instrument Researcher Data Data Copied to Portal Instrument Portal Federated Data Site System Data Federated Local Site by Portal Instrument Federated Data Site System Local Site Local Analytics & Instrument Visualizations Data Federated System Overall System is a Federation of Each Local Site is a Federation Federations CSSDP - NEP 21 11
  12. 12. Block Diagram UI Presentation Forms Reports Charts Visualizations External Process Executon & Layer Page Flow Management Java Business Logic Analytics POJOs Business Business Business Components Rules Layer Events and Resource Notifications Management Scheduler Digital Rights Management Transport Resource Factory Registry Transaction Data Data Set Management Transport Layer Storage Persistence Data Set Management Transformation Common Services Collaboration Configuration Business Process & Instrumentation, Auditing & Workflow Logging Security & Identity Messaging & Web Services CSSDP - NEP 21 12
  13. 13. Workflow CSSDP - NEP 21 13
  14. 14. Pipes Filters CSSDP - NEP 21 14
  15. 15. Scientific Workflows D a ta L o a d e r G e n e ra te D a ta A cq u ire D a ta A va ila b ility E ve n t S ch e d u le r M o n ito r / In itia te M o n ito r R e trie ve N o tify E ve n t W o rkflo w s W o rkflo w s E ve n ts S u b scrib e rs fo r D a ta S e t W o rkflo w Run S to re R u n D a ta S to re Run S to re A n a lytics R e su lts M a rke rs R e su lts V isu a liza tio n s R e su lts D a ta M a rk e r A n an a ly D a ta A lyze S to re E ve n ts R e s e a rc h e r R e ce ive R e ce ive R e vie w E ve n ts D a ta S e t R e vie w P e rfo rm D a ta M a rke r and A va ila b ility D a ta S e ts A n a lysis E ve n t V isu a liza tio n s N o tifica tio n s N o tifica tio n s CSSDP - NEP 21 15
  16. 16. Technology CSSDP - NEP 21 16
  17. 17. Data Layer Layer Layer Business Presentation HTMLw/ Data Files HTTP JavaScript log4j commons-logging XML Workflow Engine (TBD) Registry Resource Message Bus or CSSDP - NEP 21 ESB (TBD) JEE 5 XSLT Spring Jasper Wicket Reports OGSA JDBC Spring Sun AppServer JFreeChart Quartz Scheduler Technologies REST / WS-* Web Services Confluence SQL Hibernate Postgres Data IDL, Matlab, FORTRAN, etc. Transport Unix Protocols LDAP Shell Scripts (SFTP, etc.) 17
  18. 18. SPASE XML CSSDP - NEP 21 18
  19. 19. Registration: JiBX and Java CSSDP - NEP 21 19
  20. 20. Wicket CSSDP - NEP 21 20
  21. 21. class Class Mo... «<Presentati... WebPage Sample Implementation «Presentatio... «Presentatio... «Presentatio... DataProv ider ResultsPage FindDataPage «BusinessLayer» «BusinessLa... DataChunkDataProv ider SearchCriteria «DataLayer» «<DataLayer... «Spring» «ServiceLay... HibernateSession RegistryDaoImpl BeanFactory RegistryServ ice «DataLayer» «DataLayer» «ServiceLayer» DataChunk RegistryDao RegistryServ iceImpl «DataLayer» PersistedObj ect 21
  22. 22. Some of the Lessons Learned (So Far) MySQL vs Postgres Human-readable vs automated URI formation Wicket vs Struts Hibernate query formation Patterns XML binding CSSDP - NEP 21 22
  23. 23. More Lessons Learned (So Far) Scrum and kanban (BVC) UI prototyping Unit vs integration testing Administration and operations External dependencies Make it work, make it work better CSSDP - NEP 21 23
  24. 24. Outstanding Questions Web service file transfer – MTOM? Redirection? REST vs WS-* (or both) – SPASE vs IVOA, OGSA Workflow engine – Scripting language? Taverna? ESB++? CSSDP - NEP 21 24
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×