Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CSSDP Presentation April 09


Published on

CSSDP Presentation from the NEP Synergy Workshop in April 09

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

CSSDP Presentation April 09

  1. 1. Canadian Space Science Data Portal Technical Architecture CANARIE Synergy Workshop April 21 – 22, 2009
  2. 2. Technical Architecture PROBLEM DOMAIN CSSDP - NEP 21 2
  3. 3. Space Weather CSSDP - NEP 21 3
  4. 4. CSSDP - NEP 21 4
  5. 5. Key Domain Characteristics Governance-related • Many independent Data Providers • Data usage restrictions and auditing Process-related • Research groups / teams • Resource search & discovery • Routine steps to prepare data for research Data-related • Large size and volume of data sets • Heterogeneity: instruments, data formats Technology-related • Heterogeneity: repository technologies, analytic / visualization technologies • Cost of technology vs funding models CSSDP - NEP 21 5
  6. 6. Heterogeneity CSSDP - NEP 21 6
  7. 7. Technical Architecture SOLUTION DOMAIN CSSDP - NEP 21 7
  8. 8. Virtual Observatory CSSDP - NEP 21 8
  9. 9. Portal CSSDP - NEP 21 9
  10. 10. Primary Design Elements CSSDP - NEP 21 10
  11. 11. Federating Resources Local Site Overall System Local Site Local Site Federated Local Site System Local Data Repository Federated Instrument System Instrument Data Site Data Data Copied to Portal Local Site Enterprise Service Bus Instrument Researcher Data Data Copied to Portal Instrument Portal Federated Data Site System Data Federated Local Site by Portal Instrument Federated Data Site System Local Site Local Analytics & Instrument Visualizations Data Federated System Overall System is a Federation of Each Local Site is a Federation Federations CSSDP - NEP 21 11
  12. 12. Block Diagram UI Presentation Forms Reports Charts Visualizations External Process Executon & Layer Page Flow Management Java Business Logic Analytics POJOs Business Business Business Components Rules Layer Events and Resource Notifications Management Scheduler Digital Rights Management Transport Resource Factory Registry Transaction Data Data Set Management Transport Layer Storage Persistence Data Set Management Transformation Common Services Collaboration Configuration Business Process & Instrumentation, Auditing & Workflow Logging Security & Identity Messaging & Web Services CSSDP - NEP 21 12
  13. 13. Workflow CSSDP - NEP 21 13
  14. 14. Pipes Filters CSSDP - NEP 21 14
  15. 15. Scientific Workflows D a ta L o a d e r G e n e ra te D a ta A cq u ire D a ta A va ila b ility E ve n t S ch e d u le r M o n ito r / In itia te M o n ito r R e trie ve N o tify E ve n t W o rkflo w s W o rkflo w s E ve n ts S u b scrib e rs fo r D a ta S e t W o rkflo w Run S to re R u n D a ta S to re Run S to re A n a lytics R e su lts M a rke rs R e su lts V isu a liza tio n s R e su lts D a ta M a rk e r A n an a ly D a ta A lyze S to re E ve n ts R e s e a rc h e r R e ce ive R e ce ive R e vie w E ve n ts D a ta S e t R e vie w P e rfo rm D a ta M a rke r and A va ila b ility D a ta S e ts A n a lysis E ve n t V isu a liza tio n s N o tifica tio n s N o tifica tio n s CSSDP - NEP 21 15
  16. 16. Technology CSSDP - NEP 21 16
  17. 17. Data Layer Layer Layer Business Presentation HTMLw/ Data Files HTTP JavaScript log4j commons-logging XML Workflow Engine (TBD) Registry Resource Message Bus or CSSDP - NEP 21 ESB (TBD) JEE 5 XSLT Spring Jasper Wicket Reports OGSA JDBC Spring Sun AppServer JFreeChart Quartz Scheduler Technologies REST / WS-* Web Services Confluence SQL Hibernate Postgres Data IDL, Matlab, FORTRAN, etc. Transport Unix Protocols LDAP Shell Scripts (SFTP, etc.) 17
  18. 18. SPASE XML CSSDP - NEP 21 18
  19. 19. Registration: JiBX and Java CSSDP - NEP 21 19
  20. 20. Wicket CSSDP - NEP 21 20
  21. 21. class Class Mo... «<Presentati... WebPage Sample Implementation «Presentatio... «Presentatio... «Presentatio... DataProv ider ResultsPage FindDataPage «BusinessLayer» «BusinessLa... DataChunkDataProv ider SearchCriteria «DataLayer» «<DataLayer... «Spring» «ServiceLay... HibernateSession RegistryDaoImpl BeanFactory RegistryServ ice «DataLayer» «DataLayer» «ServiceLayer» DataChunk RegistryDao RegistryServ iceImpl «DataLayer» PersistedObj ect 21
  22. 22. Some of the Lessons Learned (So Far) MySQL vs Postgres Human-readable vs automated URI formation Wicket vs Struts Hibernate query formation Patterns XML binding CSSDP - NEP 21 22
  23. 23. More Lessons Learned (So Far) Scrum and kanban (BVC) UI prototyping Unit vs integration testing Administration and operations External dependencies Make it work, make it work better CSSDP - NEP 21 23
  24. 24. Outstanding Questions Web service file transfer – MTOM? Redirection? REST vs WS-* (or both) – SPASE vs IVOA, OGSA Workflow engine – Scripting language? Taverna? ESB++? CSSDP - NEP 21 24