Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Case Study: SRM 2.0 - A next generation shared resource management system built on SpringSource dm Server

2,387 views

Published on

Published in: Technology
  • Be the first to comment

Case Study: SRM 2.0 - A next generation shared resource management system built on SpringSource dm Server

  1. 1. Case Study: SRM 2.0 - A next generation shared resource management system built on ! SpringSource dm Server Matt Stine - St. Jude Children’s Research Hospital
  2. 2. Agenda • Introduction to St. Jude • Introduction to the SRM Domain • SRM 1.x • The Crossroads • SRM 2.0 • Spring DM / OSGi Benefits • Demo • SRM 2.0 Data Bus • Spring DM / OSGi Benefits • Demo • Q&A SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  3. 3. St. Jude Mission To advance cures, and means of prevention, for pediatric catastrophic diseases through research and treatment. Consistent with the vision of our founder Danny Thomas, no child is denied treatment based on race, religion or a family’s ability to pay. 3 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  4. 4. About St. Jude • Founded by entertainer Danny Thomas and opened in 1962 • Supported primarily by funds from volunteer contributions raised by ALSAC, the fund-raising arm of St. Jude • 500 new patients each year – Must be referred by physician – Generally 18 years old or younger – Must have disease currently studied at St. Jude • 4,300 “active” patients • 58 inpatient beds • 60,000 outpatient visits per year • 500,000 appointments per year • Average patient: 7-9 appointments per day • 75 staff physicians and 3,300 employees 4 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  5. 5. Research and Treatment • Known for advances in treatment of childhood leukemia • Other catastrophic diseases researched & treated at St. Jude: – Genetic diseases such as sickle cell disease, osteogenesis imperfecta (brittle bone disease) – Infectious diseases such as tuberculosis and HIV/ AIDS – Childhood cancers including tumors of the bone, brain and soft tissue 5 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  6. 6. 6 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  7. 7. Research Landscape • 16 Academic Departments • 5 Academic Divisions • Cancer Center – First and only NCI-designated Comprehensive Cancer Center solely focused on pediatric cancer – 6 Programs: • Cancer Prevention and Control • Developmental Therapeutics for Solid Malignancies • Hematological Malignancies • Molecular Oncology • Neurobiology and Brain Tumor • Signal Transduction 7 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  8. 8. Shared Resources • Centralized facilities providing services and access to specialized equipment for research activities • Often centered around technology too expensive for individual research programs to maintain their own infrastructure • Examples: – High-throughput DNA Sequencing – Gene Expression Array Technology – Electron Microscopy • St. Jude currently has 41 shared resources 8 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  9. 9. Shared Resource Operational Issues 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  10. 10. Shared Resource Operational Issues 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  11. 11. Shared Resource Operational Issues Service Request 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  12. 12. Shared Resource Operational Issues Service Sample Request Check In 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  13. 13. Shared Resource Operational Issues Service Sample Request Check In Process1 ... Processn 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  14. 14. Shared Resource Operational Issues Service Sample Request Check In Process1 ... Processn Sample Check Out 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  15. 15. Shared Resource Operational Issues Service Sample Request Check In Process1 ... Processn Generate Sample Bill Check Out 9 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  16. 16. Shared Resource Management (SRM) 1.x • Web-based application providing a single, integrated portal for management of shared resource facility activities. • Currently supports 11 St. Jude facilities offering more than 20 distinct services. • Available under the GNU Lesser General Public License (LGPL) v3.0 at http://stjude- srm.sourceforge.net. 10 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  17. 17. SRM 1.x Development Process • Requirements for 5 laboratories • Extract common requirements • Build Core Domain Model/Database Schema • Build Core Services Platform • Build Web Portal • Build laboratory specific extensions to: – DB Schema – Domain Model – Services Platform • Build laboratory specific Web UI 11 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  18. 18. SRM 1.x Development Process • Requirements for 5 laboratories • Extract common requirements • Build Core Domain Model/Database Schema • Build Core Services Platform • Build Web Portal • Build laboratory specific extensions to: – DB Schema – Domain Model ITERATE – Services Platform • Build laboratory specific Web UI 11 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  19. 19. SRM 1.x Domain Model Order Sample Workflow Workflow Detail Test Sample Data Group 12 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  20. 20. SRM 1.x Domain Model Order Sample Sample Extension Workflow Workflow Detail Sample Test Data Test Sample Group Extension Data Group Extension 12 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  21. 21. SRM 1.x Technology Stack • Java 2 Standard Edition 1.4 • Java 2 Enterprise Edition 1.3 – Enterprise JavaBeans 2.0 • CMP Entity Beans • Stateless Session Beans – Servlet 2.3 – JSP 1.2 • Oracle 10g • JBoss AS 4.2 13 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  22. 22. SRM 1.x Services Platform SRM Architecture SpringOne 2GX 2009. All rights reserved. Do not distribute without permission. 14
  23. 23. SRM 1.x Issues 15 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  24. 24. SRM 1.x Issues • Explosive System Growth – Per deployed service: • 3 new tables • 3 new entity beans (remote, local interfaces, implementation class, modified descriptors, ...) • 1 new stateless session bean • 1 new business delegate • Multiple new servlets, JSP’s 15 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  25. 25. SRM 1.x Issues • Explosive System Growth – Per deployed service: • 3 new tables • 3 new entity beans (remote, local interfaces, implementation class, modified descriptors, ...) • 1 new stateless session bean • 1 new business delegate • Multiple new servlets, JSP’s • Monolithic Deployment – Tightly coupled services – Tightly coupled layers 15 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  26. 26. SRM 1.x Issues 16 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  27. 27. SRM 1.x Issues • Isolated Defects => “Distributed Suffering” 16 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  28. 28. SRM 1.x Issues • Isolated Defects => “Distributed Suffering” • Implementation inconsistent across services – Data integrity issues – Inconsistent technology stack – Knowledge transfer burden 16 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  29. 29. SRM 1.x Issues • Isolated Defects => “Distributed Suffering” • Implementation inconsistent across services – Data integrity issues – Inconsistent technology stack – Knowledge transfer burden • Modularity Impossible – Tried at the web layer – Dependency versioning made things worse! 16 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  30. 30. The Crossroads • Information Sciences FY ‘09 – ’13 Strategic Plan – Develop ordering and billing functionality for ~10 additional lab-based shared resources – Develop SRM LIMS for those shared resources requiring such infrastructure – Develop links to other databases on campus as required 17 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  31. 31. The Crossroads • SRM 1.x is at this point roughly five years old • Core technology stack (EJB 2.0) is about eight years old • Architecture/development model mandates database schema and source code changes for any new facilities/services • 3-6 month delivery time for new facilities/services with 2-3 developers engaged • Can we do better? 18 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  32. 32. SRM 2.0 Key Goals 19 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  33. 33. SRM 2.0 Key Goals • Limit System Growth – Extend primarily by configuration – Extend secondarily by new/modified code 19 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  34. 34. SRM 2.0 Key Goals • Limit System Growth – Extend primarily by configuration – Extend secondarily by new/modified code • True Modularity – Across System Components – Across Services – Isolate risk, complexity 19 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  35. 35. SRM 2.0 Key Goals • Limit System Growth – Extend primarily by configuration – Extend secondarily by new/modified code • True Modularity – Across System Components – Across Services – Isolate risk, complexity • Upgrade Technology Stack – Spring – Hibernate/iBATIS 19 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  36. 36. SRM 2.0 Architectural Concepts Entity-Attribute-Value (EAV) Data Model 20 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  37. 37. SRM 2.0 Architectural Concepts Entity-Attribute-Value (EAV) Data Model • Discrete set of tables used for core model extensions 20 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  38. 38. SRM 2.0 Architectural Concepts Entity-Attribute-Value (EAV) Data Model • Discrete set of tables used for core model extensions • Tables => rows in Entity table 20 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  39. 39. SRM 2.0 Architectural Concepts Entity-Attribute-Value (EAV) Data Model • Discrete set of tables used for core model extensions • Tables => rows in Entity table • Table Columns => rows in Attribute table 20 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  40. 40. SRM 2.0 Architectural Concepts Entity-Attribute-Value (EAV) Data Model • Discrete set of tables used for core model extensions • Tables => rows in Entity table • Table Columns => rows in Attribute table • Table Row-Column Values => rows in Value table(s) 20 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  41. 41. Conventional Data Model Source: http://jpodb.alik.ch/twiki/bin/view/Main/DataModel 21 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  42. 42. EAV Data Model Source: http:// jpodb.alik.ch/twiki/ bin/view/Main/ DataModel 22 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  43. 43. SRM 2.x Domain Model Order Sample Worklog Task Task Set 23 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  44. 44. SRM 2.x Domain Model Order Sample EAV Worklog Task Task Set 23 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  45. 45. SRM 2.0 Architectural Concepts Plug-in Architecture 24 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  46. 46. SRM 2.0 Architectural Concepts Plug-in Architecture • System framework supports core business features 24 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  47. 47. SRM 2.0 Architectural Concepts Plug-in Architecture • System framework supports core business features • Features are implemented via pluggable “Business Activity Sources” 24 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  48. 48. SRM 2.0 Architectural Concepts Plug-in Architecture • System framework supports core business features • Features are implemented via pluggable “Business Activity Sources” • Data, layout, behavior bound via configurable “Business Activities” 24 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  49. 49. Plug-in Architecture Fe at ur e 25 SRM Kernel SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  50. 50. Plug-in Architecture Bu ur So si ce Fe ne at ss ur e Ac SRM Kernel tiv 25 ity SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  51. 51. Plug-in Architecture Bu ur Bu So si Fe si ne at ne ss ce ur ss e Ac Ac 25 tiv SRM Kernel tiv ity ity SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  52. 52. SRM Kernel SRM Kernel SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  53. 53. SRM Kernel EAV DAO Core Dynamic EAV Hybrid Services Services Engine Data Core DAO Generic Source Functions Handlers Core Platform SRM Kernel Dynamic Platform SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  54. 54. SRM Kernel Business Business Activity Activity Source Source EAV DAO Core Dynamic EAV Hybrid Services Services Engine Data Core DAO Generic Source Functions Handlers Core Platform SRM Kernel Dynamic Platform SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  55. 55. SRM 2.0 Technology Stack • Java SE 6 • Spring Framework 2.5.6 • Freemarker 2.3.15 • Prototype 1.6.1 • Hibernate 3.2.6 • iBATIS 2.3.4 • SpringSource dm Server 1.0.2 • PostgresPlus Advanced Server 8.3 27 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  56. 56. DM/OSGi Benefits • Modularization • Isolation • Plugability 28 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  57. 57. Modularization 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  58. 58. Modularization • Clear separation of responsibility 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  59. 59. Modularization • Clear separation of responsibility • Strict enforcement of module boundaries 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  60. 60. Modularization • Clear separation of responsibility • Strict enforcement of module boundaries • “Intraapplication SOA” 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  61. 61. Modularization • Clear separation of responsibility • Strict enforcement of module boundaries • “Intraapplication SOA” • Enables DRY 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  62. 62. Modularization • Clear separation of responsibility • Strict enforcement of module boundaries • “Intraapplication SOA” • Enables DRY – Especially with Web Slices! 29 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  63. 63. Isolation 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  64. 64. Isolation • Architectural Risk 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  65. 65. Isolation • Architectural Risk • Change 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  66. 66. Isolation • Architectural Risk • Change • Technology 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  67. 67. Isolation • Architectural Risk • Change • Technology • Dependencies 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  68. 68. Isolation • Architectural Risk • Change • Technology • Dependencies • Developer Responsibility 30 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  69. 69. Plugability 31 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  70. 70. Plugability • Services 31 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  71. 71. Plugability • Services • Lifecycle 31 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  72. 72. SRM 2.0 Demo SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  73. 73. Shared Resource Operational Issues • Data Management/Retrieval • Reporting – Usage Statistics – Turnaround Time – Service Request History 33 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  74. 74. Data Management (SRM 1.x) SRM Pickup Location SRM App Server Clustered Mass Storage Research Archives 34 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  75. 75. Data Management (SRM 1.x) SRM Pickup Location SRM App Server Clustered Mass Storage Research Archives 34 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  76. 76. Data Management (SRM 1.x) SRM Pickup Location SRM App Server Clustered Mass Storage Research Archives 34 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  77. 77. Data Management (SRM 1.x) SRM Pickup Location 2004 Largest File Size: < 10 MB SRM App Server Clustered Mass Storage Research Archives 34 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  78. 78. Data Management (SRM 1.x) SRM Pickup Location 2004 Largest File Size: < 10 MB SRM App Server 2009 Largest File Size: Clustered Mass Storage Approaching 100 GB!!! Research Archives 34 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  79. 79. Data Management • Need a event-based, distributed solution • Need to provide Data as a Service (DaaS) – Downstream Analytics Platforms 35 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  80. 80. Reporting • EAV Schema – Data/relationships too abstract for report writers – Performance bottleneck • Data Mart – Must be near real-time – Proper data partitioning • DaaS – Participate in other data warehouses 36 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  81. 81. SRM Data Bus SRM 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  82. 82. SRM Data Bus Data Bus SRM Broker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  83. 83. SRM Data Bus Data Bus Worker Data Bus SRM Broker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  84. 84. SRM Data Bus Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  85. 85. SRM Data Bus Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  86. 86. SRM Data Bus External Warehouse Processor Pipeline Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  87. 87. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  88. 88. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart Data Bus Worker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  89. 89. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart Lab Y Data Archival Pipeline Data Bus Worker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  90. 90. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart Lab Y Data Archival Pipeline Research Archive Data Bus Worker 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  91. 91. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart Lab Y Data Archival Pipeline Research Archive Data Bus Worker Lab X Data Archival Pipeline 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  92. 92. SRM Data Bus External Warehouse Processor Pipeline External Warehouse Data Bus Worker Data Mart Processor Pipeline Data Bus SRM Broker SRM Data Mart Lab Y Data Archival Pipeline Research Archive Data Bus Worker Lab X Data Analytics Archival Platform Pipeline 37 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  93. 93. SRM Data Bus Technology Stack • Java SE 6 • Spring Framework 2.5.6 • Spring Integration 1.0.3 • Apache ActiveMQ 5.2 • SpringSource dm Server 1.0.2 • PostgresPlus Advanced Server 8.3 38 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  94. 94. DM/OSGi Benefits • Modularization • Isolation • Plugability 39 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  95. 95. SRM Data Bus Demo SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  96. 96. Acknowledgements • SRM 2 Team • QA – Rama Gundapaneni – Asmita Vaidya – Simon Hagstroem • Business Analysts – Roshan Shrestha – Ashish Pagare – Bhagavathy Krishna – Sundeep Shakya – Kiran Putcha • OPS – JP Davaleswarapu – Scott Malone – Dinesh Devasagayam – Bill Pappas • Management • SRM Data Bus Team – Charles Hurmiz – Swetha Mandava – Clayton Naeve – Raghuver Kontham • SpringSource – Wei Cai – Keith Donald – Yingliang Du – Oleg Zhurakousky – Mark Fisher 41 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  97. 97. Feedback • Please send me your feedback about the presentation/presenter: matt.stine@stjude.org Twitter: mstine 42 SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.
  98. 98. Q&A SpringOne 2GX 2009. All rights reserved. Do not distribute without permission.

×