The QS
How did we build the new query
system?
Development Team
Andrew Gilmartin, Chuck Koscher,
Evans Lin, Hisham Shahtout
1. Interesting parts of the QS
2. Experiences building the QS
3. Lessons learned for the DS
Tools
modern, but not too modern
NetBeans 6.[89]
technique sharing, tool building
Java 6
generics, annotations, oh my!
Tomcat 5.5
servlets are components too*
HAProxy 1.4
http://haproxy.1wt.eu/
Oracle && MySQL
valuable || expendable
ActiveMQ 5.4
wicked fast loose coupling
Spring 3.0
Really?
Spring || EJB || ???
reaction, refinement, replacement,
re-implement, rejoice
POJOs || Deployment
Descriptors
i just ( do || describe ) objects
Dependency Injection
|| Coded Configuration
more than one main()
Functional Interfaces ||
Featureful Interfaces
a little well done is enough
Spring Experiences
some tales are told a little later
Design
and not doing architecture
Controllers, Services
&& Coordination
insides and outsides
“Where is this article?”!
“Does this DOI have meta-data?”!
!"!!#$%&'!'&(%)*!"#$%&'!()*!++*!!
!+!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2,-./*0$0*)12#$%&'/)10%)33#%*6!
!7!!!!!!#80/8%...
Service Engine
actual heavy lifting
Management
x-rays, JMX
Health
…, lub-dub, lub-dub, …
8G$-:,!:'5%03&,%!J%&05$%&5KL%&'!M!
!!!!$//-%&'!:.L%&5:'1NOP!
Q!
<558R??,0"F2,0/..0%32/01?<%&05$%&5?>/:S,:5&5:/'S.%&0,<!
<558R??,0"F2,0/..0%32/01?<%&05$%&5?(%5&S>&5&S,:5&5:/'S.%&0,<!
!"!!#$%&'!!
!+!!!!!!'&(%)*!'#$%0;#$0!++*!!
!7!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2J%&05$%&5=/'50/--%0*6!
!@!!!!#80/8%0...
Artifacts
secondary (lossy) data flow(s)
8G$-:,!:'5%03&,%!V05:3&,5!%W5%'>.!;%0:&-:X&$-%!M!
Q!
8G$-:,!:'5%03&,%!V05:3&,5;%0H:,%!M!
!!!!8G$-:,!H/:>!,/'50:$G5%V05:3&,...
Interim Storage
bigger response, expiration
<558R??,0/..0%32/01?0%50:%H%?Y7VCU="LSL"@ES@DC+SE@"DSE+FBBA+77E7=!
!"!!8G$-:,!:'5%03&,%!;5/0%!M!
!+!
!7!!!!$//-%&'!,0%&5%N!
!@!!!!!!ZZ[U!77*(!
!A!!!!!!-/'1!>%8/.:5/0[>!
!B!!!!!!-/'1!0%,:8:%...
!"!!8G$-:,!:'5%03&,%![5%(!M!
!+!!!
!7!!!!8G$-:,!ZZ[U!1%5ZG:>NOP!
!@!!!
!A!!!!8G$-:,!_:(%.5&(8!1%5U%8/.:5%>_:(%.5&(8NOP!
!B...
Notifications
urgency, ( now || later ), aggregates
!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';%0H:,%!M!
!+!!!
!7!!!!!!H/:>!'/5:39Nb/5:3:,&5:/'!'/5:3:,&5:/'OP!
!@!!!
!A!!!!!!b/5:3:,&5...
!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/'!M!
!+!!!
!7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP!
!@!!!
!A!!!!!!$//-%&'!:.U%-:H%09Z01%'5NOP!
!B!!!...
!"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';5&5%!M!
!+!!!
!7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP!
!@!!!
!A!!!!!!$//-%&'!:.U%-:H%09^%'>:'1NOP!...
Spring Experiences
Coming up to speed
books are useful … for about 5 hours
Is that answer for
Spring 1.x || 2.x || 3.x?
rummaging around the net
Deployment
Descriptors
they’re back and now called contexts
Spring Beans
&& Java Properties
Where do I defined this thing?
XML
XSDs, id(s) || name(s), conditional
sections
Runtime Configuration
Errors
scripting without the support tools
Bondage
bound to Spring as much as you are
to any framework
It might be there
but not designed as expected
Development lessons
from the QS to apply
with the DS
Undervalue your own
ideas
the view from the backside of the
leading edge
Existing wheels
are generally
well designed
use code libraries … which are
sometimes your own
Use the API correctly
150/second to 2000/second
Stopping
buck-passing
artifacts, costs, choices
A does not follow B
right away
synchronization and the
ReadWriteLock are your friends
Pick a monitoring tool
as-soon-as-possible
instills design for management and
monitoring
The data is already
stale
design data for management and
convert data for use
Priming takes a long
time
pool XML parsers
Data conversion takes
a longer time
break it up into multiple parts
Spring Contexts need
support tools
we have one, we need more
Design small
interfaces
federation of cliques
Front-Controller vs
Servlets
I want my componentization back
Keep talking
especially with distant developers
:)
agilmartin@crossref.org
System Update 2010 CrossRef Workshops
System Update 2010 CrossRef Workshops
System Update 2010 CrossRef Workshops
System Update 2010 CrossRef Workshops
System Update 2010 CrossRef Workshops
System Update 2010 CrossRef Workshops
Upcoming SlideShare
Loading in...5
×

System Update 2010 CrossRef Workshops

718

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
718
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

System Update 2010 CrossRef Workshops

  1. 1. The QS How did we build the new query system?
  2. 2. Development Team Andrew Gilmartin, Chuck Koscher, Evans Lin, Hisham Shahtout
  3. 3. 1. Interesting parts of the QS 2. Experiences building the QS 3. Lessons learned for the DS
  4. 4. Tools modern, but not too modern
  5. 5. NetBeans 6.[89] technique sharing, tool building
  6. 6. Java 6 generics, annotations, oh my!
  7. 7. Tomcat 5.5 servlets are components too*
  8. 8. HAProxy 1.4 http://haproxy.1wt.eu/
  9. 9. Oracle && MySQL valuable || expendable
  10. 10. ActiveMQ 5.4 wicked fast loose coupling
  11. 11. Spring 3.0 Really?
  12. 12. Spring || EJB || ??? reaction, refinement, replacement, re-implement, rejoice
  13. 13. POJOs || Deployment Descriptors i just ( do || describe ) objects
  14. 14. Dependency Injection || Coded Configuration more than one main()
  15. 15. Functional Interfaces || Featureful Interfaces a little well done is enough
  16. 16. Spring Experiences some tales are told a little later
  17. 17. Design and not doing architecture
  18. 18. Controllers, Services && Coordination insides and outsides
  19. 19. “Where is this article?”! “Does this DOI have meta-data?”!
  20. 20. !"!!#$%&'!'&(%)*!"#$%&'!()*!++*!! !+!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2,-./*0$0*)12#$%&'/)10%)33#%*6! !7!!!!!!#80/8%059!'&(%)*,:5&5:/';%&0,<=//0>:'&5/0*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6! !@!!!!!!222! !A!!#?$%&'6! !B!!#$%&'!'&(%)*!"#%53#0!67#%8!++*!! !C!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.297#%8/)10%)33#%*6! !D!!!!!!#80/8%059!'&(%)*,:5&5:/';%&0,<=//0>:'&5/0*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6! !E!!!!!!222! "F!!#?$%&'6! ""!!#$%&'!'&(%)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*!! "+!!!!!!!!,-&..)*/012,0/..0%324.2,:5&5:/'.%&0,<2/*0$0*)12#$%&'/))%(*1$0)%*6! "7!!!!!!222! "@!!#?$%&'6! "A!!#$%&'!'&(%)*()*/*0$0*)197#%82#%5*&#*!! "B!!!!!!!!,-&..)*/012,0/..0%324.2>/:,:5&5:/'.%&0,<2,)*/*0$0*)197#%82#%5*&#*6! "C!!!!!!#80/8%059!'&(%)*4G%09;%0H:,%I%1:.509*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*?6! "D!!!!!!222!! "E!!#?$%&'6! +F!!#$%&'!'&(%)*4#0$,$0$/*0$0*)197#%82#%5*&#*!! +"!!!!!!!!,-&..)*/012,0/..0%324.2(%5&>&5&,:5&5:/'.%&0,<2:#0$,$0$/*0$0*)197#%82#%5*&#*6! ++!!!!!!#80/8%059!'&(%)*4G%09;%0H:,%I%1:.509*!0%3)*&)44)1/*0$0*)12#$%&'/))%(*1$0)%*!?6! +7!!!!!!222! +@!!#?$%&'6!
  21. 21. Service Engine actual heavy lifting
  22. 22. Management x-rays, JMX
  23. 23. Health …, lub-dub, lub-dub, …
  24. 24. 8G$-:,!:'5%03&,%!J%&05$%&5KL%&'!M! !!!!$//-%&'!:.L%&5:'1NOP! Q!
  25. 25. <558R??,0"F2,0/..0%32/01?<%&05$%&5?>/:S,:5&5:/'S.%&0,<! <558R??,0"F2,0/..0%32/01?<%&05$%&5?(%5&S>&5&S,:5&5:/'S.%&0,<!
  26. 26. !"!!#$%&'!! !+!!!!!!'&(%)*!'#$%0;#$0!++*!! !7!!!!!!,-&..)*/012,0/..0%324.2,/'50/--%0.2J%&05$%&5=/'50/--%0*6! !@!!!!#80/8%059!'&(%)*<%&05$%&5.*6! !A!!!!!!#(&86! !B!!!!!!!!#%'509!! !C!!!!!!!!!!!!T%9)*()*<&*0$0*)1<"#$%&'*!! !D!!!!!!!!!!!!H&-G%S0%3)*>/:=:5&5:/';%&0,<J%&-5<J%&05$%&5*!?6! !E!!!!!!!!#%'509!! "F!!!!!!!!!!!!T%9)*4#0$<($0$<&*0$0*)1<"#$%&'*!! ""!!!!!!!!!!!!H&-G%S0%3)*(%5&U&5&=:5&5:/';%&0,<J%&-5<J%&05$%&5*!?6! "+!!!!!!!!222! "7!!!!!!#?(&86! "@!!!!#?80/8%0596! "A!!#?$%&'6!
  27. 27. Artifacts secondary (lossy) data flow(s)
  28. 28. 8G$-:,!:'5%03&,%!V05:3&,5!%W5%'>.!;%0:&-:X&$-%!M! Q! 8G$-:,!:'5%03&,%!V05:3&,5;%0H:,%!M! !!!!8G$-:,!H/:>!,/'50:$G5%V05:3&,5N!V05:3&,5!&05:3&,5!OP! Q!
  29. 29. Interim Storage bigger response, expiration
  30. 30. <558R??,0/..0%32/01?0%50:%H%?Y7VCU="LSL"@ES@DC+SE@"DSE+FBBA+77E7=!
  31. 31. !"!!8G$-:,!:'5%03&,%!;5/0%!M! !+! !7!!!!$//-%&'!,0%&5%N! !@!!!!!!ZZ[U!77*(! !A!!!!!!-/'1!>%8/.:5/0[>! !B!!!!!!-/'1!0%,:8:%'5[>! !C!!!!!!]W8:0&5:/'^/-:,9!%W8:0&5:/'^/-:,9! !D!!!!!!;50:'1!,/'5%'5_98%! !E!!!!!!.1=7020%#$4>,/'5%'5['8G5;50%&(O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! "F! ""!!!!-70=7020%#$4>,0%&5%N! "+!!!!!!ZZ[U!77*(! "7!!!!!!-/'1!>%8/.:5/0[>! "@!!!!!!-/'1!0%,:8:%'5[>! "A!!!!!!]W8:0&5:/'^/-:,9!%W8:0&5:/'^/-:,9! "B!!!!!!;50:'1!,/'5%'5_98%O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! "C! "D!!!!a:.5#[5%(6!3:'>V--NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! "E! +F!!!!.0#4>3:'>L9ZG:>NZZ[U!GG:>O!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! +"! ++!!!!H/:>!8G01%]W8:0%>[5%(.=/'5%'5NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! +7! +@!!!!H/:>!,-/.%NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! +A!!Q!
  32. 32. !"!!8G$-:,!:'5%03&,%![5%(!M! !+!!! !7!!!!8G$-:,!ZZ[U!1%5ZG:>NOP! !@!!! !A!!!!8G$-:,!_:(%.5&(8!1%5U%8/.:5%>_:(%.5&(8NOP! !B! !C!!!!8G$-:,!-/'1!1%5U%8/.:5/0[>NOP! !D! !E!!!!8G$-:,!-/'1!1%5I%,:8:%'5[>NOP! "F! ""!!!!8G$-:,!;50:'1!1%5=/'5%'5_98%NOP! "+! "7!!!!8G$-:,!['8G5;50%&(!1%5=/'5%'5NO!5<0/`.!['5%0:(;5/0&1%]W,%85:/'P! "@! "A!!!!8G$-:,!]W8:0&5:/'^/-:,9!1%5]W8:0&5:/'^/-:,9NOP! "B! "C!!!!8G$-:,!$//-%&'!:.]W8:0%>NOP! "D! "E!!!!8G$-:,!_:(%.5&(8!1%5]W8:0%>_:(%.5&(8NOP! +"! ++!!Q!
  33. 33. Notifications urgency, ( now || later ), aggregates
  34. 34. !"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';%0H:,%!M! !+!!! !7!!!!!!H/:>!'/5:39Nb/5:3:,&5:/'!'/5:3:,&5:/'OP! !@!!! !A!!!!!!b/5:3:,&5:/'!1%5b/5:3:,&5:/'NZZ[U!50&,T:'1[>OP! !B!!! !C!!!!!!b/5:3:,&5:/';5&5%!1%5b/5:3:,&5:/';5&5%NZZ[U!50&,T:'1[>OP! !D!!! !E!!!!!!b/5:3:,&5:/'!0%(/H%b/5:3:,&5:/'NZZ[U!50&,T:'1[>OP! "F!!Q!
  35. 35. !"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/'!M! !+!!! !7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP! !@!!! !A!!!!!!$//-%&'!:.U%-:H%09Z01%'5NOP! !B!!! !C!!!!!![>%'5:59!1%5;%'>%0NOP! !D!!! !E!!!!!!;%5#[>%'5:596!1%5I%,:8:%'5.NOP! "F!!! ""!!!!!!$//-%&'!<&.U%-:H%09V110%1&5:/'b&(%NOP! "+!!!!!!;50:'1!1%5U%-:H%09V110%1&5:/'b&(%NOP! "7!!! "@!!!!!!$//-%&'!:.U%-:H%09V,T'/`-%>1%>NOP! "A!!! "B!!!!!!$//-%&'!:.U%-:H%09]W8:0&5:/'V,T'/`-%>1%>NOP! "C!!! "D!!!!!!;50:'1!1%5;G$c%,5NOP! "E!!! +F!!!!!!$//-%&'!<&.K%..&1%NOP! +"!!!!!!;50:'1!1%5K%..&1%=/'5%'5NOP! ++!!!!!!;50:'1!1%5K%..&1%=/'5%'5_98%NOP! +7!!! +@!!!!!!$//-%&'!<&.['5%0:(;5/0&1%ZZ[U.NOP! +A!!!!!!;%5#ZZ[U6!1%5['5%0:(;5/0&1%ZZ[U.NOP! +B!!! +C!!!!!!$//-%&'!<&.]W8:0&5:/'_:(%.5&(8NOP! +D!!!!!!_:(%.5&(8!1%5]W8:0&5:/'_:(%.5&(8NOP! +E!!Q!
  36. 36. !"!!8G$-:,!:'5%03&,%!b/5:3:,&5:/';5&5%!M! !+!!! !7!!!!!!ZZ[U!1%5_0&,T:'1[>NOP! !@!!! !A!!!!!!$//-%&'!:.U%-:H%09^%'>:'1NOP! !B!!! !C!!!!!!$//-%&'!:.U%-:H%0%>NOP! !D!!! !E!!!!!!$//-%&'!:.]W8:0%>NOP! "F!!! ""!!!!!!$//-%&'!:.I%(/H%>NOP! "+!!! "7!!!!!!_:(%.5&(8!1%5_:(%.5&(8NO!5<0/`.![--%1&-;5&5%]W,%85:/'P! "@!!! "A!!!!!!K&8#[>%'5:59!;%5#ZZ[U66!1%5U%-:H%09V110%1&5%NO!5<0/`.!222P! "B!!Q!
  37. 37. Spring Experiences
  38. 38. Coming up to speed books are useful … for about 5 hours
  39. 39. Is that answer for Spring 1.x || 2.x || 3.x? rummaging around the net
  40. 40. Deployment Descriptors they’re back and now called contexts
  41. 41. Spring Beans && Java Properties Where do I defined this thing?
  42. 42. XML XSDs, id(s) || name(s), conditional sections
  43. 43. Runtime Configuration Errors scripting without the support tools
  44. 44. Bondage bound to Spring as much as you are to any framework
  45. 45. It might be there but not designed as expected
  46. 46. Development lessons from the QS to apply with the DS
  47. 47. Undervalue your own ideas the view from the backside of the leading edge
  48. 48. Existing wheels are generally well designed use code libraries … which are sometimes your own
  49. 49. Use the API correctly 150/second to 2000/second
  50. 50. Stopping buck-passing artifacts, costs, choices
  51. 51. A does not follow B right away synchronization and the ReadWriteLock are your friends
  52. 52. Pick a monitoring tool as-soon-as-possible instills design for management and monitoring
  53. 53. The data is already stale design data for management and convert data for use
  54. 54. Priming takes a long time pool XML parsers
  55. 55. Data conversion takes a longer time break it up into multiple parts
  56. 56. Spring Contexts need support tools we have one, we need more
  57. 57. Design small interfaces federation of cliques
  58. 58. Front-Controller vs Servlets I want my componentization back
  59. 59. Keep talking especially with distant developers
  60. 60. :) agilmartin@crossref.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×