Оптимизация Lift Framework для работы с большими потоками данных Slava Schmidt История оптимизации одного Lift проекта для работы с базой данных Oracle для сохранения больших объемов данных. Информация о тонкостях интеграции с Akka и Squeryl, возможностях ускорения работы с базой данных и разработанном кэширующем jdbc драйвере.

1. TWEAKING
LIFT FRAMEWORK
!
AND
!
ORACLE DATABASE

2. Me
• Programmer since 1992
• Java programmer since 2004
• Scala consultant since 2007

3. Project
• Price Aggregator
• Completely stateless
• From “on demand” to “prefetch” strategy
• 3 Orders of magnitude more data
• Testing is for wimps

4. Project
• Scala 2.9.2
• Lift 2.4, Mapper ORM
• ExtJS 4
• Akka 2.1
• Squeryl 0.9.5
• Oracle RAC 10

5. Application Architecture
Akka Remote Partner Systems
ExtJS
Tomcat
Lift War
Akka
Oracle RAC

6. Problems

7. Database
• 1% Space, 90% Tables – Lookup Tables
• 99% Space, 10% Tables – Aggregated Data
DATA
A
B
C
D
E
F
G
G
I
J
K
l
m
N
O
P

8. Caching JDBC Driver
Application
Target DB
Driver
In-Memory
DB

9. Statements
• select from table(s)
• delete from table
• update table
• insert into table

10. DDL & Co.
• DDL
• Triggers
• Sequences
• Configuration
• Statistics

11. Distributed Setup
• Eventually consistent
• Akka-Cluster based
• Only changed data
• No additional queries

12. Eventually Consistent
Application
Target DB
Application
Driver
In-Memory
DB
Driver
In-Memory
DB

13. Read-Only Replica
Application
Target DB
Application
Driver Driver
In-Memory DB

14. TODOs
• Transaction support
• Query optimisation in distributed configuration
• Testing
• Backup Databases
• ORM frameworks
• Database functions / procedures

15. Where is it?
• Sorry, not on Github yet
• Patent pending
• Premium features for subscribers

16. Thank you
@slavaschmidt
slava.schmidt@dancingcode.com