DBMS, Data, NoSQL, DBMS consolidation

1. ;
The Gap!
Pini Cohen
EVP
pini@stki.info
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

2. Agenda
• Major Trends and Issues
• Development and SOA
• ESM BSM CMDB
• DBMS and DATA
• Platforms – Servers
• Clients
• Storage
Source: http://astonguild.org.uk/files/NEW_MENU_FRONT_RGB%5B1%5D.jpg
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

3. Mini Agenda DBMS
• NoSQL
• DBMS appliances
• From RT’s – about DBA organization,
consolidation, etc.
• CDC tools
• DBA staffing ratios
• Ratings
Pini Cohen’s work Copyright 2011 @STKI 3
Do not remove source or attribution from any graphic or portion of graphic

4. The balance is changing
Cloud application
have different
needs
Infra is
better
Need for
more
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

5. Big Data
• Big Data is a term applied to data sets whose size is beyond the ability of
commonly used software tools to capture, manage, and process the data
within a tolerable elapsed time. Big data sizes are a constantly moving
target currently ranging from a few dozen terabytes to many petabytes of
data in a single data set.
• Examples include web logs, RFID, sensor networks, social networks,
Internet text and documents, Internet search indexing, call detail records,
genomics, astronomy, biological research, military surveillance, medical
records, photography archives, video archives, and large scale
eCommerce. (wikipedia)
Source: http://fortunewallstreet.files.wordpress.com/2010/12/matrix.jpg
Pini Cohen’s work Copyright 2011 @STKI 5
Do not remove source or attribution from any graphic or portion of graphic

6. What do we expect from DBMS?
• Availability
• Consistency
• Scalability
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

7. Brewer's (CAP) Theorem
• It is impossible for a distributed computer
system to simultaneously provide all three of
the following guarantees:
– Consistency (all nodes see the same data at the
same time)
– Availability (node failures do not prevent survivors
from continuing to operate)
– Partition Tolerance (the system continues to
operate despite arbitrary message loss)
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI http://en.wikipedia.org/wiki/CAP_theorem
Do not remove source or attribution from any graphic or portion of graphic

8. What It Means
Source: Scalebase
http://guyharrison.squarespace.com/blog/2010/6/13/consistency-models-in-non-relational-databases.html
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

9. Dealing With CAP
• Drop Partition Tolerance
– Run everything on one machine.
– This is, of course, not very scalable.
• Drop Availability
– If a partition fail, everything waits until the data is
consistent again.
– This can be very complex to handle over a large
number of nodes.
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

10. Dealing With CAP
• Drop Consistency
– Welcome to the “Eventually Consistent” term.
• At the end – everything will work out just fine - And hi,
sometimes this is a good enough solution
– When no updates occur for a long period of time,
eventually all updates will propagate through the
system and all the nodes will be consistent
– For a given accepted update and a given node,
eventually either the update reaches the node or the
node is removed from service
– Known as BASE (Basically Available, Soft state,
Eventual consistency), as opposed to ACID
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

11. NoSQL Types
• Key/Value
– A big hash table
– Examples: Voldemort, Amazon Dynamo
• Big Table
– Big table, column families
– Examples: Hbase, Cassandra
• Document based
– Collections of collections
– Examples: CouchDB, MongoDB
• Graph databases
– Based on graph theory
– Examples: Neo4J
• Each solves a different problem
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

12. NO-SQL
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI http://browsertoolkit.com/fault-tolerance.png
Do not remove source or attribution from any graphic or portion of graphic

13. Pros/Cons
• Pros:
– Performance
– BigData
– Most solutions are open source
– Data is replicated to nodes and is therefore fault-tolerant (partitioning)
– Don't require a schema
– Can scale up and down
• Cons:
– Code change
– No framework support
– Not ACID
– Eco system (BI, Backup)
– There is always a database at the backend
– Some API is just too simple
Source: Scalebase
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

14. DBMS market new frontiers:
Native Cloud Service
Hadoop
Cassandra
Database.com Xeround
Redis
Voldemort Amazon
Simple DB FathomDB
VoltDB MySQL,
Memcached Cluster Ed
Amazon RDS
NoSQL Cloud Enabled SQL
MySQL,
PostGress, Microsoft
SQL Azure
XMLDB Gemstone
Current
Object DB Clustrix IT
Microsoft
Oracle
SQL Server
DB2
Traditional
Pini Cohen’s work Copyright 2011 @STKI Source: http://xeround.com/
14
Do not remove source or attribution from any graphic or portion of graphic

15. Database.com
Pini Cohen’s work Copyright 2011 @STKI 15
Do not remove source or attribution from any graphic or portion of graphic

16. There are some NoSQL projects
out there…
Source: NoSQL Databases: Providing Extreme Scale and Flexibility By Matthew D. Sarrel
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

17. NoSQL Market Forecast 2011-2015
http://www.marketresearchmedia.com/2010/11/11/nosql-market/
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

18. Coming Soon - Oracle Database 11g on
Amazon Relational Database Service
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

19. Can we live with NoSQL
limitations?
• Facebook has dropped Cassandra
• “..we found Cassandra's eventual consistency
model to be a difficult pattern to reconcile for
our new Messages infrastructure”
• Facebook has selected HBase (Columnar
DBMS) .
http://www.facebook.com/notes/facebook-engineering/the-underlying-technology-of-messages/454991608919
Pini Cohen’s work Copyright 2011 @STKI 19
Do not remove source or attribution from any graphic or portion of graphic

20. Meanwhile, back on “Earth”, Growing
Market- special purpose DBMS machines
• Teradata
• Oracle Exadata
• EMC Greenplum
• HP
• Microsoft- Fast Track and Parallel Data
Warehouse
Pini Cohen’s work Copyright 2011 @STKI 20
Do not remove source or attribution from any graphic or portion of graphic

21. Information Management
The Netezza TwinFin™ Appliance
Slice of User Data
Disk Enclosures Swap and Mirror partitions
High speed data streaming
SQL Compiler
Query Plan
SMP Hosts
Optimize
Admin
S-Blades™
(with FPGA-based Processor &
Database Accelerator) streaming DB logic
High-performance database
engine streaming joins,
aggregations, sorts, etc.
21 © 2010 IBM Corporation

22. Information Management
Appliance family for data life-cycle management
Skimmer TwinFin Cruiser
Dev & Test System Data Warehouse Queryable Archiving
High Performance Analytics Back-up / DR
1 TB to 10 TB 1 TB to 1.5 PB 100 TB to 10 PB
22 © 2010 IBM Corporation

23. DBMS appliances
• First impression from initial testing of DBMS appliances:
• Importexport was not trivial. More effort then expected.
• Unprecedented performance boost. Examples (empty
machine):
– From 7-8 hours to 20 seconds
– From 3 hours to 1 hour
• With heavy loaded machine performance boost is lower
• Heavy IO load gets the most performance boost
Pini Cohen’s work Copyright 2011 @STKI 23
Do not remove source or attribution from any graphic or portion of graphic

24. DBA organization
• In Oracle centric organizations MSSQL will be in sub-group. In
such organizations MSSQL sub-team will have less resources that
might cause to SLA issues
• DBMS departments suffer constantly from shortage of personal
resource. In many case they will agree to support new project
with extra personal (since it Capex and not Opex ..) but will use
the help in “old” applications
• Oracle RAC is superior technology but require more effort and
skills and in some cases (bigger –complicated DBMS) might even
lead to downtime issues. Users are hoping that Exadata will help
with RAC stability.
Pini Cohen’s work Copyright 2011 @STKI 24
Do not remove source or attribution from any graphic or portion of graphic

25. From RT
• Users are deploying DBMS in virtual server environment in some
cases
• About half of application upgrades will cause some serious work
from the DBA part (performance, application issues, etc.)
• Users indicated the data compression tools and procedures (in
the DBMS level) has saved about 30% of space
• There is no “Black Box DBMS” in every productproject installed
in the organization the DBMS has to “put his hands on”
• Users believe that the market is before big changes – technologies
and players
Pini Cohen’s work Copyright 2011 @STKI 25
Do not remove source or attribution from any graphic or portion of graphic

26. Automation
• Many production error are caused by change
management issues so organizations have
development tools and procedures for change
management:
– Moving DBMS changes from one environment to the other
with labels that are related to the SCM tool
– What are the differences between two DBMS
– Automatic partition management tools
– Even RAC management tools
– DBA’s should not type directly to the DMBS/ They should
uses just scripts that where tested in all environments
Pini Cohen’s work Copyright 2011 @STKI 26
Do not remove source or attribution from any graphic or portion of graphic

27. DBMS consolidation projects
• Upgrade applicationDBMS to the same version and put
DBMS on one DBMS with several instances (up to 100
applications)
• Benefits:
– From no on do upgrades, patches on one DBMS
– Saving licenses
• Issues:
– All application should have the same upgrade cycle (from DBMS
perspective) and all should agree on down time slots
– Some technical issues (all application should have the same “init
parms”)
Pini Cohen’s work Copyright 2011 @STKI 27
Do not remove source or attribution from any graphic or portion of graphic

28. Local Trends
• Data Compression can save up to 30% and
even more.
• All DBMS in packagesapplications developed
in outsourcing needs intervention from the
local DBA team. “Black Box DBMS” model is
not working.
Pini Cohen’s work Copyright 2011 @STKI 28
Do not remove source or attribution from any graphic or portion of graphic

29. DBA organization
• DBA’s are divided in larger organizations to several groups:
– Production DBS
– Testing and Change Management
– Application – ADBA
– In larger organizations the production has sub-team for performance
• Example of DBA organization:
Non-Prod
20%
Prod
Source: STKI ADBA 55%
25%
Pini Cohen’s work Copyright 2011 @STKI 29
Do not remove source or attribution from any graphic or portion of graphic

30. Attunity – Microsoft OEM deal
Pini Cohen’s work Copyright 2011 @STKI 30
Do not remove source or attribution from any graphic or portion of graphic

31. Attunity CDC Suite for SSIS
an Operational Data Replication solution
Complete solution for real-time and efficient heterogeneous data
replication and integration, integrated with SQL Server.
1. Low impact on data server using log-based CDC technology
2. Efficiency and reduced resources, processing only incremental changes
3. Fully integrated with SSIS (2000, 2005) for ease of use
4. Monitoring and control console to manage replication processes
5. Capitalize on existing investments in SQL Server licenses
1. SQL Server
2. Oracle
3. DB2/400
4. DB2 on z/OS
5. NonStop SQL/MP
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

32. DBMS Support Ratios
• Number of developers (in the Open) supported by DBA FTE
Per FTE # of Applications New
25 percentile 11
Median 19
75 percentile 28
Source: STKI
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

33. Israel Market Positioning – DBMS Open –
General Usage
Oracle
Microsoft
Local Support
IBM
This analysis should be used
with its supporting documents
Israeli Market Presence
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

34. Market Status and Recommendations
• Users are using these integrators (support, maintenance) in DBMS
open area:
• Oracle
• Microsoft
• Veracity Many smallgood integrators in this area
• Matrix
• Valinor
• Priority
• IBM
• Inspire many other exist
Pini Cohen’s work Copyright 2011 @STKI
Do not remove source or attribution from any graphic or portion of graphic

35. STKI’s take on DBMS
• Automation Automation Automation
• Workflow, Self-service
• Focus on change management
• Standardization
• Watch for NoSQL development
• DBMS Consolidation
Pini Cohen’s work Copyright 2011 @STKI 35
Do not remove source or attribution from any graphic or portion of graphic

36. Thank you
Pini Cohen
Pini Cohen’s work Copyright 2011 @STKI 36
Do not remove source or attribution from any graphic or portion of graphic