MariaDB CONNECT Storage Engine
Upcoming SlideShare
Loading in...5
×
 

MariaDB CONNECT Storage Engine

on

  • 2,257 views

New CONNECT Storage Engine now in MariaDB 10.0.2

New CONNECT Storage Engine now in MariaDB 10.0.2

Statistics

Views

Total Views
2,257
Views on SlideShare
2,057
Embed Views
200

Actions

Likes
0
Downloads
19
Comments
0

3 Embeds 200

http://serge.frezefond.com 196
http://www.linkedin.com 3
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MariaDB CONNECT Storage Engine MariaDB CONNECT Storage Engine Presentation Transcript

  • SkySQLMariaDBCONNECT Storage EngineSerge FrezefondSkySQL Ab 2012 Confidential
  • SkySQL Ab 2012 ConfidentialGoal of the CONNECT Storage Engine :BI on various file formatsMost of the data in companies is in variousexternal datasources (many in non relationaldatabase format) :– Dbase, Firebird, SQlite– csv,– xml,– stored per column...– Microsoft Access & Excel– Distributed mysql serversTargeting BI data access on these formats.Not targeted for OLTP
  • SkySQL Ab 2012 ConfidentialBehind the sceneTraditional BIData is processed by an ETL– Change in the data model(denormalization...)Agregates are computed– Need to be defined and maintainedMight need to move data out of RDBMS toother kind of datastore– OLAP, Collumn store, Hadoop/Hbase ...Specific tools are used to query the dataIT is involved to maintain this machinery
  • SkySQL Ab 2012 ConfidentialMariaDB CONNECT Storage Engine :created by Olivier BertrandIBM database researcher– Now retired, 50 years expertise programingVery experienced on databases– Worked on system-R, DB2, natural languagequery ...Discovered MySQL when looking for friendlyplace to test new concepts.(2004)– Decided to go open source– Started to appreciate the MariaDB openessand friendlyness
  • SkySQL Ab 2012 ConfidentialHow did the CONNECT Storage Enginemove to MariaDB?Olivier met Monty creator of MySQL and MariaDB afew years ago (2004 for other concepts)SkySQL met Olivier and we start working together :– First access to launchpad, go to linux, test,Olivier start working with MariaDB team :– Testing, bug fixes, security, test cases ...SkySQL / MariaDB and Olivier agreed that is wasready to be released and supported under GPL– MariaDB flexibility ease integration
  • SkySQL Ab 2012 ConfidentialThe CONNECT Storage EngineUses the MySQL Plugin ArchitectureA major differentiator of MySQLAllow very specific datastore to interact withthe MySQL sql layerAllow advanced interaction– Specific Create Table parameters(MariaDB)– Condition push downAllow join with other storage engines– InnoDB / MyISAM tables
  • SkySQL Ab 2012 ConfidentialThe CONNECT Storage Engineimplements advanced features●Support of external data sources :–Odbc, MySQL, WMI ...●Support multi files tables●Support Big File Table > 2G●Support virtual tables (DIR)●Add autocreate of tables :–The structure is discovered from the datasource●Use MariaDB create table new parameterscapability (avoid comments polution)●Support compressed tables
  • SkySQL Ab 2012 ConfidentialThe CONNECT Storage Engineimplements advanced features●Add indexing to files– index optimized for read●Condition Push down– Used with ODBC and MySQL to push conditionto the target database. Big perf gain.●Support MariaDB virtuals columns●Support of special columns :– Rowid, fileid, tabid, servid●Muti tables table (like merge)– Different structure, not myisam only, remotelydistributed tables
  • SkySQL Ab 2012 ConfidentialCONNECT Storage Engine vs.Federated(X) storage Engine- support ODBC– With muti files ODBC- support access to MySQL table- support the limit clause
  • SkySQL Ab 2012 ConfidentialCONNECT Storage EngineODBC componentAllow to access to any datasource accessiblethrough ODBC.– Excel– Access– Firebird– SQLite– SQL Server, Oracle, DB2– ...Possibility to do multifiles ODBC– To query consolidated monthly excel datasheet
  • SkySQL Ab 2012 ConfidentialThe CONNECT Storage Engineimplements advanced featuresCatalog table :– For Example Describe for odbc table– No need to do create table– Access to data / column metadataMemory file maping– For file type table (not xml)Table format .ini
  • SkySQL Ab 2012 ConfidentialThe CONNECT Storage Engineimplements advanced featuresMultiple CONNECT tables can be created on thesame underlying file– Indexes can be shared between tables
  • SkySQL Ab 2012 ConfidentialConnect Storage EngineVEC table / Column storecol1col1col2col1col3col2col1row3row2row1col3col1freefreefreefreefreecol3free-1 or per column file- Indexes work- Fixed size record
  • SkySQL Ab 2012 ConfidentialConnect Storage EngineTable List Table (// Merge)col1col1 col2col1 col2col3col3col1 col2col3col1 col2ODBC tableMySQL tablecol4Muti tables table (like merge)– Different structure, not myisamonly, remotely distributed tables
  • SkySQL Ab 2012 ConfidentialConnect Storage Engine vs.MySQL Merge tablesTable list table :- support non MyISAM tables- no need to the exact same structure for table- underlying tables can be remote– Distributed architecture
  • SkySQL Ab 2012 ConfidentialImporting /exporting MySQL datain various formatsImporting file data into MySQL tables– Here for example from an XML file :• create table biblio select * from xsampall2;Exporting data from MySQL– Here for example we export to XML format :create table handout engine=CONNECT table_type=XMLfile_name=handout.htm header=yesoption_list=name=TABLE,coltype=HTML,attribute=border=1;cellpadding=5select plugin_name handler, plugin_version version, plugin_authorauthor, plugin_description description, plugin_maturity maturityfrom information_schema.plugins where plugin_type = STORAGE ENGINE;
  • SkySQL Ab 2012 ConfidentialWhere is the MariaDB Connect StorageEngine available ?●It is 100 % open source●It is available on MariaDB launchpad●Open Bug database●Public Roadmap●Released test cases●Improvement request / worklog
  • SkySQL Ab 2012 ConfidentialHow you can helpAdopt it / Test it.Bugs : report bugs / propose fixesDocumentation : help improve itSharing : test it, blog about it,– Share your experience about interestingusages.
  • SkySQL Ab 2012 ConfidentialConclusion●The MariaDB Connect Storage Engine :●Open MariaDB to BI and data analysis●Brings real value to MariaDB users●Illustrates openess of MariaDB community●Supported by SkySQL / MariaDB
  • SkySQL Ab 2012 Confidential20Thank YouSerge Frezefondwww.skysql.comlwww.facebook.com/skysqlwww.linkedin.com/company/skysql
  • SkySQL Ab 2012 ConfidentialEnhancements consideredfor MariaDB connect storage engine- parallel access to multitables table-