• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
SQLite Dynamic Data in Titanium
 

SQLite Dynamic Data in Titanium

on

  • 5,047 views

Updated with the session presented @tiConf - Valencia 2013 ...

Updated with the session presented @tiConf - Valencia 2013

London Titanium Meetup November 2011

- Speedup and save resources of your Titanium application, avoid reaching the API limit, never take the network for granted or simply work offline.

- Why SQLite fits better than storing different files.

Statistics

Views

Total Views
5,047
Views on SlideShare
2,789
Embed Views
2,258

Actions

Likes
4
Downloads
0
Comments
0

20 Embeds 2,258

http://www.scoop.it 1744
http://192.168.1.66 166
http://localhost 132
http://www.joemaffia.net 92
http://joemaffia.posterous.com 48
http://176.34.238.100 13
http://efdevs.posterous.com 11
http://londontitanium.co.uk 10
http://joemaffia.blogspot.com 9
http://londontitanium.posterous.com 6
http://ec2-46-137-3-203.eu-west-1.compute.amazonaws.com 5
http://ec2-176-34-210-154.eu-west-1.compute.amazonaws.com 4
http://posterous.com 4
http://lanyrd.com 4
http://176.34.237.35 3
http://a0.twimg.com 2
http://webcache.googleusercontent.com 2
https://hootsuite.scoop.it 1
http://translate.googleusercontent.com 1
http://ec2-79-125-73-34.eu-west-1.compute.amazonaws.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

SQLite Dynamic Data in Titanium SQLite Dynamic Data in Titanium Presentation Transcript

  • Yes! my real surname isMaffia!Thursday, 21 February 13
  • SQLiteDynamic Data in TitaniumtiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Joe MaffiaWeb, Mobile and Application Developer,Applemaniac, DJ - Producer, Sound engineer, musiclover and a general full-time geek!http://about.me/joemaffia“amazing what Photoshop can do...”tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {SQLite}SQLite is an in-process library that implements a self-contained, serverless, zero-configuration, transactional SQLdatabase engine...contained in a small (~350 KB) Cprogramming library.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Self-ContainedIt requires very minimal support from external libraries orfrom the operating system.This makes it well suited for usein embedded devices that lack the support infrastructure ofa desktop computer.mobile?tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • ServerlessMost SQL database engines are implemented as a separateserver process.With SQLite, the process that wants to access the databasereads and writes directly from the database files on disk.There is no intermediary server process.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • No SetupZero Configuration• No “setup” procedure.• No “installation”.• No “server process” to start,stop or configure.• No “administrator”.• No “configuration files”.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • TransactionalAll changes within a single transaction in SQLite eitheroccur completely or not at all.Program crash, operating system crash ora power failure.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Feature or Bug?It looks like a duck, it acts like a duck, it walks like aduck - it must be a duck.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Duck TypingtiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • CREATE TABLE cartoon(name TEXT,height INTEGER,....)INSERT INTO cartoon (name, height,...)VALUES (“donald”,”18cm”,...)If the string doesnt look like an integer -- itwont convert it. It will store it as a string.But it tries to convert it to integer if it can.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Limits?in this context means sizes or quantities that can not be exceeded.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Limits?in this context means sizes or quantities that can not be exceeded.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Limits in SQLiteSQLite was originally designed with a policy of avoidingarbitrary limits.The policy was that if it would fit in memory and you couldcount it with a 32-bit integer, then it should work.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Unfortunately :(Because the upper bounds were not well defined, theywere not tested, and bugs (including possible securityexploits) were often found when pushing SQLite toextremes.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • “Limits”• 1 billion Maximum length of a string or BLOB• 2000 Maximum Number Of Columns• 64 Maximum Number OfTables In A Joinhttp://sqlite.org/limits.htmltiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Ti.DatabasetiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Create vs Install• Create an empty database and define itsstructure and contents via SQL statementsin Ti.• Install a predefined database (with orwithout data) that is shipped with your app.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • CreatetiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • *iCloudiOS 5.0.1+the database will be included in any other user data backedup to iCloud.db.file.setRemoteBackup(false);tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • InstalltiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Let’s use it!tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • WritetiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • ReadtiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Tips & Tricks• Close db connection.• Transaction.• Too large pre-populated db.• Version NumbertiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • db.close();In a mobile app, youre dealing with a single-user, memoryconstrained environment...It’s vital that you close the database connection when youhave completed any INSERT or UPDATE operations.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • SQLite faq #19Use transactions to speed up inserts!By default, each INSERT statement is its own transaction.But if you surround multiple INSERT statements withBEGIN...COMMIT then all the inserts are grouped into asingle transaction.You create a single, mass operation against the databaserather than x little operations.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Big db = Big appShip a "skeleton" database file instead with the minimumamount of data required for the application to run.Then, on first boot, ask the users authorization todownload a replacement/update from a remote source.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • version 1.007αβγ• Is the user really running the old version ordid a previous update fail?• Are they downloading a new app version orupgrading an existing version?• At the end of the day... is the data correct/updated/wrong?Include a "version" table in your database!tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Tips n PlacestiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Problem{APIcall}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • ProblemUsage of the API is subject to rate limits.The default limit is 500 requests per hourTwitter is taking the default limit from 350 and cutting it to 175Unauthenticated calls are permitted 150 requests per hour.Google warns that rate limits, overage fees are coming to Maps APIwill be capped at 25,000 map loads per day{APIcall}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?• I don’t care!?! :(tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?• I don’t care!?! :(• Once reached the limit, the user will paythe consequences... :/tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?• I don’t care!?! :(• Once reached the limit, the user will paythe consequences... :/• Limit the API calls :| (timeframe or similar)tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?• I don’t care!?! :(• Once reached the limit, the user will paythe consequences... :/• Limit the API calls :| (timeframe or similar)• Saving results into files... but how many?how to manage them?tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Solution?tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Let me guess...............ah yes....SQLitetiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {code}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Thursday, 21 February 13
  • tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {initCache}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {expireCache}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {get}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {put}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {del}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • {public}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • DemoThursday, 21 February 13
  • Where do you want to go today?{next}tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • ...not here :)tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • So, how do I query thedatabase?It’s not a database. It’sa key-value store!Meet BobtiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Ok, It’s not a database.How do I query it?You write a distributedmap reduce function inErlang!tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • Did you just tell me tofuck myself?tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • I believe I did, Bob!tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • NoSQL on TitaniumWikipediaIn computing, NoSQL (sometimes expanded to "not only SQL") is a broad class of database managementsystems that differ from the classic model of the relational database management system (RDBMS) in somesignificant ways.These data stores may not require fixed table schemas, usually avoid join operations, andtypically scale horizontally.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • SQLite is great...but:db.execute(INSERT INTO favorites (api_id,tip_avatar, tip_user, tip_text, venue_icon,venue_name, venue_meters, venue_address,venue_category, venue_lat, venue_lng)SELECT ?,?,?,?,?,?,?,?,?,?,? WHERE NOT EXISTS(SELECT 1 FROM favorites WHERE api_id=?),Ti.App.pin.api_id, Ti.App.pin.tip_avatar,Ti.App.pin.tip_user, Ti.App.pin.tip_text,Ti.App.pin.venue_icon, Ti.App.pin.venue_name,Ti.App.pin.venue_meters, Ti.App.pin.venue_address,Ti.App.pin.venue_category, Ti.App.pin.venue_lat,Ti.App.pin.venue_lng, Ti.App.pin.api_id); tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • JSONDBThursday, 21 February 13
  • • It was slow and difficult to constantly modify, re-index and deploy a new schema everytime we changed the application or added a new feature• Migrating data between versions of the app became very difficult• Our data wasn’t secure – users with jail broken phones could basically change whateverthey wanted whenever they wanted (and they did)• Translating between tabular data and objects in JavaScript turned out to be fairly inefficienteven with our own custom ORM libraries• Maintaining large embedded SQL queries turned out to be a maintenance nightmare inJavaScripthttp://www.irlgaming.com/blog/nosql-ios-titanium-appcelerator-worries/:(tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • • Secure – all data committed to and read from disk is signed so that it can’t be tamperedwith by users• File Based – we didn’t want to run a server on the handset, so all data committed tostorage is encoded as JSON and stored in flat files.This also means that all your data isportable• Simple – we wanted to get rid of ugly SQL queries and make everything object based• Easy – no schema, no indexes, no complicated SQL. Just objects.That’s how JavaScriptshould be.:)http://www.irlgaming.com/blog/nosql-ios-titanium-appcelerator-worries/tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • http://www.irlgaming.com/blog/nosql-ios-titanium-appcelerator-worries/https://github.com/dan-eyles/jsondb-publicReadMORE:tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • SculeJS(from Minuscule - pronounced skyul)Thursday, 21 February 13
  • My vision with SculeJS wasnt just to replace JSONDB, but to providepowerful, generic data structures and algorithm implementations thatdevelopers could leverage in their day to day development.Daniel EylestiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • SculeJSis actually a pretty advanced piece of technology• Implements B+Tree and Hash index types.• Compiles query expressions down to bytecode that is then executed in a hybrid(stack + register) virtual machine.• The system also performs automatic queryexpression normalization and indexselection to optimize the efficiency ofqueries.tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • SculeJSSculeJS can run in your web browser, in a NodeJS process,or even inside iOS and Android applications including aJavaScript runtime environment.https://github.com/dan-eyles/sculejstiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • tiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13
  • APPLAUSE / Q&AtiConf -VALENCIA, FEB 23-24th, 2013#tiConf @joemaffiaThursday, 21 February 13