The document discusses using offline storage on mobile devices by leveraging HTML, JavaScript, and technologies like Cordova, PouchDB, and CouchDB. It summarizes the evolution of HTML5 offline storage standards including Web Storage, Web SQL, and IndexedDB. It then outlines an approach to building a mobile app that works offline by storing data locally on the device using IndexedDB and syncing with a remote CouchDB database using PouchDB when an internet connection is available.

1. 1

2. • Principal IT Specialist, Office of the Sergeant at Arms, United States Senate
• Mostly work in Python and Django
• Web Developer for Over 15 Years
• Newbie when it comes to mobile development
2

3. • Why you might want to consider doing offline storage of data on mobile devices
• How to do it using the skills you probably already know, meaning HTML and JavaScript
• A few technologies that may be new to some of you, specifically Cordova, as well as
PouchDB and CouchDB
• There are of course a million ways to do this, but these are the technologies I
used
• Caveats and challenges to developing applications this way
• Possible future enhancements
3

4. • Due to time constraints we won’t be covering:
• The ins and outs of the mechanics of deploying to specific mobile platforms, app
stores, etc.
• Platform-specific considerations for functionality and necessary visual
adjustments and tweaks (luckily Cordova makes this pretty easy)
• The minutiae of every possible method for storing data on a mobile device
• Comprehensive details on each of the tools involved
• Some of the slick cool hybrid mobile frameworks like Ionic
• That said find me after the talk throughout the rest of the conference and I’m more than
happy to go into details on any of this
4

5. • We rely on mobile devices to the point where they’ve become an appendage
• In addition to all the wholly unnecessary things we use our phones for, they’re also used
for mission critical purposes and to access important data
• Mobile devices can be particularly handy in emergencies since we have them with us,
ahem, “everywhere we go”
• And with this whole cloud thing you may have heard of, our important information is
available everywhere from any device
• Between WiFi and mobile networks, access doesn’t tend to be an issue, BUT …
5

6. • Yes, I’ll ask the question no one wants to ask: What if the Internet’s down?
• And when I use the term “Internet” I’m speaking of network access in general,
meaning assume you have no access to either WiFi or mobile networks
• The ironic thing is the time we might most need access to all that stuff we put in the
cloud is when the cloud itself, or our ability to access it, is down
• Granted if the Internet’s down you can’t do things like call or text people, but you might
have important information up in the cloud such as:
• Where you hid your zombie kit
• The passcode to get into your bunker
• The geographic coordinates of your stash of gold bullion, and
• Your grandma’s top secret chocolate chip cookie recipe
6

7. • If we don’t plan for the scenario we hope never happens, there’s not a lot you can do
other than yell at the cloud.
• Luckily with just a little bit of foresight and planning, and a healthy mix of HTML,
JavaScript, and some other handy tools, we can make sure that in the event of any flavor
of apocalypse, you’ll still be able to make your grandma’s chocolate chip cookies
(provided you can find an unlooted supermarket of course)
7

8. • I am not a mobile application developer.
• I was presented with a challenge at work and asked if I could come up with a solution,
so as developers do I of course said “yes” first and figured out how to do it after
committing
• The resultant application was my first foray into anything specifically built for mobile
devices
• Thankfully with the tools I’m going to discuss it turned out to not be too terribly difficult,
but I want to make it clear that there may be be better/more preferable ways to do all
this and if there are people here who ARE experienced mobile developers and I’ve done
anything horribly wrong I’d love to know more about how to do all of this better
8

9. • The state of tools is such that even if like me you don’t have a lot of experience with
mobile development, it’s dangerously easy to build mobile apps
• On the client side of the app, of course the foundation of all this is HTML5, and
specifically HTML5 local storage that allows us to easily store data locally on any device
• Add in CSS3 and Twitter Bootstrap 3 to get an easy mobile-first design layer for UX-
challenged people such as myself
• JavaScript and jQuery of course are pretty much essential to any webapp you’re going to
build these days, mobile or otherwise
• A tool you may not have heard of that I’ll talk about in more detail later is PouchDB,
which is a JavaScript database that acts as an abstraction layer for HTML5 local storage
• PouchDB is also the secret sauce in this particular example application because as you
might guess from the name it syncs seamlessly with CouchDB, making keeping the data
between the mobile clients and the server in sync dead simple
• CouchDB, if you’re not familiar with it, is a NoSQL document database that’s completely
built around HTTP, JavaScript, and JSON so it’s a natural fit for a webapp, and a
particularly good fit here
• Apache Cordova, which you might also know by the name PhoneGap, is what builds the
client code into a mobile app for each targeted platform, which in my case was iOS and –
yes it’s true – BlackBerry
• On the server side of the equation, I have a really simple Python and Django application
9

10. used for managing the data, but you could of course use any backend
• Honestly if your needs are simple enough you could get away with just syncing to
CouchDB and be done with it, provided you let users manage the data from the
mobile device
• In the case of the real world app I built following this basic model the mobile app
is read-only by design, hence why we needed a way to manage the data
• Also in the real world app I build I chose to use Postgres instead of CouchDB since
I had some other considerations but that didn’t negate the benefit of using
PouchDB, it’s just a whole lot slicker with CouchDB in the mix which is why I
decided to use it for the app I built for this presentation
9

11. • When you first load the application on your phone it will try to connect to the cloud and
get a data update
• If it can connect, and if it needs an update (we’ll get into how to control that later), it
will pull the latest data and repopulate the local database
• If it can’t connect, it can simply continue to operate using the local database
• You’ll notice I also have some connections directly between the database on the mobile
device and CouchDB on the server, and we’ll talk about all that in a bit as well
10

12. • HTML5 local storage follows the one standard of all things interwebs, and that standard
is that there are multiple incompatible implementations with varying degrees of support
in various browsers
• If you’re not familiar with the evolution of HTML5 offline storage, the short story is that
there have been three major standards: Web Storage, Web SQL, and IndexedDB
11

13. • Web Storage is a W3C spec, reaching the recommendation stage in July of 2013
• Made up of 2 parts: localStorage, and sessionStorage
• localStorage was the early favorite solution for in-browser, offline storage
• Was originally part of the HTML5 spec but it’s now its own spec
• Available in all browsers early on
• Simple key/value API
• Downsides:
• It doesn’t look or work anything remotely like what people would consider to be
a database
• Can only store strings
• There’s no query language associated with it, no schemas, no transactional
safety
• Didn’t scale well for large datasets since to find anything you basically had to
loop over the entire object store, has other performance issues
• Still exists and is still widely supported, but isn’t a particularly viable solution for most
applications
12

14. • WebSQL was a W3C standard for an API that allowed for storing data that could be
queried using SQL
• Web SQL seemed like the direction things were going for a while since it makes logical
sense and addresses the problems with localStorage
• Initial implementation in Webkit-based browsers like Chrome and Safari was to use
SQLite in the browser
• SQLite is a mature, proven, solid solution even for high traffic and with large datasets
• As the name implies, it has SQL!
• But, it was only available in Chrome and Safari, it was never implemented in Firefox or
IE, and will likely never be implemented in either
• The W3C abandoned the specification for Web SQL in 2010 since there was only one,
and would evidently only ever be one, implementation of the standard, which was
SQLite
• There is still a movement behind revising Web SQL because as we’ll see in a second, the
implementation of in-browser storage that won out has its quirks as well
13

15. • IndexedDB is a W3C standard, reached recommendation stage in January of 2015
• The easiest way to describe IndexedDB’s characteristics is to think of it as a NoSQL-style
database
• Specifically it’s most closely related to document-based datastores such as CouchDB or
MongoDB
• Just so it’s clear, unlike the abandoned Web SQL standard, you can’t use SQL with
IndexedDB
• On the plus side, with IndexedDB you’re basically just throwing JavaScript objects at it as
they are and it’ll store it
• Note that I said “as they are” which means, unlike localStorage, it is capable of
storing more than just strings
• Objects are indexed and retrieved with a unique key
• The key can be created by you, or you can specify a field in the object to use as
the key, or you can use auto-incrementing keys
• All changes to the database are transactional, which is a big advantage over localStorage
• For pulling back multiple documents, you can use a cursor and loop over the objects in
the database
• Also, as the name indicates, in IndexedDB you can create indexes on fields within your
documents and retrieve one of multiple documents via the indexes
• Remember that you don’t have SQL capabilities so you can’t do something like SELECT *
14

16. FROM MYDB WHERE FIRSTNAME = ‘Bob’ but with judicious use of indexes you can grab
things and loop over the results to find what you need pretty easily
• IndexedDB has full support in Chrome (both desktop and Android) and Firefox
• Partial support in IE starting with IE 10
• Even works on BlackBerry as of BlackBerry OS 10
• So you might be thinking my short answer to the question of which in-browser storage to
choose would that IndexedDB is the clear winner. And it is. Or it should be anyway.
• If you were paying attention – and I trust you were – you noticed I failed to mention a
pretty big browser in the mobile world.
14

17. • Apple waited a long time to implement IndexedDB in Safari
• Didn’t exist at all on the desktop until 7.1
• Wasn’t in iOS until version 8
• Good news is, it’s there
• Bad news is, it’s broken. Like FUBAR broken. Like the primary keys don’t work right and
data is overwritten unexpectedly broken.
• I have some links in the resources slide but Ray Camden as well as Nolan Lawson from
PouchDB have great blog posts covering the issues in detail if you’re interested
• Short answer: don’t use IndexedDB on Safari.
• So if we want an app to work on multiple platforms the picture looks pretty bleak! We’re
faced with using something:
• Not very good – localStorage, or
• An abandoned standard – WebSQL – that will work in Chrome and Safari, but
won’t work in Firefox and IE
• So you can’t just use IndexedDB. Because Safari.
15

18. • PouchDB is an in-browser database written in JavaScript that simplifies working with
HTML5 local storage
• Simplifies how you ask?
• As the old adage goes, “All problems in computer science can be solved by another level
of indirection”
• What PouchDB does is provide a standard layer on top of the underlying browser-
specific implementations
• This allows you to write your application code to use PouchDB, and then PouchDB
decides what is the best underlying storage engine to use for the browser on which the
application is running
• You can tell PouchDB which engine to preferably use if it can, but for example with Safari
what PouchDB will do is use IndexedDB on Android and BlackBerry, but on Safari it will
use Web SQL instead since Safari’s implementation of IndexedDB is horribly broken
• So to you as a developer you only have to figure out how to work with PouchDB, and
PouchDB handles the rest
• The other nice thing is PouchDB is, in my opinion, a better, simpler API than using
IndexedDB directly, and it also gives you capabilities for querying that IndexedDB doesn’t
have
• If you’re familiar with CouchDB, PouchDB lets you write views which is some
seriously powerful stuff
16

19. • So my true short answer to the question of which in-browser storage to use is “use
PouchDB.”
• If for some crazy reason you don’t want to use PouchDB there are other options for
putting a layer on top of in-browser storage
• Best one I’ve come across is called localForage – it’s a Mozilla project and is
available on github
16

20. • In addition to what I said earlier about PouchDB being a JavaScript in-browser database,
it’s also a JavaScript implementation of CouchDB, which is a popular NoSQL document
database
• CouchDB is extremely well-suited to web and mobile apps by virtue of the fact that it’s
built entirely around HTTP and REST, but its major killer feature is that it was built from
the ground up with amazing replication and synchronization capabilities
• PouchDB is great in and of itself, and you can choose to use it alone, or even as a
JavaScript client that connects directly to a remote CouchDB database.
• It’s when you combine PouchDB on the mobile device with CouchDB on the server
things get really interesting, particularly given the challenges of online/offline mobile
development
• PouchDB can sync to anything that implements the CouchDB replication protocol
• In practice that pretty much means it syncs to CouchDB and its commercial
variants Cloudant and Couchbase
• If you want to sync to other datasources you can easily write scripts to keep
things updated between an RDBMS like Postgres and CouchDB
• Or, as I said if you want to just have your mobile app grab data via API that fronts
a relational database, that’s fine as well
• Due to the design of CouchDB you get extremely powerful online/offline sync and data
conflict detection and resolution “for free” so if you are in a situation where you need to
17

21. build an app of this nature, it’s well worth considering
17

22. • Great quote from Jason Smith, a CouchDB committer: “The way I like to think about
CouchDB is this: CouchDB is bad at everything, except syncing. And it turns out
that's the most important feature you could ever ask for, for many types of
software.”
• If you’re in a situation where users need to be able to update data from their mobile
devices, good sync capabilities are pretty important
• To do bi-directional syncing between PouchDB on the mobile device to CouchDB on the
server, despite what Luke might think, it’s literally just that one line of JavaScript where
localDB is a variable that represents your local database, and remoteDB is a URL pointing
to a remote CouchDB server
• You can either call replication on demand, or set it to do continuous replication, which
sounds great at first but has a lot of implications for an online/offline app
• You can also do unidirectional replication in either direction by calling replicate to and
replicate from functions
• Replication is asynchronous and PouchDB’s sync function has on complete and on error
callback functions
18

23. 19

24. 20

25. • The immediate concern with synchronizing between multiple databases is data conflicts,
particularly if people are able to change data when they’re offline
• CouchDB supports multi-master architecture and every document in the database is
versioned
• Specifically this means every document has a revision key that is unique, and
that has to be included if you’re going to update an existing document
• All this is a topic in and of itself, but the long and short of it is if you have two conflicting
updates to the same document, CouchDB will handle the conflict, and either when an
update to a document is rejected, or by using the conflicts feed to audit conflicting
documents, you can handle this as you see fit in your app
• To give a specific example, when someone wants to update a document and they’re
offline, you could let them do this locally and save it in PouchDB.
• Then when they’re online and sync to CouchDB, if someone else had changed the same
document and synced before they did, CouchDB would reject the update and you could
then alert the user that the update was rejected because someone else already updated
that document.
• And if you wanted to take things a step further, you could even grab the newer
version of the document and show them the changes so they could see what
differences there are between the two versions and make a decision as to which
version, or what combination of versions, should be sent to CouchDB.
21

26. • If you want to see what’s going on in your local databases, the developer tools in the
browsers all have really good support for inspecting and messing around with local
databases of all flavors
• PouchDB also has an add-on for both Chrome and Firefox that’s very similar to the
CouchDB Futon tool
22

27. • Now with all the concepts in place, let’s see how they’re put into practice in our sample
application
23

28. • As I mentioned at the beginning I didn’t have any time to talk about the mechanics of
deploying to the various platforms
• Long and short of it is use Cordova – you can specify target platforms and get a build
specific to each platform
• From there you follow the procedure for each specific platform such as code signing,
etc. to make it fully ready for the target platform
• Cordova also supports platform-specific tweaks such as having icons, CSS, etc.
appropriate to each platform
• Very easy, very powerful tool that is a huge piece of how we web developer types can
get away with building cool mobile apps using the skills and tools we already know
• Warrants further discussion but for the purposes of this talk know that Cordova pretty
much just works and is an indispensable piece of this puzzle
24

29. • Other stuff I didn’t have time to cover includes:
• Application security! Remember, as a web-based app you have many of the same
concerns when you deploy to mobile as you do with webapps in general.
• Data security! Once the data is on the device, how is it secured?
• Frameworks like Ionic that make your life a whole lot easier and your apps a
whole lot better. I purposely did plain old HTML for this app so we didn’t get
bogged down with the complexities of any particular hybrid mobile development
framework.
• Platform-specific customizations, which thankfully most of the tools used in this
type of development handle pretty well.
• App store deployment and all the rigamaroll around that.
25

30. • I know I covered a lot of low-ish-level technical information but if there’s one thing I
hope you take away from this presentation it’s that as a web developer, building pretty
sophisticated mobile apps that make critical data available in an emergency is relatively
easy.
• You can use the languages and tools you already know, including HTML5, JavaScript, and
CSS
• Combine that with a few other key technologies like PouchDB to take away some of the
pain of dealing with in-browser storage, CouchDB on the server for data synchronization
and distribution, and Cordova to build your app for each of your necessary target
platforms, and – just as I did – you can pretend that you’re a real mobile developer in no
time!
26

31. 27

32. 28