For developers, having the data where it is needed is a non-trivial challenge. Mobile, IoT, and edge computing challenge cloud-centric solutions where data and processing is done centrally. Decentralized computing along with data locality bring back autonomous apps, which don't depend but coexist symbiotically with the cloud. This does not only reduce cloud costs, but also increases security by reducing the amount of data shared with central instances. Can blockchain be part of the solution?

1. Bringing Data
to the Edge

2. Everything is connected
Dirk Gently

3. Hello daho.am :)
• Local event
Decentralized
Efficient information exchange
Connects local people
• Local data at the edge
Decentralized
Efficient information exchange
Connects local devices

4. Mobile Data – My personal Journey
• Previous life: enterprise stuff
Java, expensive DBs, SQL, O/R Mappers (ORMs)
• Apps, build two mobile ORMs on top of SQLite
Object lover, never a fan of SQL, respect for SQLite
• "We can do better than that"
ObjectBox – founder and CTO

5. What is the "Edge" and Edge Computing?
• Nothing really new
• Edge: devices on the edge of the Internet
E.g. mobile phones, desktops, IoT devices
• Edge gateways
Servers living on the edge
• Locality is a good thing
Goal: provide better solutions directly at the edge

6. And what's "Fog Computing"?
• Sometimes used in the same context (EC)
• "A hierarchic cloud"
Bring computing and data closer to the edge
• Locality levels
E.g. building, city block, city, …

7. So, what's wrong with the Cloud?
Nothing really.
It's just a little overused.

8. 42
Life
Universe Everything

9. Cloud
Web
Mobile IoT

10. But are we asking
the right Question?

11. What happened to "It depends"?
• Cloud == 42?
• The universal answer of IT people: "It depends"
Complex stuff, 1000 different solutions
• Optimal solutions depend on the use case
• When is the Cloud the best solution?
It depends!

12. Cloud Advantages
• Vs. servers: scale & redundancy
Less to worry about, "managed" & less down time
• Central processing is simple
(Mostly) consistent state, easy to monitor, etc.
• Implementing logic just once
Clients just call an interface (e.g. Web/REST)

13. Cloud Disadvantages
• Central everything
Controlled by big companies (governments?)
Hack-able: one breach – millions affected
Who owns my data? Does private data exist?
• Depends on network
Clients are dead when offline, latency
• Cloud costs

14. Cloud adds Latency
• (Near) Real time requirements
E.g. Industrial setting
Cloud not an option

15. Cloud Costs for Apps
• Medium app, 150k monthly users
• Significant work to reduce cloud time
3 caching layers on the cloud side
• US $400-500 per month
• Wouldn't be affordable without caches
Also on-device database with "conditional gets"
• Big apps often spend > $100K per year on cloud

16. Cloud costs: IoT
• Source: CTO of an IoT company
• Single application, huge amount of sensor data
• Local K/V cache to mitigate costs
• Multiple € 100K per month
• "Cloud costs are a problem"
Very hard to make a project profitable

17. Is the Cloud the new Mainframe?
• Once upon a time…
• You could submit "jobs" to mainframes
• "Jobs" moved to desktop computers
• It happened before and it will happen again
• Move "Jobs" to the edge

18. Utilizing the Edge
• While you wait for the cloud…
• What are client devices doing?
• Nothing!
Waiting == wasting resources
• Edge computing is resourceful
Using the powerful resources we already have

19. Data on the Edge – local Data is good Data
• Upload of all data is often wasteful
Example: IoT sensors
Preprocessing & only upload what's needed
• Data that doesn't leave the edge
Private data stays "in the house"
• Again: Edge computing is resourceful
Decrease traffic & increase privacy

20. Moving to the Edge

21. Cloud:
Compute
& Data
Cloud ClientClient
Cloud:
(Compute
& Data)
Edge Edge Device:
Data & Compute
Edge Device:
Data & Compute
Cloud:
(Compute
& Data)
Edge*
Edge
Gateway:
Data &
Compute
Client
Edge Device:
Data & Compute

22. Simple Example: In-App Search is already EC
• App with some kind of catalog
E.g. news, TV data, messages, …
• Sync all (relevant) data to device
Archive could stay in the cloud
• Search data at the edge
Functionality at the device, not in the cloud

23. Edge Computing & IoT

24. IoT devices
• More devices than humans
Likely to double in the next few years
• Big range of technical capability
From wearables to seriously powered devices
• The typical "dumb" device
Sends sensor data to the cloud

25. From IoT to EoT
• Does an "Internet of Things" make sense?
Again: it depends
• Edge of Things
"Internet exposure considered harmful"
• Smart home, edge version
Local area network
Optional gateway to the cloud
Decide what to share

26. Bringing Data to the Edge: Challenges
• Distributed data
Local data, some of it must be synchronized
• Keeping data in sync is complex
Concurrent edits, conflicts, …
• More data stored on device
Higher capacity needed (e.g. flash memory)
Data storage needs to scale (performance)

27. Status quo for getting Data to the Edge
• REST based APIs (or GraphQL)
Typically return JSON data
• No standard to store returned data
E.g. custom logic to insert into a SQL database
• Requests often fetch all data
Often redundant and inefficient

28. Data Sync is a Key Technology
for Edge Computing

29. Data Synchronization
• Generic approach to keep data up-to-date
Pushing updates two way
• Offline support
When not connected: queue updates for later
• Delta synchronization
Sending only deltas is much more efficient
• Conflict resolution

30. git: Good Example of Distributed Data
• Decentralized
• History is not linear
Branches introduce another dimension
• Merge operations
Automatic, or manual conflict handling
• git concepts  data synchronization?
Makes a pretty good starting point

34. git & Crypto: more Inspiration
• Chain of cryptographic hashes
Content is hashed
Previous commit hash is part of the hash
• Tampering data would break the hash chain
Considered a different branch, easy to track down
• Optional: signing
E.g. prove that a commit comes from you

35. Data Synchronization Implementation
• Transaction based databases
A transaction is a list of actions
• A transaction is a state transition
Actions & data can be stored in a log
• Each state should be identifiable (ID)
Enable clients to pick up from previous state
(aka delta synchronization)

36. Data Synchronization Implementation
• State ID can be a hash
Yep, just like git…
• State transition A  B with transaction log TXL
hash(B) = hash(hash(A) + hash(TXL))
• Chain of hashes
• Clients can pick up from their last hash ID
Delta sync unlocked

37. How about…
100% Decentralized?

38. Full Decentralization – Though Challenges
• Anarchy and Chaos?
No central instance may/can control/interfere
• Data bubbles
Data gets separated and inconsistent
• Trust issues
Which peers to trust?

39. Blockchain!
The new silver bullet…

40. Blockchain is a DB
Decentralized P2P, public, crypto-based

41. Blockchain Achievements
• Alternative investments & a new mining industry
Eventually also new currencies
• Smart contracts
"If this then that" - combining data and code
• Crypto is cool again
• Working well enough
Showed the world that there are alternatives

42. Blockchain Tech
• Block of transactions
Merkle tree to produce a hash
• Diverging data/history
One branch is selected using a scoring model
(e.g. longest chain)
• Proof-of-work
Calc nonce to find special hash(content + nonce)

43. Blockchain Tech – the Dark Side
• Decentralized consensus: ridiculously inefficient
Compared to central instance approval
• Proof of work: "useless" expensive computation
Redundantly done by thousands of miner nodes
• Nonce lottery is like "wasting" ~99.99% of energy
• Effects on the environment?
• (Today's) blockchain is not the end of the journey

44. Data Sync vs. Blockchain
• Implementations can share concepts
Transactions, chained hashes
• Key difference: consensus and trust
Centralized vs. decentralized
• Centralized consensus is preferred for apps
Producing companies "own" their apps
(Efficiency restored… )

45. What we are working on

46. ObjectBox – a Database for the Edge
• "A DropBox for data objects"
Allows to work offline
• We start at the edge: embedded database
Based on objects and relations
Sync: stored bytes can be sent and applied 1:1
• Simplicity and efficiency
Guiding principles

47. ObjectBox
• Runs on mobile, desktop, IoT devices
Android, Linux, Windows, i/MacOS, Raspberry, …
• Low footprint: less than 1 MB
• 10x faster CRUD operations
Outperforms SQLite, fastest embedded DB
• ACID compliant

48. ObjectBox
• Objects all the way through
No transformations required
(no SQL, no REST/JSON parsing)
• Convenience at K/V store speed
Indexing, queries, relations
• ObjectBox 2.0 Release
25th July 2018

49. Wrapping up

50. Edge Computing - the Golden Hybrid?
Centralized Decentralized
Server
Cloud
Peer-to-Peer
Edge
"Super Peers"

51. Data Dimensions
• Data localilty
Cloud, edge-only, shared cloud-edge
• Online vs. offline data
Online-only cloud, offline: synced or edge-only
• Consensus: which data prevails?
Central authority, decentralized heuristics

52. Questions to ask
• Who owns "my data"?
Can data be private at all?
• How much (de)centralization do you need?
What works best for your use case?
• How can we make efficiency a virtue again?
Can efficiency reduces your costs too?

53. Thanks! 🐬🐬
Contact:
markus @ objectbox.io
Follow me:
@greenrobot_de