Three years ago Store4, the little library that could, was released at KotlinConf'19. Store has been simplifying data loading on Android for close to a decade and was supercharged by being 100% Kotlin. Today we're here to talk about the next paradigm shift in data loading Store5 - a Kotlin Multiplatform solution for reading, writing and resolving data conflicts on any platform that Kotlin supports (Android, iOS, Web and Desktop). The Android community has embraced Store for close to a decade, Kotlin is making it possible to adopt the same patterns on other mobile platforms and beyond. With the addition of support for updating remote sources, network resilience, pain free conflict resolution, and a highly extensible api - Store5 aims to make reading and writing data effortless on all Kotlin platforms. This talk will focus on Store5 foundational concepts and usage in production and at scale. We will be covering adopting KMP, applying Google's offline-first guiding principles beyond Android and how we hope to establish a seamless way for all apps, regardless of platform, to work with local and remote data. This talk is not to be missed for folks (like us) who have battle scars from years of working on hard to fix bugs in offline first applications.

1. MobileNativeFoundation/Store
meet
Store 5
mattramotar@androidDev.social
friendlyMike@androidDev.Social
Droidcon SF 2023

2. What is Store?

3. Store is a Java library for effortless, reactive data loading.
Guarantee things [you don’t want to think about] are done in the same way
What is Store?

4. Contracts + RELIABILITY
Guarantee things are done in the same way
What is Store?

5. Store
A brief history
2017 2019 2021 2022 2023

6. Store A brief history
2017
2019
1.0 Released Jan 17th, 2017 by NY Times

7. 2017
2019
Why did we need Store?
We wanted to do complicated thing once

8. Store Version 1.0 Api
2017
2019
Store [was] a Java library for effortless, reactive data loading built with RxJava
Store<Article> Store = ParsingStoreBuilder.<BufferedSource, String>builder()
.fetcher(this::ResponseAsSource)
.persister(SourcePersisterFactory.create(context.getFilesDir())
.parser(GsonParserFactory.createSourceParser(gson, Article.class))
.open();
Barcode barcode = new Barcode("Article", "42");
store.get(barcode).subscribe(onNext, onError, onComplete)

9. 2017
2019
Fresh == skip memory/persister go directly to fetcher
store.fresh(param).subscribe(onNext, onError, onComplete)
Fresh Skip Caches

10. Store Why You Need User Research
2.0 Released 1 month later (oops!)
2017
2019

11. 3.0 Api Rxjava2
RxSingle instead of Observable
Store<Article> Store = ParsingStoreBuilder.<BufferedSource,ArticleParam, String>builder()
.fetcher(this::ResponseAsSource) //responseBody.source()
.persister(SourcePersisterFactory.create(context.getFilesDir())
.parser(GsonParserFactory.createSourceParser(gson, Article.class))
.open();
ArticleParam param = new ArticleParam("42");
store.get(param).subscribe(onNext, onError)
2017
2019
Store

12. Coroutines have hit stable
2017
2019
2021
Fast forward to 2019

13. Bye bye RxJava Hello Kotlin
2017
2019
2021
Store4
Fast forward to 2019

14. Hello Kotlin
2017
2019
2021
Store4
StoreBuilder.from { api.fetchSubreddit(it, "10")}
.sourceOfTruth(
reader = db.postDao()::loadPosts,
writer = db.postDao()::insertPosts,
delete = db.postDao()::clearFeed)
.cachePolicy(MemoryPolicy)
.build()
Api

15. Streaming as first class citizen
2017
2019
2021
fun stream(request: StoreRequest<Key>): Flow<StoreResponse>Output>>
lifecycleScope.launchWhenStarted {
store.stream(StoreRequest.cached(3, refresh = false))
.collect{ }
store.stream(StoreRequest.get(3)) .collect{ storeResponse -> }
Store4 Api

16. Loading|Content|Error Return Types
2017
2019
2021
Store 4
store.stream(StoreRequest.cached(key = key, refresh=true)).collect { response ->
when(response) {
is StoreResponse.Loading -> showLoadingSpinner()
is StoreResponse.Data -> {
if (response.origin == ResponseOrigin.Fetcher) hideLoadingSpinner()
updateUI(response.value)
}
is StoreResponse.Error -> {
if (response.origin == ResponseOrigin.Fetcher) hideLoadingSpinner()
showError(response.error)
}}}}
LCE

17. Using Store 4
Examples are from Firefly for Mastodon
https://github.com/digitalbuddha/Firefly
2017
2019
2021

18. You create a Store using a builder. The only requirement is to include a Fetcher which is just a typealias to a
function that returns a Flow<FetcherResult<ReturnType>>.
//build it
val store = StoreBuilder.from(
Fetcher.of { key: Unit ->
userApi.noti
fi
cations(
authHeader = " Bearer ${oauthRepository.getCurrent()}",
offset = null)
}).build()
//use it
store.stream(StoreRequest.cached(Unit, refresh = true)).map { it.dataOrNull()
}
.collect{response-> }
Fetcher as a Function
2017
2019
2021

19. How we get data
2017
2019
2021

20. Do the hard thing once
How to work with local persistence

21. Source of Truth
suspend fun fetcher(key:String): Token = api.createAccessToken(key)
fun sourceOfTruth = SourceOfTruth.of<String, Token, String>(
reader = {dataStore.data.map { it.currentUser?.accessToken }},
writer = { _, token -> dataStore.updateData {
it.user.copy(accessToken = token.accessToken)
}})
private val userTokenStore: Store<String, String> =
StoreBuilder.from(
fetcher = Fetcher.of { key: String -> fetcher(key) },
sourceOfTruth = sourceOfTruth
).build()
2017
2019
2021

22. 2017
2019
2021
Store Data Flow

23. Fast forward to 2022
Where we were 365 days ago
2021
2022
2023

24. Store 4 is stable
Less than a dozen open bugs
2021
2022
2023

25. API was perfect
As long as you only asked me
2021
2022
2023

26. We had LOTS of feature requests
2021
2022
2023

27. That I was ignoring
2021
2022
2023

28. Write Support (CRUD)
2021
2022
2023

29. Conflict Resolution
2021
2022
2023

30. Fallback Mechanisms
2021
2022
2023

31. Support for Collections
2021
2022
2023

32. Store 5
Introducing
Store now with batteries included

34. Mission-Critical
Server-Driven Architecture

35. Sync

36. Collaboration

37. Upload + Download

38. Data Protection

39. Offline Access

40. Campaigns

41. Server-driven UI

42. Don’t worry
Store 5 is a superset of Store4
Which targets JVM/iOS/Web

43. How we got here…

44. Try 1…Adding Write Support
Rewrite what you don’t understand

45. Mike: What is the bene
fi
t of starting over?
Matt: Hmm
Mike: Is it easier to add old to new or new to old?

46. Some time later… (half hour)
Matt: safeguards will be easier to add to new
Matt: because I don’t understand old
Mike: SGTM as long as all tests are migrated, I also
don’t understand old
Matt: Sure how hard can it be?

47. Many days later:
Mike: ….
Matt: …

48. More time passes
Mike: My spider sense is tingling, I found more
gaps in new
Mike: let’s sit together and go through Store4
code

49. Try 2…Extending the old
Rather than boiling the ocean

50. Few weeks later…
Matt:Just finished last test cases everything passes
Mike: Everything?
Matt: Yup in iOS, android, JVM and JS, I took all the
work https://github.com/aclassen did and got it to pass
tests

52. What’sinStore5

53. Mutations
Con
fl
ict Resolution
Fallback Mechanisms
List Decomposition
Cache Delegation
Validation
Concepts

54. meet
Trails

55. An offline-first companion
Trails

59. User Story - Hiking
without Internet

60. Trails Architecture TLDR

61. Data Pulled & Pushed

62. Source of Truth to
Repository
Data Flow

63. Can’t do with Store 4
Source of Truth to
Repository
Data Flow

64. interface Updater<Key, Output, Response> {
suspend fun post(key: Key, value: Output):
UpdaterResult
val onCompletion:
OnUpdaterCompletion<Response>?}
Store 5 New Mutation Api
Creating an Updater (like a fetcher)

65. UpdatingRemoteData
sample

66. val updater = Updater.by(
post = { _, hike ->
try {
val latest = api.updateHike(hike)
UpdaterResult.Success.Typed(latest)
} catch (error: Throwable) {
UpdaterResult.Error.Exception(error)
}})
Store 5 Updater Factory

67. val mutableStore = store.asMutableStore(
updater = updater,
bookkeeper = bookkeeper)
Store 5 Mutable Store Builder

68. class RealHikeManager @Inject constructor(
private val mutableStore: MutableStore<Int, Hike>,
private val location:locationRepository,
private val map:mapRepository
): HikeManager {
private suspend fun track() {
combine(location.current(),
map.current()) { location, map ->
val request =
createStoreWriteRequest(location, map)
val response = mutableStore.write(request)
handleResponse(response)
}}}
Store 5 Usage of MutableStore

69. class RealHikeManager @Inject constructor(
private val mutableStore: MutableStore<Int, Hike>,
private val location:locationRepository,
private val map:mapRepository
): HikeManager {
private suspend fun track() {
combine(location.current(),
map.current()) { location, map ->
val request =
createStoreWriteRequest(location, map)
val response = mutableStore.write(request)
handleResponse(response)
}}}
Store 5 Usage of MutableStore

70. What happens if write fails
or a competing write?

71. Con
fl
ict Resolution
Store 5 Concepts

72. https://developer.android.com/topic/architecture/data-layer/o
ffl
ine-
fi
rst#last-write
Non Happy Path == Hard

74. How Does Store 5 Resolves Con
fl
icts?
Implementing Principles

75. How Store 5 Resolves Conflicts
We Don’t You Do!
With Delegation

76. What we need from you
A Bookkeeper (delegate)

77. Bookkeeper?
Tracks when local changes fail to sync with
network

78. Store 5 Bookkeeper Api
interface Bookkeeper<Key : Any> {
suspend fun getLastFailedSync(key: Key): Long?
suspend fun setLastFailedSync(key: Key, timestamp: Long): Boolean
suspend fun clear(key: Key): Boolean
suspend fun clearAll(): Boolean}

79. MutableStore
+ Bookkeeper
Before completing a read request,
MutableStore asks Bookkeeper
whether any conflicts might exist

80. Store 5 - Any Conflicts? RealMutableStore
private suspend fun con
fl
ictsMightExist(key: Key): Boolean {
val lastFailedSync = bookkeeper?.getLastFailedSync(key)
return lastFailedSync != null || writeRequestsQueueIsEmpty(key).not()
}

81. MutableStore
+ Bookkeeper
And if conflicts might exist,
MutableStore tries to eagerly resolve them

82. What we think you should do
Resolve conflicts on remote

83. On each write request
1. Init thread safety (key-scoped)
2. Add request to stack (key-scoped)
3. Write to memory cache + SOT!!!
4. Try to update network (posting latest)
Mutations In the Weeds

84. On each write request
1. Init thread safety (key-scoped)
2. Add request to stack (key-scoped)
3. Write to memory cache + SOT!!!
4. Try to update network (posting latest)
In the Weeds
Mutations

85. On each write request
1. Init thread safety (key-scoped)
2. Add request to stack (key-scoped)
3. Write to memory cache + SOT!!!
4. Try to update network (posting latest)
In the Weeds
Mutations

86. On each write request
1. Init thread safety (key-scoped)
2. Add request to stack (key-scoped)
3. Write to memory cache + SOT!!!
4. Try to update network (posting latest)
In the Weeds
Mutations

87. When network response
1. Success
1. Reset stack
2. Reset bookkeeper
2. Failure
1. Log with bookkeeper
Try to update network

88. When network response
1. Success
1. Reset stack
2. Reset bookkeeper
2. Failure
1. Log with bookkeeper
Try to update network

89. When network response
1. Success
1. Reset stack
2. Reset bookkeeper
2. Failure
1. Log with bookkeeper
Try to update network

90. When network response
1. Success
1. Reset stack
2. Reset bookkeeper
2. Failure
1. Log with bookkeeper
Try to update network

91. When network response
1. Success
1. Reset stack
2. Reset bookkeeper
2. Failure
1. Log with bookkeeper
Try to update network

92. Fallback Mechanisms

93. Always have a backup plan!
Fallback Mechanisms

94. Store 5 Fetcher Api
fun <Key : Any, Network : Any> of(
name: String? = null,
fetch: suspend (key: Key) -> Network
): Fetcher<Key, Network> = ofFlow(name, fetch.asFlow())

95. fun <Key : Any, Network : Any> of(
name: String? = null,
fallback: Fetcher<Key, Network>,
fetch: suspend (key: Key) -> Network
): Fetcher<Key, Network> =
ofFlowWithFallback(name, fallback, fetch.asFlow())
Store 5 Fetcher Api with fallback

96. User Story - A/B Testing

97. data class FeatureFlag(
val key: String,
val name: String,
val description: String,
val kind: Kind,
val version: Int,
val creationDate: Long,
) : Identi
fi
able<String> {
enum class Kind { Boolean, Multivariate}}
Trails Feature Flag Model

98. Trails Feature Flag Status Model
sealed class FeatureFlagStatus: Identi
fi
able<String> {
data class Multivariate(
val key: String,
val value: FeatureFlagVariation,
val lastRequested: Long,
val links: Links)}

99. Trails Feature Flag
Architecture

100. Fallback Mechanisms
sample

101. Network Feature Flag Status Fetcher
With Fallback
val networkFetcher = Fetcher.ofWithFallback(
name = “networkFetcher”,
fallback = hardcodedFetcher
) { key ->
when (key) {
is Collection -> fetchFeatureFlagStatuses(key.userId)
is Single -> fetchFeatureFlagStatus(key.userId, key.
fl
agId)}}

102. Hardcoded Feature Flag Status Fetcher
val hardcodedFetcher = Fetcher.of(
name = “hardcodedFetcher”
) { key ->
when (key) {
is Collection -> loadFeatureFlagStatuses(key.userId)
is Single -> loadFeatureFlagStatus(key.userId, key.
fl
agId)}}

103. Build Store as normal
val store = StoreBuilder.from(
fetcher = networkFetcher,
sourceOfTruth = sourceOfTruth,
memoryCache = multiCache
).toMutableStoreBuilder<FeatureFlagStatusData,
FeatureFlagStatusData>().build(
updater = updater,
bookkeeper = bookkeeper)

104. Concepts
List Decomposition
Cache Delegation

105. WorkingWithCollections
sample

106. Just works
Store Loves KMP
We target Android/
Desktop/iOS/JS

107. Store 5 Beta is out now
github.com/mobilenativefoundation/store