Open Sourcing the Store
What we open sourced and how you can too!
Mike Nakhimovich - New York Times
Agenda For Today
Walkthrough of New York Times’ first Android
Open Source Library
How you can make open source a library
yourself
Problem Set
Modern Android Apps need their data representations to be fluid and always
available.
Users expect their UI experience to never be compromised (blocked) by new
data loads
International users expect minimal data downloads
Android === Open Source
Libraries fill gaps left by framework
Network Client? OKHTTP/Retrofit
Threading/Functional Programming? RxJava
Dependency Injection? Dagger
View Bindings? Butterknife
Android === Open Source
Libraries fill gaps left by framework
Network Client? OKHTTP/Retrofit
Threading/Functional Programming? RxJava
Dependency Injection? Dagger
View Bindings? Butterknife
Store? Data Flow & Caching
Introducing Store
https://github.com/NYTimes/Store
A Store abstracts fetching, parsing,
caching, and retrieval of data in
your application.
Stores are configurable and enforce
uni-direcitonal data flow while
exposing data as RxJava
Observables
Sounds Great, but how do I
use them?
Add to build.gradle
compile 'com.nytimes.android:store:1.0.1'
Open A Store
Store<Article> articleStore = StoreBuilder.<Article>builder()
.open();
Add a fetcher
public interface Fetcher<T> {
Observable<T> fetch(BarCode barCode);
}
Store<Article> articleStore = StoreBuilder.<Article>builder()
.fetcher(barCode -> api.getArticle(barCode.getKey()))
.open();
Or NonObservable Fetcher
Use nonObservbleFetcher when connecting to a synchronous & non observable
API
Store<Article> articleStore = StoreBuilder.<Article>builder()
.nonObservableFetcher(barCode -> api.getArticle(barCode.getKey()))
.open();
Call your Store with a Barcode
...
BarCode barcode = new BarCode("Article", "42");
articleStore.fetch(barcode)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Article>() {
public void onCompleted() {}
public void onError(Throwable throwable) {//handle error}
public void onNext(Article article) {//bind Article to UI}});
Fetch Dataflow (Unidirectional)
Same request <1min will
get throttled response
Getting Cached Data
articleStore.get(barcode)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Article>() {
public void onCompleted() {}
public void onError(Throwable throwable) {//handle error}
public void onNext(Article article) {//bind Article to UI}});
Note: if nothing cached get() will act exactly like fetch()
Getting Cached Data
Note: if nothing
cached get() will act
exactly like fetch()
Get Dataflow
What happens if we
need to transform
response?
Adding a Parser
Adding a Parser
Store<Article> articleStore = ParsingStoreBuilder.<BufferedSource,
Article>builder()
.fetcher(barCode -> api.getSource(barCode.getKey()))
.parser(source -> {
try (InputStreamReader reader = new
InputStreamReader(source.inputStream())) {
return gson.fromJson(reader, Article.class);
} catch (IOException e) {
throw new RuntimeException(e);} })
.open();
Dataflow with Parser
Introducing Middleware
Use our Middleware
Before:
.parser(source -> {
try (InputStreamReader reader = new
InputStreamReader(source.inputStream())) {
return gson.fromJson(reader, Article.class);
} catch (IOException e) {
throw new RuntimeException(e);} })
Use our Middleware
Before:
.parser(source -> {
try (InputStreamReader reader = new
InputStreamReader(source.inputStream())) {
return gson.fromJson(reader, Article.class);
} catch (IOException e) {
throw new RuntimeException(e);} })
After:
.parser(new GsonSourceParser<>(gson, Article.class))
Disk Caching
ParsingStoreBuilder.<BufferedSource, Article>builder()
.fetcher(barCode -> api.getSource(barCode.getKey()))
.parser(new GsonSourceParser<>(gson, Article.class))
.persister(new Persister<BufferedSource>() {
public Observable<BufferedSource> read(BarCode barCode) {
return Observable.fromCallable(() -> diskCache.get(barCode));
}
public Observable<Boolean> write(BarCode barCode, BufferedSource source) {
diskCache.save(barCode, source);
return Observable.just(true);
}})
.open();
Dataflow with Cache
No Need to implement
yourself
articleStore = ParsingStoreBuilder.<BufferedSource, Article>builder()
.fetcher(barCode -> api.getSource(barCode.getKey()))
.parser(new GsonSourceParser<>(gson, Article.class))
.persister(new SourcePersister(FileSystemFactory.create(getFilesDir())))
.open();
Hope you’ll use Stores
And contribute back to the project
Part 2
How to go from 0 to Open
Source Project
Android Studio
Create a Project
Add a library module
Add some code to library
Use Android Studio to
create Github Project
Add a
Descriptive
Readme.md
● Who is project for?
● What was motivation for creation?
● What does it do?
● How does it do it?
Add a License
Part 2B: How to upload
project to Maven Central
Create a Sonatype Jira
Account (yes really)
Open A ticket to claim an organization
Wait for Response
Whats a GPG Key?
Use GPG to create a key
http://central.sonatype.org/pages/working-with-pgp-signatures.html
Add to top level gradle.properties
(don’t check into github!)
Add to library module gradle.properties
Add to top level build.gradle
Add Gradle Maven Plugin & Script
https://github.com/NYTimes/Store/blob/develop/gradle/maven-push.gradle
Run UploadArchives task
Go To Nexus Staging Repo to Verify
https://oss.sonatype.org/index.html#stagingRepositories
Close staging Repo and Deploy to Maven Central
Update jira with successful
first upload
Enable Travis
Add Travis.yml
Add Contributing.md
Submit a talk to GDG
Thanks for listening

Open sourcing the store

Editor's Notes

  • #3 First part of the talk will be a quick primer in a new library that we released at NY Times Second part will be how to open source a library yourself.
  • #4 If we ever fetched data when online, we should be able
  • #5 Android Framework give us components like Activities, Services, Adapters and Widgets Open Source Libraries try to fill the gap Retrofit - Defacto standard for networking RxJava - Threading and functional programming Dagger - Dependecy Injection and Code Organization Butterknife - Boilerplate reduction
  • #6 Android Framework give us components like Activities, Services, Adapters and Widgets Open Source Libraries try to fill the gap Retrofit - Defacto standard for networking RxJava - Threading and functional programming Dagger - Dependecy Injection and Code Organization Butterknife - Boilerplate reduction
  • #7 Github link so that you all can check it out
  • #8 Store’s follow the repository pattern and try to abstract working with data into a few public methods. You ask a store for data and it gets it from either local storage or network. If data needs to be parsed, the store takes care of that as well.
  • #9 Store’s follow the repository pattern and try to abstract working with data into a few public methods. You ask a store for data and it gets it from either local storage or network. If data needs to be parsed, the store takes care of that as well.
  • #10 Store’s follow the repository pattern and try to abstract working with data into a few public methods. You ask a store for data and it gets it from either local storage or network. If data needs to be parsed, the store takes care of that as well.
  • #11 Each store is a representation of a single data call
  • #12 Each store is a representation of a single data call
  • #13 A fetcher will tell your Store how to load data. Think of a store as a wrapper for a particular endpoint you want to superpower. The nice part of using Stores is that it will work with both observable and non observable data sources
  • #14 A fetcher will tell your Store how to load data. Think of a store as a wrapper for a particular endpoint you want to superpower. The nice part of using Stores is that it will work with both observable and non observable data sources
  • #15 Stores use barcode to identify data, we will first create a barcode and then pass it to the store. When we subscribe to the store, it will call the fetcher and return to use an observable of the data
  • #16 Uni Direct
  • #18 Besides fetch you can also call get method on store. Calling get will return cached data if it is available rather than hitting your fetcher
  • #19 Walk user through new data flow
  • #21 Data usually comes in a format different from what UI expects. Stores have the ability to have a parser as well. Talk about the different builder & extra generic
  • #24 The parsing step looks generic maybe we can do something about it? We created some parsers that are shared between different stores. Here you see an example of a GsonSourceParser which takes an input of a buffered source and outputs a POJO. We also included GsonStringParser and GsonStreamParser
  • #25 The parsing step looks generic maybe we can do something about it? We created some parsers that are shared between different stores. Here you see an example of a GsonSourceParser which takes an input of a buffered source and outputs a POJO. We also included GsonStringParser and GsonStreamParser
  • #26 Just like a Fetcher and Persister, You can implement a Perisister.
  • #28 Don’t want to implement a persister? You don’t have to as long as you start with a BufferedSource.