SlideShare a Scribd company logo
Stashaway
Fixing digital media organization
What problems are we solving?
● Devices are getting smaller
  ○ local storage is becoming expensive
● Cloud Storage is getting popular
  ○ remote storage is becoming cheaper
● Data is moving from Local to Remote
● "Sync paradigm"
  ○ Dropbox model, git model
  ○ is great for mutable data (documents)
  ○ Works best with hierarchical file structures
● "Send paradigm"
  ○ physical disk model
  ○ is better for immutable data (media)
  ○ Works best with metadata-based filesystems
Sync paradigm sucks for mutable
data!
● Sync is everywhere
   ○ Photostream
   ○ Duplicates everywhere!
   ○ Organizational nightmare
● Sync for media is terrible!
   ○ Google music and Itunes
● stashaway is bringing send paradigm back!
● stashaway is for mutable data
   ○ Videos, Pictures, music, archives, etc.
● stashaway strongly encourages workflows
  that favor move/delete
   ○ discourages copy
stashaway in detail
● brew install stashaway
● stashaway setup
  ○ Makes a ~/.stashaway file.
  ○ Takes API keys, and preferences.
● stashaway upload file.avi
  ○ sends file to the remote file storage
  ○ resume when connection is interrupted
  ○ deletes file locally
● stashaway list
  ○ List all files in all remote stores uploaded with
    stashaway
● stashaway stream file
  ○ streams file from remote store for viewing/listening
Additional features
(to be added later)
●   file tagging
●   automatic duplicate detection
●   GUI
●   extensible storage backends
●   other bells and whistles
Development Timeline
● Phase one - Research and Experimentation
  ○   Been going on since 2011
  ○   look at the problem from many perspectives
  ○   Everything is up in the air
  ○   Contains many false starts
● Phase two - Breaking ground
  ○ Will be fast, possibly one week
  ○ Do everything right, low technical debt
● Phase three - Beta and beyond
  ○ Launch is unceremonious
  ○ Iterations happen at the speed of phase two
     ■ quick phase two == fast iteration == agile
     ■ slow phase two == slow iteration == expensive
Phase One status
● Language
   ○   Compiled -> C languages, haskell
   ○   Interpreted -> Python (restricted to STDlib)
   ○   Distribution will be difficult if using python
   ○   Current Winner -> C
● Hard stuff
   ○   Amazon API signing
   ○   SSL / encryption
   ○   Reacquaintance with C language/ecosystem
   ○   Distribution (installer/homebrew/apt-get integration)
● Potentially Hard stuff
   ○ Dropbox
● README Driven Approach
Upcoming videos
● Video 2
   ○ Getting C up and running
   ○ More small refinements to the README
● Still to come:
   ○   Easier Distribution using Python
   ○   Homebrew / apt-get bureaucracy
   ○   Refinements to the README
   ○   feedback
● Production details
   ○ github: https://github.com/priestc/stashaway
   ○ Next video comes when I have enough content to
     make it into a video
   ○ If you want to contribute, please do so!

More Related Content

What's hot

From NodeJS to Rust
From NodeJS to RustFrom NodeJS to Rust
From NodeJS to Rust
Bastian Gruber
 
A Shallow Survey of Alternative Languages on the JVM
A Shallow Survey of Alternative Languages on the JVMA Shallow Survey of Alternative Languages on the JVM
A Shallow Survey of Alternative Languages on the JVM
Charles Anderson
 
Golang
GolangGolang
Intro to GO (Bangkok Launchpad 2014)
Intro to GO (Bangkok Launchpad 2014)Intro to GO (Bangkok Launchpad 2014)
Intro to GO (Bangkok Launchpad 2014)
Matthew Campbell
 
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Artur Trzop
 
Chromium OS Introduction
Chromium OS IntroductionChromium OS Introduction
Chromium OS Introduction
Wei-Ning Huang
 
openSUSE tools on Debian
openSUSE tools on DebianopenSUSE tools on Debian
openSUSE tools on Debian
Hideki Yamane
 
A Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to GoA Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to Go
Matt Stine
 
Hello android
Hello androidHello android
Hello android
DENNIS JUNG
 
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
Yiran Wang
 
10 reasons to be excited about go
10 reasons to be excited about go10 reasons to be excited about go
10 reasons to be excited about go
Dvir Volk
 
Command line git
Command line gitCommand line git
Command line git
Manos Emmanouilidis
 
Messing with binary formats
Messing with binary formatsMessing with binary formats
Messing with binary formats
Ange Albertini
 
Oshw documentation project video 1 - lars zimmermann
Oshw documentation project   video 1 - lars zimmermannOshw documentation project   video 1 - lars zimmermann
Oshw documentation project video 1 - lars zimmermann
Lars Zimmermann
 
A tale of two(many) proxies
A tale of two(many) proxiesA tale of two(many) proxies
A tale of two(many) proxies
Mohan Dutt
 

What's hot (15)

From NodeJS to Rust
From NodeJS to RustFrom NodeJS to Rust
From NodeJS to Rust
 
A Shallow Survey of Alternative Languages on the JVM
A Shallow Survey of Alternative Languages on the JVMA Shallow Survey of Alternative Languages on the JVM
A Shallow Survey of Alternative Languages on the JVM
 
Golang
GolangGolang
Golang
 
Intro to GO (Bangkok Launchpad 2014)
Intro to GO (Bangkok Launchpad 2014)Intro to GO (Bangkok Launchpad 2014)
Intro to GO (Bangkok Launchpad 2014)
 
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
 
Chromium OS Introduction
Chromium OS IntroductionChromium OS Introduction
Chromium OS Introduction
 
openSUSE tools on Debian
openSUSE tools on DebianopenSUSE tools on Debian
openSUSE tools on Debian
 
A Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to GoA Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to Go
 
Hello android
Hello androidHello android
Hello android
 
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
KubeCon EU 2019 - P2P Docker Image Distribution in Hybrid Cloud Environment w...
 
10 reasons to be excited about go
10 reasons to be excited about go10 reasons to be excited about go
10 reasons to be excited about go
 
Command line git
Command line gitCommand line git
Command line git
 
Messing with binary formats
Messing with binary formatsMessing with binary formats
Messing with binary formats
 
Oshw documentation project video 1 - lars zimmermann
Oshw documentation project   video 1 - lars zimmermannOshw documentation project   video 1 - lars zimmermann
Oshw documentation project video 1 - lars zimmermann
 
A tale of two(many) proxies
A tale of two(many) proxiesA tale of two(many) proxies
A tale of two(many) proxies
 

Similar to Stashaway 1

Strategies for developing and deploying your embedded applications and images
Strategies for developing and deploying your embedded applications and imagesStrategies for developing and deploying your embedded applications and images
Strategies for developing and deploying your embedded applications and images
Mender.io
 
Hles 2021 Digital transformation - How to use digital tools to improve our ev...
Hles 2021 Digital transformation - How to use digital tools to improve our ev...Hles 2021 Digital transformation - How to use digital tools to improve our ev...
Hles 2021 Digital transformation - How to use digital tools to improve our ev...
Henning Spjelkavik
 
Customer Highlight: Craftsy
Customer Highlight: CraftsyCustomer Highlight: Craftsy
Customer Highlight: Craftsy
Amazon Web Services
 
Pipenv - The Python Companion You Wish You Always Had
Pipenv - The Python Companion You Wish You Always HadPipenv - The Python Companion You Wish You Always Had
Pipenv - The Python Companion You Wish You Always Had
Avi Aminov
 
Continuous Deployment Applied at MyHeritage
Continuous Deployment Applied at MyHeritageContinuous Deployment Applied at MyHeritage
Continuous Deployment Applied at MyHeritage
Ran Levy
 
Go frugal with web services
Go frugal with web servicesGo frugal with web services
Go frugal with web services
Daniel Fireman
 
Boosting I/O Performance with KVM io_uring
Boosting I/O Performance with KVM io_uringBoosting I/O Performance with KVM io_uring
Boosting I/O Performance with KVM io_uring
ShapeBlue
 
Improve the deployment process step by step
Improve the deployment process step by stepImprove the deployment process step by step
Improve the deployment process step by step
Daniel Fahlke
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapRuslan Meshenberg
 
PyConFR 2014 - DEPOT, Story of a file.write() gone wrong
PyConFR 2014 - DEPOT, Story of a file.write() gone wrongPyConFR 2014 - DEPOT, Story of a file.write() gone wrong
PyConFR 2014 - DEPOT, Story of a file.write() gone wrong
Alessandro Molina
 
Pen Testing Development
Pen Testing DevelopmentPen Testing Development
Pen Testing Development
CTruncer
 
Do You Get Git?
Do You Get Git? Do You Get Git?
Do You Get Git?
Prasid Pathak
 
Spark and S3 with Ryan Blue
Spark and S3 with Ryan BlueSpark and S3 with Ryan Blue
Spark and S3 with Ryan Blue
Databricks
 
AWS Customer Highlight - Craftsy
AWS Customer Highlight - CraftsyAWS Customer Highlight - Craftsy
AWS Customer Highlight - Craftsy
Amazon Web Services
 
Schizophrenic files v2
Schizophrenic files v2Schizophrenic files v2
Schizophrenic files v2
Ange Albertini
 
Hadoop Meetup Jan 2019 - Overview of Ozone
Hadoop Meetup Jan 2019 - Overview of OzoneHadoop Meetup Jan 2019 - Overview of Ozone
Hadoop Meetup Jan 2019 - Overview of Ozone
Erik Krogen
 
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
 
LAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMGLAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMG
Linaro
 
The DuraCloud Workshop - Open Repositories 2015
The DuraCloud Workshop - Open Repositories 2015The DuraCloud Workshop - Open Repositories 2015
The DuraCloud Workshop - Open Repositories 2015
DuraSpace
 
The history of Prometheus at SoundCloud
The history of Prometheus at SoundCloudThe history of Prometheus at SoundCloud
The history of Prometheus at SoundCloud
Tobias Schmidt
 

Similar to Stashaway 1 (20)

Strategies for developing and deploying your embedded applications and images
Strategies for developing and deploying your embedded applications and imagesStrategies for developing and deploying your embedded applications and images
Strategies for developing and deploying your embedded applications and images
 
Hles 2021 Digital transformation - How to use digital tools to improve our ev...
Hles 2021 Digital transformation - How to use digital tools to improve our ev...Hles 2021 Digital transformation - How to use digital tools to improve our ev...
Hles 2021 Digital transformation - How to use digital tools to improve our ev...
 
Customer Highlight: Craftsy
Customer Highlight: CraftsyCustomer Highlight: Craftsy
Customer Highlight: Craftsy
 
Pipenv - The Python Companion You Wish You Always Had
Pipenv - The Python Companion You Wish You Always HadPipenv - The Python Companion You Wish You Always Had
Pipenv - The Python Companion You Wish You Always Had
 
Continuous Deployment Applied at MyHeritage
Continuous Deployment Applied at MyHeritageContinuous Deployment Applied at MyHeritage
Continuous Deployment Applied at MyHeritage
 
Go frugal with web services
Go frugal with web servicesGo frugal with web services
Go frugal with web services
 
Boosting I/O Performance with KVM io_uring
Boosting I/O Performance with KVM io_uringBoosting I/O Performance with KVM io_uring
Boosting I/O Performance with KVM io_uring
 
Improve the deployment process step by step
Improve the deployment process step by stepImprove the deployment process step by step
Improve the deployment process step by step
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmap
 
PyConFR 2014 - DEPOT, Story of a file.write() gone wrong
PyConFR 2014 - DEPOT, Story of a file.write() gone wrongPyConFR 2014 - DEPOT, Story of a file.write() gone wrong
PyConFR 2014 - DEPOT, Story of a file.write() gone wrong
 
Pen Testing Development
Pen Testing DevelopmentPen Testing Development
Pen Testing Development
 
Do You Get Git?
Do You Get Git? Do You Get Git?
Do You Get Git?
 
Spark and S3 with Ryan Blue
Spark and S3 with Ryan BlueSpark and S3 with Ryan Blue
Spark and S3 with Ryan Blue
 
AWS Customer Highlight - Craftsy
AWS Customer Highlight - CraftsyAWS Customer Highlight - Craftsy
AWS Customer Highlight - Craftsy
 
Schizophrenic files v2
Schizophrenic files v2Schizophrenic files v2
Schizophrenic files v2
 
Hadoop Meetup Jan 2019 - Overview of Ozone
Hadoop Meetup Jan 2019 - Overview of OzoneHadoop Meetup Jan 2019 - Overview of Ozone
Hadoop Meetup Jan 2019 - Overview of Ozone
 
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
 
LAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMGLAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMG
 
The DuraCloud Workshop - Open Repositories 2015
The DuraCloud Workshop - Open Repositories 2015The DuraCloud Workshop - Open Repositories 2015
The DuraCloud Workshop - Open Repositories 2015
 
The history of Prometheus at SoundCloud
The history of Prometheus at SoundCloudThe history of Prometheus at SoundCloud
The history of Prometheus at SoundCloud
 

Stashaway 1

  • 2. What problems are we solving? ● Devices are getting smaller ○ local storage is becoming expensive ● Cloud Storage is getting popular ○ remote storage is becoming cheaper ● Data is moving from Local to Remote ● "Sync paradigm" ○ Dropbox model, git model ○ is great for mutable data (documents) ○ Works best with hierarchical file structures ● "Send paradigm" ○ physical disk model ○ is better for immutable data (media) ○ Works best with metadata-based filesystems
  • 3. Sync paradigm sucks for mutable data! ● Sync is everywhere ○ Photostream ○ Duplicates everywhere! ○ Organizational nightmare ● Sync for media is terrible! ○ Google music and Itunes ● stashaway is bringing send paradigm back! ● stashaway is for mutable data ○ Videos, Pictures, music, archives, etc. ● stashaway strongly encourages workflows that favor move/delete ○ discourages copy
  • 4. stashaway in detail ● brew install stashaway ● stashaway setup ○ Makes a ~/.stashaway file. ○ Takes API keys, and preferences. ● stashaway upload file.avi ○ sends file to the remote file storage ○ resume when connection is interrupted ○ deletes file locally ● stashaway list ○ List all files in all remote stores uploaded with stashaway ● stashaway stream file ○ streams file from remote store for viewing/listening
  • 5. Additional features (to be added later) ● file tagging ● automatic duplicate detection ● GUI ● extensible storage backends ● other bells and whistles
  • 6. Development Timeline ● Phase one - Research and Experimentation ○ Been going on since 2011 ○ look at the problem from many perspectives ○ Everything is up in the air ○ Contains many false starts ● Phase two - Breaking ground ○ Will be fast, possibly one week ○ Do everything right, low technical debt ● Phase three - Beta and beyond ○ Launch is unceremonious ○ Iterations happen at the speed of phase two ■ quick phase two == fast iteration == agile ■ slow phase two == slow iteration == expensive
  • 7. Phase One status ● Language ○ Compiled -> C languages, haskell ○ Interpreted -> Python (restricted to STDlib) ○ Distribution will be difficult if using python ○ Current Winner -> C ● Hard stuff ○ Amazon API signing ○ SSL / encryption ○ Reacquaintance with C language/ecosystem ○ Distribution (installer/homebrew/apt-get integration) ● Potentially Hard stuff ○ Dropbox ● README Driven Approach
  • 8. Upcoming videos ● Video 2 ○ Getting C up and running ○ More small refinements to the README ● Still to come: ○ Easier Distribution using Python ○ Homebrew / apt-get bureaucracy ○ Refinements to the README ○ feedback ● Production details ○ github: https://github.com/priestc/stashaway ○ Next video comes when I have enough content to make it into a video ○ If you want to contribute, please do so!