SlideShare a Scribd company logo
1 of 23
Download to read offline
.
.
Building Bridges
pyfilesystem: unified access to storages and services
Andreas Jung
@MacYET

info@zopyx.com

www.zopyx.com

Python Users Berlin 03/2016
/about
• 20 years in publishing
business since 1995
• Python, Zope, Plone
• XML, PDF, EPUB, 

DOCX, DITA
Agenda
• pyfilesystem
• storages systems
• storage APIs
• unified access to
storages
• not: storage/service
specific higher API
functionality
Publishing solutions require
connectivity to different storages
Our Publishing Universe
Our Publishing Universe
Characteristics of storages
• usually hierarchical model (folders, collections,
files)
• folder operations:
• makedir, renamedir, removedir, exists…
• file operations:
• open, read, write, exists, stat…
Common problem
• every storage/filesystem/service has its own API/SDK 

(or several )
• standard filesystem API
• Dropbox API
• Google Drive API
• Sharepoint: WebDAV, CMIS, REST
• specific code for each storage type
• storages not easily interchangeable
pyfilesystem
• abstraction layer on top of storages,
access through a uniform API
• Python 2/3 compatible
• various filesystem/webservices drivers
• Goal: your code must not know about
the underlaying storage system. The
backend is just a configuration option.
• extensible (writing a new driver is
straight forward
• sandboxed filesystem operations
• OOTB support for: WebDAV, S(FTP),
RPCFS, OSFS, S3, ZIP, Memory,
MultiFS, WrapFS
handle = fs.opener(some_url)
with handle.open('foo', 'w') as fp:
fp.write(data)
handle.listdir(dirname)
handle.makedir('foo/bar/test')
handle.removedir('foo/bar/test)
handle.exists(some_filename)
handle.isfile(some_name)
handle.move(src, dst)
handle.copy(src, dst)
….
pyfilesystem concepts
• Sandboxing
• no access outside to resources outside a configured root path
• Paths
• '/' as path separator
• '.' current directory
• '..' parent directory
• Errors
• Identical exceptions across filesystem/storage types
Opening filesystems/storages
from fs.osfs import OSFS
my_f = OSFS('/foo/bar')
from fs.contrib.davfs import DAVFS
dav_fs = DAVFS('http://host:port/webdav', credentials=…)
from fs.opener import opener
ftp_fs, path = opener.parse('ftp://ftp.mozilla.org/pub')
davs_fs, path = opener.parse('http://user:pass@host:port/webdav')
s3_fs, path = opener.parse('s3://mybucket/some/folter')
Multi-filesystem (read-only)
from fs.osfs import OSFS
from fs.multifs import MultiFS
themed_template_fs = MultiFS()
themed_template_fs.addfs('templates', OSFS('templates'))
themed_template_fs.addfs('theme', OSFS('theme'))
-- templates
|-- snippets
| `-- panel.html
|-- index.html
|-- profile.html
`-- base.html
-- theme
|-- snippets
| |-- widget.html
| `-- extra.html
|-- index.html
`-- theme.html
Multi-filesystem (read-only)
from fs.osfs import OSFS
from fs.multifs import MultiFS
themed_template_fs = MultiFS()
themed_template_fs.addfs('templates', OSFS('templates'))
themed_template_fs.addfs('theme', OSFS('theme'))
-- templates
|-- snippets
| `-- panel.html
|-- index.html
|-- profile.html
`-- base.html
-- theme
|-- snippets
| |-- widget.html
| `-- extra.html
|-- index.html
`-- theme.html
|-- snippets
| |-- panel.html
| |-- widget.html
| `-- extra.html
|-- index.html
|-- profile.html
|-- base.html
`-- theme.html
Implementing drivers/

essential methods
open() Opens a file for read/writing
isfile() Check whether the path exists and is a file
isdir() Check whether a path exists and is a directory
listdir() List the contents of a directory
makedir() Create a new directory
remove() Remove an existing file
removedir() Remove an existing directory
rename() Atomically rename a file or directory
getinfo() Return information about the path e.g. size, mtime
Implementing drivers/

non-essential methods
copy() Copy a file to a new location
copydir() Recursively copy a directory to a new location
desc() Return a short descriptive text regarding a path
exists() Check whether a path exists as file or directory
listdirinfo() Get a directory listing along 

with the info dict for each entry
ilistdir() Generator version of the listdir method
ilistdirinfo() Generator version of the listdirinfo method
getpathurl() Get an external URL at which the 

given file can be accessed, if possible
getsyspath() Get a file’s name in the local filesystem, if possible
getmeta() Get the value of a filesystem meta value, if it exists
getmmap() Gets an mmap object for the given resource, if supported
hassyspath() Check if a path maps to a system path 

(recognized by the OS)
haspathurl() Check if a path maps to an external URL
hasmeta() Check if a filesystem meta value exists
move() Move a file to a new location
movedir() Recursively move a directory to a new location
settimes() Sets the accessed and modified times of a path
WebDAV (S)FTP
pyfilesystem
Plone
xmldirector.plonecore
Dropbox
GDriveAWS S3
Local FS
Architecture
OwnCloud

Alfresco

eXistDB

BaseX
Dropbox
Sharepoint Evernote
Facebook Flickr
Yandex
OneDrive
many others
Driver Driver Driver
SMEOtixo DropDav
WebDAV
native

protocols native

protocols
Your setup SaaS setup
Storage/

Web Service
self-hosted
(Privacy)
via external
SaaS Bridge
(limited privacy?)
WebDAV 

(Owncloud, BaseX,

eXist-DB, Alfresco, etc.)
YES YES
Amazon S3 YES YES
Local filesystem YES NO
Dropbox (YES, auth token issues) YES
FTP/SFTP (YES, V1.4) YES
4Shared ADrive Alfresco Amazon Cloud
Amazon S3 Box CloudMe Copy Cubby
Digital Bucket DriveOnWeb Dropbox Dump
Truck Evernote FTP Fabasoft Facebook
FilesAnywhere Flickr GMX.DE Google Drive
HiDrive Huddle LiveDrive Mediencenter
MyDrive OneDrive Online FileFolder
OwnCloud Picasa SugarSync TrendMicro
SafeSync Web.de WebDAV Yandex
NO YES
pyfilesystem driver options
Supported services through 

3rd party services (example)
Further development
(Funding)
• native Dropbox support (via Dropbox SDK)
• NTML authentication support for WebDAV driver
• native Sharepoint/Office 365 support (via
CMIS, REST, WebDAV NTLM, Microsoft Graph)
• anyone interested in funding further drivers?
Conclusions
• the underlaying storage/filesystem is just a
configuration option
• same code will work across different storage types
• unit-test your pyfilesystem-based code against
used storage types
• minor behavioral differences between drivers
• dealing with OAuth (Dropbox, Google Drive)
pyfilesystem
> pip install fs
https://pyfilesystem.readthedocs.org/
https://github.com/revolunet/pyfilesystem

More Related Content

What's hot

Using existing language skillsets to create large-scale, cloud-based analytics
Using existing language skillsets to create large-scale, cloud-based analyticsUsing existing language skillsets to create large-scale, cloud-based analytics
Using existing language skillsets to create large-scale, cloud-based analyticsMicrosoft Tech Community
 
Serialization (Avro, Message Pack, Kryo)
Serialization (Avro, Message Pack, Kryo)Serialization (Avro, Message Pack, Kryo)
Serialization (Avro, Message Pack, Kryo)오석 한
 
8b. Column Oriented Databases Lab
8b. Column Oriented Databases Lab8b. Column Oriented Databases Lab
8b. Column Oriented Databases LabFabio Fumarola
 
Sanjip Shah: Internationalizing and Localizing WordPress Themes
Sanjip Shah: Internationalizing and Localizing  WordPress ThemesSanjip Shah: Internationalizing and Localizing  WordPress Themes
Sanjip Shah: Internationalizing and Localizing WordPress Themeswpnepal
 
Drupal in 30 Minutes
Drupal in 30 MinutesDrupal in 30 Minutes
Drupal in 30 MinutesRobert Carr
 
Web performances : Is It not the right time to (re)consider CMS ?
Web performances : Is It not the right time to (re)consider CMS ?Web performances : Is It not the right time to (re)consider CMS ?
Web performances : Is It not the right time to (re)consider CMS ?Kaliop-slide
 
From XML to eBooks Part 2: The Details
From XML to eBooks Part 2: The DetailsFrom XML to eBooks Part 2: The Details
From XML to eBooks Part 2: The DetailsRichard Hamilton
 
Solr Powered Lucene
Solr Powered LuceneSolr Powered Lucene
Solr Powered LuceneErik Hatcher
 
Decoupled Libraries for PHP
Decoupled Libraries for PHPDecoupled Libraries for PHP
Decoupled Libraries for PHPPaul Jones
 
Consuming RESTful services in PHP
Consuming RESTful services in PHPConsuming RESTful services in PHP
Consuming RESTful services in PHPZoran Jeremic
 
Serialization and performance by Sergey Morenets
Serialization and performance by Sergey MorenetsSerialization and performance by Sergey Morenets
Serialization and performance by Sergey MorenetsAlex Tumanoff
 
DSpace Tutorial : Open Source Digital Library
DSpace Tutorial : Open Source Digital LibraryDSpace Tutorial : Open Source Digital Library
DSpace Tutorial : Open Source Digital Libraryrajivkumarmca
 
What's brewing in the eZ Systems extensions kitchen
What's brewing in the eZ Systems extensions kitchenWhat's brewing in the eZ Systems extensions kitchen
What's brewing in the eZ Systems extensions kitchenPaul Borgermans
 
Ancient To Modern: Upgrading nearly a decade of Plone in public radio
Ancient To Modern: Upgrading nearly a decade of Plone in public radioAncient To Modern: Upgrading nearly a decade of Plone in public radio
Ancient To Modern: Upgrading nearly a decade of Plone in public radioCristopher Ewing
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonIgor Anishchenko
 
OpenNTF Domino API - Overview Introduction
OpenNTF Domino API - Overview IntroductionOpenNTF Domino API - Overview Introduction
OpenNTF Domino API - Overview IntroductionPaul Withers
 
Introduction to MongoDB with PHP
Introduction to MongoDB with PHPIntroduction to MongoDB with PHP
Introduction to MongoDB with PHPfwso
 
Introduction to Drupal (7) Theming
Introduction to Drupal (7) ThemingIntroduction to Drupal (7) Theming
Introduction to Drupal (7) ThemingRobert Carr
 

What's hot (20)

Using existing language skillsets to create large-scale, cloud-based analytics
Using existing language skillsets to create large-scale, cloud-based analyticsUsing existing language skillsets to create large-scale, cloud-based analytics
Using existing language skillsets to create large-scale, cloud-based analytics
 
Serialization (Avro, Message Pack, Kryo)
Serialization (Avro, Message Pack, Kryo)Serialization (Avro, Message Pack, Kryo)
Serialization (Avro, Message Pack, Kryo)
 
8b. Column Oriented Databases Lab
8b. Column Oriented Databases Lab8b. Column Oriented Databases Lab
8b. Column Oriented Databases Lab
 
Sanjip Shah: Internationalizing and Localizing WordPress Themes
Sanjip Shah: Internationalizing and Localizing  WordPress ThemesSanjip Shah: Internationalizing and Localizing  WordPress Themes
Sanjip Shah: Internationalizing and Localizing WordPress Themes
 
Drupal in 30 Minutes
Drupal in 30 MinutesDrupal in 30 Minutes
Drupal in 30 Minutes
 
Web performances : Is It not the right time to (re)consider CMS ?
Web performances : Is It not the right time to (re)consider CMS ?Web performances : Is It not the right time to (re)consider CMS ?
Web performances : Is It not the right time to (re)consider CMS ?
 
From XML to eBooks Part 2: The Details
From XML to eBooks Part 2: The DetailsFrom XML to eBooks Part 2: The Details
From XML to eBooks Part 2: The Details
 
Solr Powered Lucene
Solr Powered LuceneSolr Powered Lucene
Solr Powered Lucene
 
Decoupled Libraries for PHP
Decoupled Libraries for PHPDecoupled Libraries for PHP
Decoupled Libraries for PHP
 
ApacheCon09: Avro
ApacheCon09: AvroApacheCon09: Avro
ApacheCon09: Avro
 
Consuming RESTful services in PHP
Consuming RESTful services in PHPConsuming RESTful services in PHP
Consuming RESTful services in PHP
 
Serialization and performance by Sergey Morenets
Serialization and performance by Sergey MorenetsSerialization and performance by Sergey Morenets
Serialization and performance by Sergey Morenets
 
DSpace Tutorial : Open Source Digital Library
DSpace Tutorial : Open Source Digital LibraryDSpace Tutorial : Open Source Digital Library
DSpace Tutorial : Open Source Digital Library
 
What's brewing in the eZ Systems extensions kitchen
What's brewing in the eZ Systems extensions kitchenWhat's brewing in the eZ Systems extensions kitchen
What's brewing in the eZ Systems extensions kitchen
 
Ancient To Modern: Upgrading nearly a decade of Plone in public radio
Ancient To Modern: Upgrading nearly a decade of Plone in public radioAncient To Modern: Upgrading nearly a decade of Plone in public radio
Ancient To Modern: Upgrading nearly a decade of Plone in public radio
 
Thrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased ComparisonThrift vs Protocol Buffers vs Avro - Biased Comparison
Thrift vs Protocol Buffers vs Avro - Biased Comparison
 
OpenNTF Domino API - Overview Introduction
OpenNTF Domino API - Overview IntroductionOpenNTF Domino API - Overview Introduction
OpenNTF Domino API - Overview Introduction
 
Php
PhpPhp
Php
 
Introduction to MongoDB with PHP
Introduction to MongoDB with PHPIntroduction to MongoDB with PHP
Introduction to MongoDB with PHP
 
Introduction to Drupal (7) Theming
Introduction to Drupal (7) ThemingIntroduction to Drupal (7) Theming
Introduction to Drupal (7) Theming
 

Viewers also liked

State Of Zope Linuxtag 2008
State Of Zope Linuxtag 2008State Of Zope Linuxtag 2008
State Of Zope Linuxtag 2008Andreas Jung
 
Integration of Plone with eXist-db
Integration of Plone with eXist-dbIntegration of Plone with eXist-db
Integration of Plone with eXist-dbAndreas Jung
 
Plone4Universities
Plone4UniversitiesPlone4Universities
Plone4UniversitiesAndreas Jung
 
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...Andreas Jung
 
XML Director - the technical foundation of onkopedia.com
XML Director - the technical foundation of onkopedia.comXML Director - the technical foundation of onkopedia.com
XML Director - the technical foundation of onkopedia.comAndreas Jung
 
Making Py Pi Sux Less Key
Making Py Pi Sux Less KeyMaking Py Pi Sux Less Key
Making Py Pi Sux Less KeyAndreas Jung
 
BRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLBRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLAndreas Jung
 
Plone Integration with eXist-db - Structured Content rocks
Plone Integration with eXist-db - Structured Content rocksPlone Integration with eXist-db - Structured Content rocks
Plone Integration with eXist-db - Structured Content rocksAndreas Jung
 
Pragmatic plone projects
Pragmatic plone projectsPragmatic plone projects
Pragmatic plone projectsAndreas Jung
 
Producing high-quality documents with Plone
Producing high-quality documents with PloneProducing high-quality documents with Plone
Producing high-quality documents with PloneAndreas Jung
 
Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)Andreas Jung
 
Produce & Publish Authoring Environment World Plone Day 2010 - Berlin
Produce & Publish Authoring Environment World Plone Day 2010 - BerlinProduce & Publish Authoring Environment World Plone Day 2010 - Berlin
Produce & Publish Authoring Environment World Plone Day 2010 - BerlinAndreas Jung
 
Building bridges - Plone Conference 2015 Bucharest
Building bridges   - Plone Conference 2015 BucharestBuilding bridges   - Plone Conference 2015 Bucharest
Building bridges - Plone Conference 2015 BucharestAndreas Jung
 
CSS Paged Media - A review of tools and techniques
CSS Paged Media - A review of tools and techniquesCSS Paged Media - A review of tools and techniques
CSS Paged Media - A review of tools and techniquesAndreas Jung
 
Python mongo db-training-europython-2011
Python mongo db-training-europython-2011Python mongo db-training-europython-2011
Python mongo db-training-europython-2011Andreas Jung
 
Why Plone Will Die
Why Plone Will DieWhy Plone Will Die
Why Plone Will DieAndreas Jung
 
Why we love ArangoDB. The hunt for the right NosQL Database
Why we love ArangoDB. The hunt for the right NosQL DatabaseWhy we love ArangoDB. The hunt for the right NosQL Database
Why we love ArangoDB. The hunt for the right NosQL DatabaseAndreas Jung
 
zopyx.plone migration - Plone Hochschultagung 2013
zopyx.plone migration - Plone Hochschultagung 2013zopyx.plone migration - Plone Hochschultagung 2013
zopyx.plone migration - Plone Hochschultagung 2013Andreas Jung
 
Eggs, Buildouts und andere Wunderlichkeiten
Eggs, Buildouts und andere WunderlichkeitenEggs, Buildouts und andere Wunderlichkeiten
Eggs, Buildouts und andere WunderlichkeitenAndreas Jung
 
Dzug Zope Optimized
Dzug Zope OptimizedDzug Zope Optimized
Dzug Zope OptimizedAndreas Jung
 

Viewers also liked (20)

State Of Zope Linuxtag 2008
State Of Zope Linuxtag 2008State Of Zope Linuxtag 2008
State Of Zope Linuxtag 2008
 
Integration of Plone with eXist-db
Integration of Plone with eXist-dbIntegration of Plone with eXist-db
Integration of Plone with eXist-db
 
Plone4Universities
Plone4UniversitiesPlone4Universities
Plone4Universities
 
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...
Onkopedia - Ein medizinisches Leitlinienportal auf dem Weg zu XML-basierten P...
 
XML Director - the technical foundation of onkopedia.com
XML Director - the technical foundation of onkopedia.comXML Director - the technical foundation of onkopedia.com
XML Director - the technical foundation of onkopedia.com
 
Making Py Pi Sux Less Key
Making Py Pi Sux Less KeyMaking Py Pi Sux Less Key
Making Py Pi Sux Less Key
 
BRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQLBRAINREPUBLIC - Powered by no-SQL
BRAINREPUBLIC - Powered by no-SQL
 
Plone Integration with eXist-db - Structured Content rocks
Plone Integration with eXist-db - Structured Content rocksPlone Integration with eXist-db - Structured Content rocks
Plone Integration with eXist-db - Structured Content rocks
 
Pragmatic plone projects
Pragmatic plone projectsPragmatic plone projects
Pragmatic plone projects
 
Producing high-quality documents with Plone
Producing high-quality documents with PloneProducing high-quality documents with Plone
Producing high-quality documents with Plone
 
Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)Produce & Publish Authoring Environment V 2.0 (english version)
Produce & Publish Authoring Environment V 2.0 (english version)
 
Produce & Publish Authoring Environment World Plone Day 2010 - Berlin
Produce & Publish Authoring Environment World Plone Day 2010 - BerlinProduce & Publish Authoring Environment World Plone Day 2010 - Berlin
Produce & Publish Authoring Environment World Plone Day 2010 - Berlin
 
Building bridges - Plone Conference 2015 Bucharest
Building bridges   - Plone Conference 2015 BucharestBuilding bridges   - Plone Conference 2015 Bucharest
Building bridges - Plone Conference 2015 Bucharest
 
CSS Paged Media - A review of tools and techniques
CSS Paged Media - A review of tools and techniquesCSS Paged Media - A review of tools and techniques
CSS Paged Media - A review of tools and techniques
 
Python mongo db-training-europython-2011
Python mongo db-training-europython-2011Python mongo db-training-europython-2011
Python mongo db-training-europython-2011
 
Why Plone Will Die
Why Plone Will DieWhy Plone Will Die
Why Plone Will Die
 
Why we love ArangoDB. The hunt for the right NosQL Database
Why we love ArangoDB. The hunt for the right NosQL DatabaseWhy we love ArangoDB. The hunt for the right NosQL Database
Why we love ArangoDB. The hunt for the right NosQL Database
 
zopyx.plone migration - Plone Hochschultagung 2013
zopyx.plone migration - Plone Hochschultagung 2013zopyx.plone migration - Plone Hochschultagung 2013
zopyx.plone migration - Plone Hochschultagung 2013
 
Eggs, Buildouts und andere Wunderlichkeiten
Eggs, Buildouts und andere WunderlichkeitenEggs, Buildouts und andere Wunderlichkeiten
Eggs, Buildouts und andere Wunderlichkeiten
 
Dzug Zope Optimized
Dzug Zope OptimizedDzug Zope Optimized
Dzug Zope Optimized
 

Similar to Unified Access to Storages

Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117exsuns
 
Javase7 1641812
Javase7 1641812Javase7 1641812
Javase7 1641812Vinay H G
 
Alfresco In An Hour - Document Management, Web Content Management, and Collab...
Alfresco In An Hour - Document Management, Web Content Management, and Collab...Alfresco In An Hour - Document Management, Web Content Management, and Collab...
Alfresco In An Hour - Document Management, Web Content Management, and Collab...Alfresco Software
 
Dspace4 150227090306-conversion-gate01
Dspace4 150227090306-conversion-gate01Dspace4 150227090306-conversion-gate01
Dspace4 150227090306-conversion-gate01walaba06
 
Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023Mark Niebergall
 
Fuse'ing python for rapid development of storage efficient FS
Fuse'ing python for rapid development of storage efficient FSFuse'ing python for rapid development of storage efficient FS
Fuse'ing python for rapid development of storage efficient FSChetan Giridhar
 
Essential Camel Components
Essential Camel ComponentsEssential Camel Components
Essential Camel ComponentsChristian Posta
 
Filesystem Management with Flysystem at PHP UK 2023
Filesystem Management with Flysystem at PHP UK 2023Filesystem Management with Flysystem at PHP UK 2023
Filesystem Management with Flysystem at PHP UK 2023Mark Niebergall
 
20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasaggarrett honeycutt
 
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...J V
 
[DanNotes] XPages - Beyound the Basics
[DanNotes] XPages - Beyound the Basics[DanNotes] XPages - Beyound the Basics
[DanNotes] XPages - Beyound the BasicsUlrich Krause
 
Building File Systems with FUSE
Building File Systems with FUSEBuilding File Systems with FUSE
Building File Systems with FUSEelliando dias
 
Powershell Training
Powershell TrainingPowershell Training
Powershell TrainingFahad Noaman
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesYevgeniy Brikman
 
Docker Tips And Tricks at the Docker Beijing Meetup
Docker Tips And Tricks at the Docker Beijing MeetupDocker Tips And Tricks at the Docker Beijing Meetup
Docker Tips And Tricks at the Docker Beijing MeetupJérôme Petazzoni
 

Similar to Unified Access to Storages (20)

Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117
 
Javase7 1641812
Javase7 1641812Javase7 1641812
Javase7 1641812
 
Alfresco In An Hour - Document Management, Web Content Management, and Collab...
Alfresco In An Hour - Document Management, Web Content Management, and Collab...Alfresco In An Hour - Document Management, Web Content Management, and Collab...
Alfresco In An Hour - Document Management, Web Content Management, and Collab...
 
Dspace4 150227090306-conversion-gate01
Dspace4 150227090306-conversion-gate01Dspace4 150227090306-conversion-gate01
Dspace4 150227090306-conversion-gate01
 
Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023
 
Fuse'ing python for rapid development of storage efficient FS
Fuse'ing python for rapid development of storage efficient FSFuse'ing python for rapid development of storage efficient FS
Fuse'ing python for rapid development of storage efficient FS
 
Essential Camel Components
Essential Camel ComponentsEssential Camel Components
Essential Camel Components
 
Filesystem Management with Flysystem at PHP UK 2023
Filesystem Management with Flysystem at PHP UK 2023Filesystem Management with Flysystem at PHP UK 2023
Filesystem Management with Flysystem at PHP UK 2023
 
20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag
 
Symfony2 revealed
Symfony2 revealedSymfony2 revealed
Symfony2 revealed
 
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...
Deep Dive: Alfresco Core Repository (... embedded in a micro-services style a...
 
[DanNotes] XPages - Beyound the Basics
[DanNotes] XPages - Beyound the Basics[DanNotes] XPages - Beyound the Basics
[DanNotes] XPages - Beyound the Basics
 
Hdfs java api
Hdfs java apiHdfs java api
Hdfs java api
 
AHUG Presentation: Fun with Hadoop File Systems
AHUG Presentation: Fun with Hadoop File SystemsAHUG Presentation: Fun with Hadoop File Systems
AHUG Presentation: Fun with Hadoop File Systems
 
Building File Systems with FUSE
Building File Systems with FUSEBuilding File Systems with FUSE
Building File Systems with FUSE
 
Introduction to Monsoon PHP framework
Introduction to Monsoon PHP frameworkIntroduction to Monsoon PHP framework
Introduction to Monsoon PHP framework
 
Powershell Training
Powershell TrainingPowershell Training
Powershell Training
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
 
Docker Tips And Tricks at the Docker Beijing Meetup
Docker Tips And Tricks at the Docker Beijing MeetupDocker Tips And Tricks at the Docker Beijing Meetup
Docker Tips And Tricks at the Docker Beijing Meetup
 
Fuse technology-2015
Fuse technology-2015Fuse technology-2015
Fuse technology-2015
 

More from Andreas Jung

State of PrintCSS - MarkupUK 2023.pdf
State of PrintCSS - MarkupUK 2023.pdfState of PrintCSS - MarkupUK 2023.pdf
State of PrintCSS - MarkupUK 2023.pdfAndreas Jung
 
Typesense Plone Integration Plone Conference 2022 Namur
Typesense Plone Integration Plone Conference 2022 NamurTypesense Plone Integration Plone Conference 2022 Namur
Typesense Plone Integration Plone Conference 2022 NamurAndreas Jung
 
Onkopedia - Plone Tagung 2020 Dresden
Onkopedia - Plone Tagung 2020 DresdenOnkopedia - Plone Tagung 2020 Dresden
Onkopedia - Plone Tagung 2020 DresdenAndreas Jung
 
PrintCSS W3C workshop at XMLPrague 2020
PrintCSS W3C workshop at XMLPrague 2020PrintCSS W3C workshop at XMLPrague 2020
PrintCSS W3C workshop at XMLPrague 2020Andreas Jung
 
PrintCSS workshop XMLPrague 2020
PrintCSS workshop XMLPrague 2020PrintCSS workshop XMLPrague 2020
PrintCSS workshop XMLPrague 2020Andreas Jung
 
Plone 5.2 migration at University Ghent, Belgium
Plone 5.2 migration at University Ghent, BelgiumPlone 5.2 migration at University Ghent, Belgium
Plone 5.2 migration at University Ghent, BelgiumAndreas Jung
 
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel Onkopedia
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel OnkopediaBack to the future - Plone 5.2 und Python 3 Migration am Beispiel Onkopedia
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel OnkopediaAndreas Jung
 
Plone migrations using plone.restapi
Plone migrations using plone.restapiPlone migrations using plone.restapi
Plone migrations using plone.restapiAndreas Jung
 
Plone Migrationen mit Plone REST API
Plone Migrationen mit Plone REST APIPlone Migrationen mit Plone REST API
Plone Migrationen mit Plone REST APIAndreas Jung
 
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...Andreas Jung
 
Generierung von PDF aus XML/HTML mit PrintCSS
Generierung von PDF aus XML/HTML mit PrintCSSGenerierung von PDF aus XML/HTML mit PrintCSS
Generierung von PDF aus XML/HTML mit PrintCSSAndreas Jung
 
Creating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsCreating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsAndreas Jung
 
Creating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsCreating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsAndreas Jung
 
The Plone and The Blockchain
The Plone and The BlockchainThe Plone and The Blockchain
The Plone and The BlockchainAndreas Jung
 
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCs
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCsContent Gemeinsam Erstellen: Integration Plone mit SMASHDOCs
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCsAndreas Jung
 
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.Andreas Jung
 
Produce & Publish Cloud Edition
Produce & Publish Cloud EditionProduce & Publish Cloud Edition
Produce & Publish Cloud EditionAndreas Jung
 
Onkopedia - ein medizinisches Fachportal auf Basis von Plone
Onkopedia  - ein medizinisches Fachportal auf Basis von PloneOnkopedia  - ein medizinisches Fachportal auf Basis von Plone
Onkopedia - ein medizinisches Fachportal auf Basis von PloneAndreas Jung
 

More from Andreas Jung (18)

State of PrintCSS - MarkupUK 2023.pdf
State of PrintCSS - MarkupUK 2023.pdfState of PrintCSS - MarkupUK 2023.pdf
State of PrintCSS - MarkupUK 2023.pdf
 
Typesense Plone Integration Plone Conference 2022 Namur
Typesense Plone Integration Plone Conference 2022 NamurTypesense Plone Integration Plone Conference 2022 Namur
Typesense Plone Integration Plone Conference 2022 Namur
 
Onkopedia - Plone Tagung 2020 Dresden
Onkopedia - Plone Tagung 2020 DresdenOnkopedia - Plone Tagung 2020 Dresden
Onkopedia - Plone Tagung 2020 Dresden
 
PrintCSS W3C workshop at XMLPrague 2020
PrintCSS W3C workshop at XMLPrague 2020PrintCSS W3C workshop at XMLPrague 2020
PrintCSS W3C workshop at XMLPrague 2020
 
PrintCSS workshop XMLPrague 2020
PrintCSS workshop XMLPrague 2020PrintCSS workshop XMLPrague 2020
PrintCSS workshop XMLPrague 2020
 
Plone 5.2 migration at University Ghent, Belgium
Plone 5.2 migration at University Ghent, BelgiumPlone 5.2 migration at University Ghent, Belgium
Plone 5.2 migration at University Ghent, Belgium
 
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel Onkopedia
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel OnkopediaBack to the future - Plone 5.2 und Python 3 Migration am Beispiel Onkopedia
Back to the future - Plone 5.2 und Python 3 Migration am Beispiel Onkopedia
 
Plone migrations using plone.restapi
Plone migrations using plone.restapiPlone migrations using plone.restapi
Plone migrations using plone.restapi
 
Plone Migrationen mit Plone REST API
Plone Migrationen mit Plone REST APIPlone Migrationen mit Plone REST API
Plone Migrationen mit Plone REST API
 
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...
Plone im Einsatz bei der Universität des Saarländes als Shop-System und Gefah...
 
Generierung von PDF aus XML/HTML mit PrintCSS
Generierung von PDF aus XML/HTML mit PrintCSSGenerierung von PDF aus XML/HTML mit PrintCSS
Generierung von PDF aus XML/HTML mit PrintCSS
 
Creating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsCreating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCs
 
Creating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCsCreating Content Together - Plone Integration with SMASHDOCs
Creating Content Together - Plone Integration with SMASHDOCs
 
The Plone and The Blockchain
The Plone and The BlockchainThe Plone and The Blockchain
The Plone and The Blockchain
 
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCs
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCsContent Gemeinsam Erstellen: Integration Plone mit SMASHDOCs
Content Gemeinsam Erstellen: Integration Plone mit SMASHDOCs
 
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.
PDF Generierung mit XML/HTML und CSS - was die Tools können und was nicht.
 
Produce & Publish Cloud Edition
Produce & Publish Cloud EditionProduce & Publish Cloud Edition
Produce & Publish Cloud Edition
 
Onkopedia - ein medizinisches Fachportal auf Basis von Plone
Onkopedia  - ein medizinisches Fachportal auf Basis von PloneOnkopedia  - ein medizinisches Fachportal auf Basis von Plone
Onkopedia - ein medizinisches Fachportal auf Basis von Plone
 

Recently uploaded

Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfkalichargn70th171
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptxVinzoCenzo
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...kalichargn70th171
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxSasikiranMarri
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonApplitools
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITmanoharjgpsolutions
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxRTS corp
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxAS Design & AST.
 

Recently uploaded (20)

Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptx
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh IT
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptx
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptx
 

Unified Access to Storages

  • 1. . . Building Bridges pyfilesystem: unified access to storages and services Andreas Jung @MacYET
 info@zopyx.com
 www.zopyx.com
 Python Users Berlin 03/2016
  • 2. /about • 20 years in publishing business since 1995 • Python, Zope, Plone • XML, PDF, EPUB, 
 DOCX, DITA
  • 3. Agenda • pyfilesystem • storages systems • storage APIs • unified access to storages • not: storage/service specific higher API functionality
  • 7. Characteristics of storages • usually hierarchical model (folders, collections, files) • folder operations: • makedir, renamedir, removedir, exists… • file operations: • open, read, write, exists, stat…
  • 8. Common problem • every storage/filesystem/service has its own API/SDK 
 (or several ) • standard filesystem API • Dropbox API • Google Drive API • Sharepoint: WebDAV, CMIS, REST • specific code for each storage type • storages not easily interchangeable
  • 9. pyfilesystem • abstraction layer on top of storages, access through a uniform API • Python 2/3 compatible • various filesystem/webservices drivers • Goal: your code must not know about the underlaying storage system. The backend is just a configuration option. • extensible (writing a new driver is straight forward • sandboxed filesystem operations • OOTB support for: WebDAV, S(FTP), RPCFS, OSFS, S3, ZIP, Memory, MultiFS, WrapFS handle = fs.opener(some_url) with handle.open('foo', 'w') as fp: fp.write(data) handle.listdir(dirname) handle.makedir('foo/bar/test') handle.removedir('foo/bar/test) handle.exists(some_filename) handle.isfile(some_name) handle.move(src, dst) handle.copy(src, dst) ….
  • 10. pyfilesystem concepts • Sandboxing • no access outside to resources outside a configured root path • Paths • '/' as path separator • '.' current directory • '..' parent directory • Errors • Identical exceptions across filesystem/storage types
  • 11. Opening filesystems/storages from fs.osfs import OSFS my_f = OSFS('/foo/bar') from fs.contrib.davfs import DAVFS dav_fs = DAVFS('http://host:port/webdav', credentials=…) from fs.opener import opener ftp_fs, path = opener.parse('ftp://ftp.mozilla.org/pub') davs_fs, path = opener.parse('http://user:pass@host:port/webdav') s3_fs, path = opener.parse('s3://mybucket/some/folter')
  • 12. Multi-filesystem (read-only) from fs.osfs import OSFS from fs.multifs import MultiFS themed_template_fs = MultiFS() themed_template_fs.addfs('templates', OSFS('templates')) themed_template_fs.addfs('theme', OSFS('theme')) -- templates |-- snippets | `-- panel.html |-- index.html |-- profile.html `-- base.html -- theme |-- snippets | |-- widget.html | `-- extra.html |-- index.html `-- theme.html
  • 13. Multi-filesystem (read-only) from fs.osfs import OSFS from fs.multifs import MultiFS themed_template_fs = MultiFS() themed_template_fs.addfs('templates', OSFS('templates')) themed_template_fs.addfs('theme', OSFS('theme')) -- templates |-- snippets | `-- panel.html |-- index.html |-- profile.html `-- base.html -- theme |-- snippets | |-- widget.html | `-- extra.html |-- index.html `-- theme.html |-- snippets | |-- panel.html | |-- widget.html | `-- extra.html |-- index.html |-- profile.html |-- base.html `-- theme.html
  • 14. Implementing drivers/
 essential methods open() Opens a file for read/writing isfile() Check whether the path exists and is a file isdir() Check whether a path exists and is a directory listdir() List the contents of a directory makedir() Create a new directory remove() Remove an existing file removedir() Remove an existing directory rename() Atomically rename a file or directory getinfo() Return information about the path e.g. size, mtime
  • 15. Implementing drivers/
 non-essential methods copy() Copy a file to a new location copydir() Recursively copy a directory to a new location desc() Return a short descriptive text regarding a path exists() Check whether a path exists as file or directory listdirinfo() Get a directory listing along 
 with the info dict for each entry ilistdir() Generator version of the listdir method ilistdirinfo() Generator version of the listdirinfo method getpathurl() Get an external URL at which the 
 given file can be accessed, if possible getsyspath() Get a file’s name in the local filesystem, if possible getmeta() Get the value of a filesystem meta value, if it exists getmmap() Gets an mmap object for the given resource, if supported hassyspath() Check if a path maps to a system path 
 (recognized by the OS) haspathurl() Check if a path maps to an external URL hasmeta() Check if a filesystem meta value exists move() Move a file to a new location movedir() Recursively move a directory to a new location settimes() Sets the accessed and modified times of a path
  • 16.
  • 17. WebDAV (S)FTP pyfilesystem Plone xmldirector.plonecore Dropbox GDriveAWS S3 Local FS Architecture OwnCloud
 Alfresco
 eXistDB
 BaseX Dropbox Sharepoint Evernote Facebook Flickr Yandex OneDrive many others Driver Driver Driver SMEOtixo DropDav WebDAV native
 protocols native
 protocols Your setup SaaS setup
  • 18. Storage/
 Web Service self-hosted (Privacy) via external SaaS Bridge (limited privacy?) WebDAV 
 (Owncloud, BaseX,
 eXist-DB, Alfresco, etc.) YES YES Amazon S3 YES YES Local filesystem YES NO Dropbox (YES, auth token issues) YES FTP/SFTP (YES, V1.4) YES 4Shared ADrive Alfresco Amazon Cloud Amazon S3 Box CloudMe Copy Cubby Digital Bucket DriveOnWeb Dropbox Dump Truck Evernote FTP Fabasoft Facebook FilesAnywhere Flickr GMX.DE Google Drive HiDrive Huddle LiveDrive Mediencenter MyDrive OneDrive Online FileFolder OwnCloud Picasa SugarSync TrendMicro SafeSync Web.de WebDAV Yandex NO YES pyfilesystem driver options
  • 19. Supported services through 
 3rd party services (example)
  • 20. Further development (Funding) • native Dropbox support (via Dropbox SDK) • NTML authentication support for WebDAV driver • native Sharepoint/Office 365 support (via CMIS, REST, WebDAV NTLM, Microsoft Graph) • anyone interested in funding further drivers?
  • 21.
  • 22. Conclusions • the underlaying storage/filesystem is just a configuration option • same code will work across different storage types • unit-test your pyfilesystem-based code against used storage types • minor behavioral differences between drivers • dealing with OAuth (Dropbox, Google Drive)
  • 23. pyfilesystem > pip install fs https://pyfilesystem.readthedocs.org/ https://github.com/revolunet/pyfilesystem