Volkan Esgel from Turkcell developed a personal cloud storage service to replace their legacy solution. The new solution uses distributed technologies like OpenStack, Spring Framework, Elasticsearch and RabbitMQ. It provides a scalable and extensible platform for storage and syncing files across devices. Key components include using OpenStack services like Keystone for authentication and Swift for storage, with custom middleware to handle notifications, security and client synchronization.

1. Building a Personal Cloud
Storage Service
Volkan Esgel
Turkcell
August 24, 2016

2. 2
About Me
Volkan Esgel
Senior Software Engineer
TURKCELL

3. About Turkcell
3
* http://investor.turkcell.com.tr/2015/turkcell-group
Integrated communication
and technology services
player in Turkey
Turkcell Global
9 countries
68,9 million total subscribers
1,5 million fixed
subscribers 600 thousand TV subscribers

4. Introduction
Legacy Solution  Current Solution

5. Legacy Solution
5
Adding features was
costly (time & budget) User Experience
not good
No Folder Structure
only tagging
Security Issues

6. Current Solution
6
Distributed
Fast
Fault Tolerant
Highly Scalable
Extendable
New features can be
added easily

7. Technologies
7
Spring Framework
OpenStack
Keystone & SWIFT
ElasticSearch
RabbitMQ
Oracle DB
ImageMagick
FFmpeg

8. 8

9. 9
OpenStack Projects

10. 10
Main Projects
Business OpenStack

11. 11
* Keystone v2.0 (with OS-KSADM extension) / SWIFT v1
Account (Project / Tenant)
User
Container(s)
Main Extended

12. 12
Containers
Main Container
• Main Storage
• UUID as filename
Extended Container
• Thumbnail
• Video Preview
• Profile Photo

13. 13
Uploading a File
Client
Oracle DB
Transcoding
ImageMagick
FFmpeg
SWIFT

14. Temporary URL
14
Adding X-Auth-Token to the
request header
not possible for all cases
Temporary URL
from security perspective
Our Usage Cases

15. 15
Our Usage
33  OpenStack Servers
3.3 PB  Storage Space
6 M  Daily File Upload
1.6 B  Total Files

16. OpenStack Middleware
Custom middleware modules

17. Keystone
17
Business & OpenStack
must be use the same
authentication token
Several authentication
methods
Turkcell Auth, Mobile Network Auth,
Remember Me, etc.
Authentication methods
should be
easily extensible
Solution for these cases
not easy
on Keystone Side
Custom Keystone
Middleware
authenticates via
RESTful API

18. Keystone
18
No need to access
Keystone
from Internet
Call
Keystone Auth API
from Business API

19. Authentication
19
API
- BUSINESS -
Keystone
- OPENSTACK -
Authenticate User
- BUSINESS -
Token Cache
- BUSINESS -
USER
- CLIENT -

20. SWIFT
20
Client Sync Middleware
Notification Middleware
Security Middleware

21. SWIFT – Notification Middleware
21
Notify BACKEND
about file uploads
No failure
any uploaded file
yet
Transfer notifications
over RabbitMQ
Python Kombu
Get custom params
using X-Object-Meta-*
headers
Only for
Main Container

22. SWIFT – Security Middleware
22
Open Internet
MAIN
Only GET requests
are allowed
EXTENDED
Define IP Blocks of
Internal Servers
in conf file
Reject invalid
PUT requests
( X-Object-Meta-File-Name header required )
Allow only
OBJECT operations
Block
ACCOUNT & CONTAINER
operations
Only PUT, GET & OPTIONS
requests are allowed

23. SWIFT – Client Sync Middleware
23
PUT
X-Meta-Strategy:
0  Check for conflict
1  Override existing object
X-Meta-Recent-Server-Hash:
Known ETag value of object on the server
X-Meta-Recent-Server-Hash & ETag :
equals  no conflict, allow PUT request and update existing one
not equals  conflict, return bad response with status
USER
- CLIENT -

24. File System
Advantages of using custom filesystem on DB instead of
SWIFT Object Paths & Container Listing

25. File System
25
All Objects
located under root path of
the container
Object Names
UUIDs
Display Name
Metadata Header
File Listings
Oracle DB
Photo & Video Listing
ElasticSearch Unified (Metadata) Search
ElasticSearch

26. Difficulties of SWIFT File System
26
File Statistics
user and/or content based file
statistics
New Features
adding new features to
the filesystem
Pseudo Folder (Virtual)
renaming a folder requires
copying all sub-objects and
deleting old files – costly
Dropbox & Google Drive

27. Conclusion
27
Developed a Personal Cloud
Storage Service
just in 6 months
from scratch
No critical security
issue is found
tested multiple times by the internal &
independent security organizations
No vendor lock-in
hardware / software
Highly
Scalable & Extendable

28. Demonstration
Final Product

29. 29

30. 30

31. 31

32. 32

33. 33
?
https://akillidepo.turkcell.com.tr
Turkcell Akıllı Depo