Your SlideShare is downloading. ×
YALMS: A Group Drive API for Cloud-Based Classrooms
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

YALMS: A Group Drive API for Cloud-Based Classrooms

392

Published on

Although there are many cloud drive APIs and applications (clients mostly) today, they are not optimized for group activities in cloud-based classrooms. Specifically, such APIs are weak in face of …

Although there are many cloud drive APIs and applications (clients mostly) today, they are not optimized for group activities in cloud-based classrooms. Specifically, such APIs are weak in face of realtime interactivity and large data bulks. YALMS is a newly developed open source educational package which in this presentation is augmented with a group drive functionality. This paper poses the optimization problem involved in a group drive in clouds and presents implementations of both server and client sides of the respective API.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
392
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. . . YALMS Basics M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 2 /26 2/26
  • 2. . . YALMS Basics . YALMS stands for ... . nother ecture . et Y A • L Management System next generation digital classroom 02 robots • several : video annotation, content packing, multimedia storage, etc. • can run in clouds and cloud-based classrooms ◦ specifically, in VM-per-student environments • knowledge extraction and social components 02 "Homepage of the YALMS software presented in this paper" https://github.com/maratishe/yalms (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 3 /26 3/26
  • 3. . . YALMS : The Big Picture 2013.07・ET研 YALMS: Intro 2013.11・IN研 YALMS.cloud VM sync groupdrive content 2013.09・ET研 YALMS.annotation PDF video 2014.01・LOIS研 slices YALMS.format 2014.01・ET研 stream record replay storage YALMS.streaming DASH Adaptive HTML5 2013.10・ITE-MMS研 YALMS.hiperf SSD multicore YALMS.done! M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 4 /26 4/26
  • 4. . . YALMS.groupdrive : The Specs M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 5 /26 5/26
  • 5. . . YALMS.groupdrive : Overall Idea Master Copy Slave Copy M.Zhanikeev, H.Koide -- maratishe@gmail.com -- Slave Copy Slave Copy YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 6 /26 6/26
  • 6. . . YALMS.groupdrive : The Specs 1. realtime async. asymm. P2P sync -- but not distributed storage fully distributed, can work without a master 3. smart and adaptive, fit for any environment 2. ◦ remote master, fully distributed, throughput-limited, clouds, etc. 4. efficient with 5. API with large files -- a sub-file data grain cross-platform implementation ◦ Linux, Windows, VMs M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 7 /26 7/26
  • 7. . . YALMS.groupdrive : Existing (bad) Solutions M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 8 /26 8/26
  • 8. . . Bad Solution : BitTorrent bad solution needs central registry -- Tracker good only with many peers, too much overhead with few ◦ rarest-first policy is not fit for the problem good for download, but what about upload? • a very • • • M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 9 /26 9/26
  • 9. . . Bad Solution : DSync • a paper in 05 -- the closest rival • aggregation from multiple sources while optimizing load • not really distributed -• single point prospective -- o2m or o2o no practical considerations -- no measurement, ordered lists, etc. 05 H.Pucha+3 "Adaptive File Transfers for Diverse Environments" USENIX Annual Technical Conference (2008) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 10 /26 10/26
  • 10. . . Bad Solution : P2P Streaming … Share video streaming 04 • good for 06 • again, bad for bi-directional Content Provider (origin) syncs P2P Network 04 myself+0 "Multi-Source Stream Aggregation in the Cloud" Wiley Book on Content Delivery (ACDN) (2013) 06 myself+0 "Extremely Scalable and Low Demand Live P2P Streaming based on Variable Bitrate" CANDAR (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 11 /26 11/26
  • 11. . . Bad Solution : Group Virtualization Service traffic Service Application • specific to remote services 07 Service Provider Virtual Wireless User Wireless User Wireless User Wireless User • does not work fully local syncs not specific to well with • syncs 07 myself "Virtual Wireless User: A Practical Design for Parallel MultiConnect Using WiFi Direct..." Mobiquitous (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 12 /26 12/26
  • 12. . . Proposal and Design M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 13 /26 13/26
  • 13. . . Proposal : Fundamentals . Objective . + . API Implementation of the proposed group drive • standard RESTful API • implemented in PHP • cross-platform e2e throughput -- response time avoid congesting the master • adaptive to • M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 14 /26 14/26
  • 14. . . Design : PUSH/PULL Basics • simple: 1 PULL and . The PULL . • regularly X PUSH rounds, X peers in each round -- X is scale ask peers for changes, new content comes in replies • do not push the newly recieved content . . The PUSH . • push to for quick distribution . . Conflicts . ... ignored for now, treat like in GoogleDrive, etc. X peers in X rounds M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 15 /26 15/26
  • 15. . . Design : The MFFxR Unit • basic • • list unit for PUSHes and PULLs -- ordered by decreasing throughput Random Peer is basically for measurement/discovery scale is a global parameter Self List:M M.Zhanikeev, H.Koide -- maratishe@gmail.com -- List:MF List:MFF List:MFF+R YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 16 /26 16/26
  • 16. . . Design : Peer Discovery Roadmap • dynamically incorporate new peers Master REP( peers) REQ() M.Zhanikeev, H.Koide -- maratishe@gmail.com -- Update peers Done YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 17 /26 17/26
  • 17. . . Design : PULL Roadmap • a roadmap rather than state machine Error, failure, timeout REP( null) REP( block, epoch) REQ( state) Scale (s) Align peers M.Zhanikeev, H.Koide -- maratishe@gmail.com -- MFFF… +R Notify Replace (if new block) Done YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 18 /26 18/26
  • 18. . . Design : PUSH Roadmap REP( OK) REP( Collision) REQ(block) Scale (s) Align peers MFFF… +R Pick next from MFFF… for 2nd round I already have it! (double up) Done M.Zhanikeev, H.Koide -- maratishe@gmail.com -- Notify If M or R YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 19 /26 19/26
  • 19. . . Experiments M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 20 /26 20/26
  • 20. . . Experiments Traditional (10 clients) Traditional (1 client) YALMS.groupdrive Remote Master (10 peers and scale 2) YALMS.groupdrive Local Only (10 peers and scale 2) 1 • has the file • try various environments 0.8 Peer completion ratio 10Gb file initial state: only master • one 0.6 0.4 0.2 0 0 M.Zhanikeev, H.Koide -- maratishe@gmail.com -- 0.5 1 1.5 2 2.5 log( time) 3 3.5 YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 4 4.5 21 /26 21/26
  • 21. . . Implementation and API M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 22 /26 22/26
  • 22. . . Implementation : Basic Unit of Info. Exchange • standard unit of data exchange -- JSON -- part of API 03 • some parts are not used -- blob only used for PUSHes/PULLs Datatype Parameter Comments name string relpath Relative to top folder string filename long blockpos int blocksize double mtime string abspath blob content Position of the head byte of the block in file Fixed for each session Epoch time of last update for that block at Master Path for this block at Master. Not used by Slave Copies. Blob: binary string. Used when needed. 03 "Homepage of the YALMS.groupdrive project" https://github.com/maratishe/yalms.groupdrive (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 23 /26 23/26
  • 23. . . Implementation : Actual API MFFxR is outside of the API (part of implementation) state is a flat JSON of items (keys) and MD5 digests (values) 03 APIs are standard RESTfull HTTP request/reply routines • per-peer, • • API1: pull(state) − −reply(dataunit) • API2: push(dataunit) − −reply(status) • 03 "Homepage of the YALMS.groupdrive project" https://github.com/maratishe/yalms.groupdrive (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 24 /26 24/26
  • 24. . . Implementation : Software Design Linux but will work on Windows using portable TinyWeb 03 Web Server Web Server OS, System Reps Reqs OS, System • prefers wget wget Reps Reqs 03 "Homepage of the YALMS.groupdrive project" https://github.com/maratishe/yalms.groupdrive (2013) M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 25 /26 25/26
  • 25. . . That’s all, thank you ... M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 26 /26 26/26
  • 26. . . [01] (2013) The enPiT Project http://www.enpit.jp [02] (2013) Homepage of the YALMS software presented in this paper https://github.com/maratishe/yalms [03] (2013) Homepage of the YALMS.groupdrive project https://github.com/maratishe/yalms.groupdrive [04] myself+0 (2013) Multi-Source Stream Aggregation in the Cloud Wiley Book on Content Delivery (ACDN) [05] H.Pucha+3 (2008) Adaptive File Transfers for Diverse Environments USENIX Annual Technical Conference M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 26 /26 26/26
  • 27. . . [06] myself+0 (2013) Extremely Scalable and Low Demand Live P2P Streaming based on Variable Bitrate CANDAR [07] myself (2013) Virtual Wireless User: A Practical Design for Parallel MultiConnect Using WiFi Direct... Mobiquitous [08] (2013) Dropbox Core API https://www.dropbox.com/developers/core M.Zhanikeev, H.Koide -- maratishe@gmail.com -- YALMS: A Group Drive API for Cloud-Based Classrooms -- http://tinyurl.com/kyutech131121 --- 26 /26 26/26

×