SlideShare a Scribd company logo
1 of 52
Download to read offline
TÉCNICO LISBOA
browserCloud.js
A federated community cloud using a P2P overlay
network on top of the Web Platform
Telecommunications and Computer Science Engineering project
David Dias - 65963 - mail@daviddias.me - github.com/diasdavid
TÉCNICO LISBOA
AGENDA
• Motivation
• Related work
• Architecture
• Implementation
• Evaluation
• Conclusions
• Publications and Impact
2
TÉCNICO LISBOA
MOTIVATION
• User generated data has been increasing
exponentially.
• There is a huge demand of large amounts of
computing power.
• There is no really sustainable and economic
solution for the masses.
3
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CHALLENGES
• Access to large untapped computing infrastructure.
• Execute distributed jobs in a heterogeneous
environment, not confined to specific non-universal
platforms.
• Network bottleneck of constant moving data-to
computation vs computation-to-data.
• Lack of capability to quickly adapt to different types of
jobs, as seen in BOINC based systems.
• Centralised Computing platforms have scalability
problems when usage and users number grows.
4
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
OBJECTIVES
5
• Have a decentralised infrastructure, virtually
self-organised
• Enable flexible types of jobs to be executed,
defined in pre-execution time.
• Efficient decentralised lookup system.
• Enable common users to take advantage of
free Computing system.
• Easy volunteer sign up.
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CONTRIBUTIONS
6
• We’ve successfully delivered a:
• Decentralised communication platform for the browser
• WebRTC DHT
• Job Distribution and Agnostic Execution System
• Case Study: Ray Tracing framework
• Validation: Decentralised browser testing framework
• Every component is open source, MIT licensed
• Contributions can be found at http://github.com/diasdavid/,
identified with the INESC ID Lisboa and Técnico Lisboa emblem.
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
RELATED WORK
7
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
RELATED WORK
• Cloud computing and Open Cloud Platforms
• Volunteered resource sharing
• Web platform
8
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CLOUD COMPUTING
• Cloud Computing describes a larger number of computers, connected
through a network, with special differentiating properties[4]:
• Several offers in the Cloud Computing space
• It is an old concept, envisioned around 1961 by Prof. John McCarthy, MIT
• Created the “lock-in” syndrome
9
IaaS Infrastructure as a Service
PaaS Platform as a Service
SaaS Software as a Service
… …
Advantage Cloud Conventional
Data Center
Virtually infinite computing resources Yes No
Freedom from up-front commitment Yes No
“Pay-as-you-go” Yes No
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
VOLUNTEERED 

RESOURCE SHARING
• Definition: Sharing idle computing resources on the network, that
otherwise would be wasted
• Computing Jobs are defined with meta-heuristics, creating a bag-of-
tasks, able to be run in parallel
• Community Clouds - Known also as “bottom up networking”, where
individuals join a community effort to create data-links, community
managed, most known are: guifi.net(over 20000 nodes!) , AWMN and
funkfeur freenet
• CONFINE[34] - Community Cloud federation,

start of the community-lab test bed and

Clommunity, a community network in a box.







10
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CYCLE AND STORAGE
SHARING
• Improve flexibility on the type of jobs you execute
with Gridlets, a unit of workload that packs the
data and the logic necessary to process it
11
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
Type System Architecture Organization
Storage Sharing
Napster
P2P
Kazaa
Freenet
BitTorrent
Cycle Sharing
@HOME projects
(BOINC Based)
Centralized
TÉCNICO LISBOA
PEER-TO-PEER
ARCHITECTURES
12
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
WEB PLATFORM
• Ubiquitous platform
• Highly dynamic runtime for differential updates
and code migration for job execution
• Recent developments have enabled:
• Close to native performance for highly CPU
bound jobs
• P2P Interconnectivity
• Scalable storage and fast indexing
13
Motivation
Challenges
Objectives
Contributions
Related work
CC and Open Cloud
Platforms
Volunteered Resource
Sharing
Web platform
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
Key learnings from the
related work
14
Motivation
Challenges
Objectives
Contributions
Related work
Architecture
Implementation
Evaluation
Conclusions
Publications
Section Learnings
Cloud
Computing
Interoperability is easier to happen when
abstracted
Community
Clouds
Gridlets mitigate the network bottleneck of moving
the data to the computation by moving the
computation to the data (also seen in Manta)
P2P
Efficient storage/lookups requires structured
organization of the data
Web
Platform
Highly dynamic runtime is possible thanks to
Javascript
Near native performance is possible in a browser
P2P in a browser is now possible with WebRTC
TÉCNICO LISBOA
ARCHITECTURE
15
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
16
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
• Membership management - Peers to
join and leave browserCloud.js. A peer
only has the knowledge of a small group
of other peers.
• Message routing - Messages are
routed between peers.
• Job scheduling and results
aggregation - Peers interact between
each other to send tasks and retrieve
the results to the peer executing the job.
• Dynamic runtime support - Job
agnostic.
• Reduced entrance cost to enable
greater adoption - Simple APIs design,
abstracting the complexity in favor of
greater extendability.
• Enable integration and compliance
tests - Automate the process of
verifying browserCloudjs integrity and
functionality.
17
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
Entities:
• browser - The points on our network that
will be able to issue jobs, execute tasks
and route messages.
• Rendezvous point - The only centralised
component in this architecture, its purpose
is for the clients to have a way to connect
to and join the overlay network.
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
Membership Management:
1 - Registration
2 - New peer available - Finger tables
are updated.
3 - Connection establishment
between two peers
18
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
Membership Management:
1 - Registration
2 - New peer available - Finger tables
are updated.
3 - Connection establishment
between two peers
19
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
Membership Management:
1 - Registration
2 - New peer available - Finger tables
are updated.
3 - Connection establishment
between two peers
20
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
DISTRIBUTED
ARCHITECTURE
Message routing:
• Chord routing scheme
• 48 bit unique identifiers
• Flexible finger table
21
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
RESOURCE 

MANAGEMENT
Job workflow:
1 - Divide a Job into smaller units (bag of tasks)
2 - Select how many browsers we want to distribute the job to.
3 - Query the network for available browsers.
4 - Compose the several gridlets with task plus a data partition.
5 - Send each gridlets to the network.
6 - Browsers compute the results and send them back to the
job issuer.
7 - Browser submitting the job gathers all the tasks results and
constructs the job result.


22
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
SOFTWARE STACK
Communication layer:
• Holds the connections with
other peers.
• Performs the necessary
logic, using fingers, for
efficient routing.
• Keeps the peer connected to
the network by updating its
routing table as necessary.
23
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
SOFTWARE STACK
Service router:
• Plugin like module.
• Interface with
communication layer.
• Enables new services to
leverage the P2P browser
infrastructure.
24
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
SOFTWARE STACK
Job Scheduler:
• Find browsers available in
the network.
• Orchestrates the Job over
the network.
• Creates gridlets from data
+ tasks to be executed in
each browser.
25
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING
Testing framework:
1 - A Web Server is started by the Control Center, this endpoint will be serving
the necessary static assets (e,g .html, .css and .js files).
2 + 3 - The number of required browsers for the test being executed, are
spawned.
4 - The Control Center starts sending commands to each browser to execute.
5 - Browsers report to the Control Center which events were triggered.
6 - Control Center assesses the order in which these events happened and
asserts if the behaviour was the expected.
26
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING
Testing framework:
1 - A Web Server is started by the Control Center, this endpoint will be serving
the necessary static assets (e,g .html, .css and .js files).
2 + 3 - The number of required browsers for the test being executed, are
spawned.
4 - The Control Center starts sending commands to each browser to execute.
5 - Browsers report to the Control Center which events were triggered.
6 - Control Center assesses the order in which these events happened and
asserts if the behaviour was the expected.
27
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING
Testing framework:
1 - A Web Server is started by the Control Center, this endpoint will be serving
the necessary static assets (e,g .html, .css and .js files).
2 + 3 - The number of required browsers for the test being executed, are
spawned.
4 - The Control Center starts sending commands to each browser to execute.
5 - Browsers report to the Control Center which events were triggered.
6 - Control Center assesses the order in which these events happened and
asserts if the behaviour was the expected.
28
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING
Testing framework:
1 - A Web Server is started by the Control Center, this endpoint will be serving
the necessary static assets (e,g .html, .css and .js files).
2 + 3 - The number of required browsers for the test being executed, are
spawned.
4 - The Control Center starts sending commands to each browser to execute.
5 - Browsers report to the Control Center which events were triggered.
6 - Control Center assesses the order in which these events happened and
asserts if the behaviour was the expected.
29
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING
Testing framework:
1 - A Web Server is started by the Control Center, this endpoint will be serving
the necessary static assets (e,g .html, .css and .js files).
2 + 3 - The number of required browsers for the test being executed, are
spawned.
4 - The Control Center starts sending commands to each browser to execute.
5 - Browsers report to the Control Center which events were triggered.
6 - Control Center assesses the order in which these events happened and
asserts if the behaviour was the expected.
30
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Distributed Architecture
Resource Management
Software Stack
Testing
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
IMPLEMENTATION
31
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
IMPLEMENTATION
• Browser module
• Signalling Server
• Testing Framework
• Visualizer
• Simulator
• Ray Tracing module
32
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
BROWSER 

MODULE
• Establishes the link between the browsers
• Holds the intelligence for Chord routing
• Capable of distribute jobs over the network
33
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
SIGNALLING 

SERVER
• HTTP + WebSockets Server
• Rendezvous point for browsers to connect
• Has a snapshot of the DHT state
34
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
TESTING 

FRAMEWORK
• Spawns browsers
• Exposes a HTTP server
• Uses WebSockets to interact with the browsers
35
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
VISUALIZER
• Retrieves DHT
information from
Signalling Server
• Represents the DHT in
a graph using D3JS
36
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
SIMULATOR
• Similar to the Visualizer.
• Able to create
customised generations
of a browserCloudjs
network for visualisation.
37
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
RAY TRACING
• Parse a CSS file describing the scene
• Creates individual tasks
• Runs tasks
• Glues the results
• Exports to .png or canvas
38
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Browser module
Signalling Server
Testing Framework
Visualizer
Simulator
Ray Tracing
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
EVALUATION
39
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
QUALITATIVE
ASSESSMENT
✓ Efficient resource discovery through peer-to-peer routing
over a structured overlay network, using a DHT.
✓ Distribute Jobs through the peers available, submitting,
coordinating and aggregating the results.
✓ Remove the need for centralised indexes or points of
control. 

There is still a need of a rendezvous point, however the
data transmitted, computed and stored inside the network
is peer’s responsibility.
✓ Enable every machine equipped with a WebRTC enabled
browser to be part of a browserCloudjs instance.
40
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
QUALITATIVE
ASSESSMENT
✓ Enable peers to both participate and contribute to a job
and at the same time submitting and requesting the
network to process their own.
✓ Enable browsers to be part of one or more browserCloudjs
instance.
✓ browserCloudjs’ Job Scheduler is job agnostic, this means
that different types of jobs can be executed on demand
without any previous configuration or preparation.
✓ browserCloudjs solves the decentralised communication
problem between browsers in a scalable way.

Enable new scenarios to be developed on top of it through
its modular and pluggable approach.
41
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
QUANTITATIVE
ASSESSMENT
• Setup: Core i7 with 16Gb of RAM running Chrome
v39
• Goals
• Measuring the time lapsed for a single browser to
compute a CPU bound job and several browsers to
compute that same job, but in parallel.
• Measuring the RTT time between any of two browsers
in the network and evaluate as routing efficiency
evolves with the increase in number of browsers.
• Assessing if there are significant speedups.
42
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
QUANTITATIVE
ASSESSMENT
Ideal scenario(1):
jobTime = DelayFromResourceDiscovery + timeOfExecutingSlowestTask +
slowestDelayFromResultReply
Using shared resources(2):
jobTime =
︎
DelayFromResourceDiscovery + (timeOfExecuting N Tasks on M Resources)+
︎
DelayFromResultReply
43
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
RTT between 2 browsers in a 10 browser network
TÉCNICO LISBOA
QUANTITATIVE
ASSESSMENT
• Length of running
the Ray Tracing Job
in one machine -
24ms
44
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
QUANTITATIVE
ASSESSMENT
45
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
QUANTITATIVE
ASSESSMENT
• Identified performance bottlenecks
• Logging - Since V8 runs in a single thread, any synchronous operation will
block the event loop and add delay to the whole processing; although these logs
are essential for us to assess the efficiency of the infrastructure, they are not vital
to the job.
• Delay added - One technique we used to simulate the network delay is to
use the ‘set-Timeout’ native function of V8’s JavaScript implementation, however it
does not necessarily guarantee that the function will be executed in X amount of
milliseconds.
• Tasks can not be sent in parallel - A node has to send each individual
computing unit sequentially and independently, meaning that if we divide a job
into 2000 tasks for e.g, each task will have to wait for the previous to be sent.
• We have noticed a considerable machine resources bottle neck when we had
more than 40 data channels present in same machine - This means that is is unviable
to leverage the full 48 fingers space our finger table has the capacity, since one fingers
equals 2 data channels (one in and one out).
46
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Qualitative
Quantitative
Conclusions
Publications
TÉCNICO LISBOA
CONCLUSIONS
47
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CONCLUSIONS
Major Achievements:
• The first browser based DHT - browserCloudjs offers
for the first time in browser history, a fully functional
DHT, performing resource decentralised resource
discovery on the browser.
• The first peer-to-peer browser computing platform
- the research of using browsers to leverage the idle
computer cycles have been in the literature for a while,
however, always following the centralised/BOINC
model. browserCloudjs offers the first peer-2- peer
browser computing framework with proven speedups.
48
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
CONCLUSIONS
Future Work:
• New job strategies - Currently browserCloudjs only supports
mapping jobs, however, there is no practical limitation to execute full
map/reduce jobs and/or streaming functions (for realtime data
scenarios).
• Hybrid peers - peers that live inside of a server that can both act as a
signalling server and stabilising the network when the churn rate is
high.
• Geographic distribution awareness - Select finger tables based in
optimal RTT distribution and geographic positioning of peers.
• Optimisation of the JavaScript code developed - Increasing the
performance of crucial functions.
• Continuous upgrade of browserCloudjs platform as the Web Platform
APIs evolve and as WebRTC moves from the draft state to finished
spec.
49
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
PUBLICATIONS 

AND IMPACT
• Delivered a talk at Data Terra Nemo, P2P Conf, Berlin,
May 23rd & 24th. http://dtn.is/
• Delivered a talk at OpoJS, Oporto, Feb 26th. http://www.opojs.com
• Delivered a talk at require(‘lx’), Lisbon, Jun 3rd.
• Invited to deliver a talk a LNUG in the end of June. http://lnug.org/
• WebRTC Weekly Issue #60 mention, the number one WebRTC
newsletter with more than 1000 subscribers (https://
webrtcweekly.com/issue/webrtc-weekly-issue-60/)
• Number one Top article in EchoJS for 3 days in a row and Top-5 for
7 days. (http://www.echojs.com/news/14009)
• browserCloud.js demo video - Over 220 impressions. (https://
www.youtube.com/watch?v=kjwIjoENCRE).
• Started a collaboration with Karma to join forces on the browser
testing framework https://github.com/diasdavid/piri-piri/issues/6
50
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
TÉCNICO LISBOA
PUBLICATIONS 

AND IMPACT
51
Motivation
Challenges
Objectives
Contributions
Related Work
Architecture
Implementation
Evaluation
Conclusions
Publications
Name Downloads Github Stars
webrtc-explorer 615 63
webrtc-explorer-signalling-server NA 11
webrtc-explorer-visualizer NA 10
webrtc-explorer-simulator NA 9
webrtc-explorer-browser-process NA 8
piri-piri
 444 14
piri-piri.client
 343 9
simple-raytracer
 973 18
dht-id
 201 11
canela
 648 9
raytracer-browser-p2p NA 8
webrtc-chord 668 8
webrtc-chord-signalling-server NA 9
webrtc-chord-uui
 138 9
webrtc-ring 408 15
webrtc-ring-signalling-server NA 8
Totals 4438 219
TÉCNICO LISBOA
browserCloud.js
Thank you!
Questions?

More Related Content

Similar to browserCloud.js - David Dias M.Sc Thesis Defense Deck

GECon2017_ Lean_architecturemanagement_Andrei Kavaleu
GECon2017_ Lean_architecturemanagement_Andrei KavaleuGECon2017_ Lean_architecturemanagement_Andrei Kavaleu
GECon2017_ Lean_architecturemanagement_Andrei KavaleuGECon_Org Team
 
Unraveling OpenStack Clouds
 Unraveling OpenStack Clouds Unraveling OpenStack Clouds
Unraveling OpenStack CloudsIndicThreads
 
CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018Krishna-Kumar
 
opencdn_iecco18.pptx
opencdn_iecco18.pptxopencdn_iecco18.pptx
opencdn_iecco18.pptxssuser3855be
 
key research challenges in cloud computing
key research challenges in cloud computingkey research challenges in cloud computing
key research challenges in cloud computingIgnacio M. Llorente
 
IWMW 1998: Dataweb: Three Worlds Colide
IWMW 1998: Dataweb: Three Worlds ColideIWMW 1998: Dataweb: Three Worlds Colide
IWMW 1998: Dataweb: Three Worlds ColideIWMW
 
OAC - From Cloud Entry to Data Engineering to Data Science
OAC - From Cloud Entry to Data Engineering to Data ScienceOAC - From Cloud Entry to Data Engineering to Data Science
OAC - From Cloud Entry to Data Engineering to Data ScienceChristian Berg
 
Data integration with Apache Kafka
Data integration with Apache KafkaData integration with Apache Kafka
Data integration with Apache Kafkaconfluent
 
The Download: Tech Talks by the HPCC Systems Community, Episode 11
The Download: Tech Talks by the HPCC Systems Community, Episode 11The Download: Tech Talks by the HPCC Systems Community, Episode 11
The Download: Tech Talks by the HPCC Systems Community, Episode 11HPCC Systems
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...Srijan Technologies
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableComsysto Reply GmbH
 
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformation
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformationEvolutionary evnt-driven-architecture-for-accelerated-digital-transformation
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformationSlobodan Sipcic
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data EngineeringC4Media
 
How to Contribute to Cloud Native Computing Foundation
How to Contribute to Cloud Native Computing FoundationHow to Contribute to Cloud Native Computing Foundation
How to Contribute to Cloud Native Computing FoundationCodeOps Technologies LLP
 
How to contribute to cloud native computing foundation (CNCF)
How to contribute to cloud native computing foundation (CNCF)How to contribute to cloud native computing foundation (CNCF)
How to contribute to cloud native computing foundation (CNCF)Krishna-Kumar
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyComsysto Reply GmbH
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyComsysto Reply GmbH
 
Serverless microservices
Serverless microservicesServerless microservices
Serverless microservicesLalit Kale
 

Similar to browserCloud.js - David Dias M.Sc Thesis Defense Deck (20)

GECon2017_ Lean_architecturemanagement_Andrei Kavaleu
GECon2017_ Lean_architecturemanagement_Andrei KavaleuGECon2017_ Lean_architecturemanagement_Andrei Kavaleu
GECon2017_ Lean_architecturemanagement_Andrei Kavaleu
 
Unraveling OpenStack Clouds
 Unraveling OpenStack Clouds Unraveling OpenStack Clouds
Unraveling OpenStack Clouds
 
CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018CNCF Introduction - Feb 2018
CNCF Introduction - Feb 2018
 
opencdn_iecco18.pptx
opencdn_iecco18.pptxopencdn_iecco18.pptx
opencdn_iecco18.pptx
 
key research challenges in cloud computing
key research challenges in cloud computingkey research challenges in cloud computing
key research challenges in cloud computing
 
IWMW 1998: Dataweb: Three Worlds Colide
IWMW 1998: Dataweb: Three Worlds ColideIWMW 1998: Dataweb: Three Worlds Colide
IWMW 1998: Dataweb: Three Worlds Colide
 
OAC - From Cloud Entry to Data Engineering to Data Science
OAC - From Cloud Entry to Data Engineering to Data ScienceOAC - From Cloud Entry to Data Engineering to Data Science
OAC - From Cloud Entry to Data Engineering to Data Science
 
Data integration with Apache Kafka
Data integration with Apache KafkaData integration with Apache Kafka
Data integration with Apache Kafka
 
Docker12 factor
Docker12 factorDocker12 factor
Docker12 factor
 
The Download: Tech Talks by the HPCC Systems Community, Episode 11
The Download: Tech Talks by the HPCC Systems Community, Episode 11The Download: Tech Talks by the HPCC Systems Community, Episode 11
The Download: Tech Talks by the HPCC Systems Community, Episode 11
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
 
Software Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuableSoftware Architecture and Architectors: useless VS valuable
Software Architecture and Architectors: useless VS valuable
 
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformation
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformationEvolutionary evnt-driven-architecture-for-accelerated-digital-transformation
Evolutionary evnt-driven-architecture-for-accelerated-digital-transformation
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
How to Contribute to Cloud Native Computing Foundation
How to Contribute to Cloud Native Computing FoundationHow to Contribute to Cloud Native Computing Foundation
How to Contribute to Cloud Native Computing Foundation
 
How to contribute to cloud native computing foundation (CNCF)
How to contribute to cloud native computing foundation (CNCF)How to contribute to cloud native computing foundation (CNCF)
How to contribute to cloud native computing foundation (CNCF)
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Cloud Computing Basics
Cloud Computing BasicsCloud Computing Basics
Cloud Computing Basics
 
Serverless microservices
Serverless microservicesServerless microservices
Serverless microservices
 

More from David Dias

Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocol
Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocolEnter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocol
Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocolDavid Dias
 
IPWB and IPFS at WAC2017
IPWB and IPFS at WAC2017IPWB and IPFS at WAC2017
IPWB and IPFS at WAC2017David Dias
 
RDM#2- The Distributed Web
RDM#2- The Distributed WebRDM#2- The Distributed Web
RDM#2- The Distributed WebDavid Dias
 
Node.js Interactive
Node.js InteractiveNode.js Interactive
Node.js InteractiveDavid Dias
 
Understanding The Community Lifecycle
Understanding The Community LifecycleUnderstanding The Community Lifecycle
Understanding The Community LifecycleDavid Dias
 
P2P Resource Discovery for the Browser
P2P Resource Discovery for the BrowserP2P Resource Discovery for the Browser
P2P Resource Discovery for the BrowserDavid Dias
 
Lisboa WebRTC - May 21, 2015 - Intro to WebRTC
Lisboa WebRTC - May 21, 2015 - Intro to WebRTCLisboa WebRTC - May 21, 2015 - Intro to WebRTC
Lisboa WebRTC - May 21, 2015 - Intro to WebRTCDavid Dias
 
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...David Dias
 
TriConf 2014 - LXJS, the Lisbon Javascript Conference
TriConf 2014 - LXJS, the Lisbon Javascript ConferenceTriConf 2014 - LXJS, the Lisbon Javascript Conference
TriConf 2014 - LXJS, the Lisbon Javascript ConferenceDavid Dias
 
JSConfBR - Securing Node.js App, by the community and for the community
JSConfBR - Securing Node.js App, by the community and for the community JSConfBR - Securing Node.js App, by the community and for the community
JSConfBR - Securing Node.js App, by the community and for the community David Dias
 

More from David Dias (10)

Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocol
Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocolEnter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocol
Enter Gossipsub, A scalable, extensible & hardened P2P PubSub Router protocol
 
IPWB and IPFS at WAC2017
IPWB and IPFS at WAC2017IPWB and IPFS at WAC2017
IPWB and IPFS at WAC2017
 
RDM#2- The Distributed Web
RDM#2- The Distributed WebRDM#2- The Distributed Web
RDM#2- The Distributed Web
 
Node.js Interactive
Node.js InteractiveNode.js Interactive
Node.js Interactive
 
Understanding The Community Lifecycle
Understanding The Community LifecycleUnderstanding The Community Lifecycle
Understanding The Community Lifecycle
 
P2P Resource Discovery for the Browser
P2P Resource Discovery for the BrowserP2P Resource Discovery for the Browser
P2P Resource Discovery for the Browser
 
Lisboa WebRTC - May 21, 2015 - Intro to WebRTC
Lisboa WebRTC - May 21, 2015 - Intro to WebRTCLisboa WebRTC - May 21, 2015 - Intro to WebRTC
Lisboa WebRTC - May 21, 2015 - Intro to WebRTC
 
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...
Resource Discovery for the Web Platform using a P2P Overlay Network with WebR...
 
TriConf 2014 - LXJS, the Lisbon Javascript Conference
TriConf 2014 - LXJS, the Lisbon Javascript ConferenceTriConf 2014 - LXJS, the Lisbon Javascript Conference
TriConf 2014 - LXJS, the Lisbon Javascript Conference
 
JSConfBR - Securing Node.js App, by the community and for the community
JSConfBR - Securing Node.js App, by the community and for the community JSConfBR - Securing Node.js App, by the community and for the community
JSConfBR - Securing Node.js App, by the community and for the community
 

Recently uploaded

Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptxIntroduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptxk795866
 
Current Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCLCurrent Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCLDeelipZope
 
power system scada applications and uses
power system scada applications and usespower system scada applications and uses
power system scada applications and usesDevarapalliHaritha
 
Internship report on mechanical engineering
Internship report on mechanical engineeringInternship report on mechanical engineering
Internship report on mechanical engineeringmalavadedarshan25
 
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxDecoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxJoão Esperancinha
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...Soham Mondal
 
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...srsj9000
 
Biology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptxBiology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptxDeepakSakkari2
 
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile serviceCall Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile servicerehmti665
 
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdfCCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdfAsst.prof M.Gokilavani
 
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxConcrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxKartikeyaDwivedi3
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxpurnimasatapathy1234
 
Past, Present and Future of Generative AI
Past, Present and Future of Generative AIPast, Present and Future of Generative AI
Past, Present and Future of Generative AIabhishek36461
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )Tsuyoshi Horigome
 
Software and Systems Engineering Standards: Verification and Validation of Sy...
Software and Systems Engineering Standards: Verification and Validation of Sy...Software and Systems Engineering Standards: Verification and Validation of Sy...
Software and Systems Engineering Standards: Verification and Validation of Sy...VICTOR MAESTRE RAMIREZ
 
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfCCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfAsst.prof M.Gokilavani
 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024hassan khalil
 
Application of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptxApplication of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptx959SahilShah
 

Recently uploaded (20)

Introduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptxIntroduction-To-Agricultural-Surveillance-Rover.pptx
Introduction-To-Agricultural-Surveillance-Rover.pptx
 
Current Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCLCurrent Transformer Drawing and GTP for MSETCL
Current Transformer Drawing and GTP for MSETCL
 
power system scada applications and uses
power system scada applications and usespower system scada applications and uses
power system scada applications and uses
 
Internship report on mechanical engineering
Internship report on mechanical engineeringInternship report on mechanical engineering
Internship report on mechanical engineering
 
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptxDecoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
Decoding Kotlin - Your guide to solving the mysterious in Kotlin.pptx
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
 
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
Gfe Mayur Vihar Call Girls Service WhatsApp -> 9999965857 Available 24x7 ^ De...
 
Biology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptxBiology for Computer Engineers Course Handout.pptx
Biology for Computer Engineers Course Handout.pptx
 
Call Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile serviceCall Girls Delhi {Jodhpur} 9711199012 high profile service
Call Girls Delhi {Jodhpur} 9711199012 high profile service
 
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdfCCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
CCS355 Neural Network & Deep Learning Unit II Notes with Question bank .pdf
 
Concrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptxConcrete Mix Design - IS 10262-2019 - .pptx
Concrete Mix Design - IS 10262-2019 - .pptx
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptx
 
Design and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdfDesign and analysis of solar grass cutter.pdf
Design and analysis of solar grass cutter.pdf
 
Past, Present and Future of Generative AI
Past, Present and Future of Generative AIPast, Present and Future of Generative AI
Past, Present and Future of Generative AI
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )
 
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
🔝9953056974🔝!!-YOUNG call girls in Rajendra Nagar Escort rvice Shot 2000 nigh...
 
Software and Systems Engineering Standards: Verification and Validation of Sy...
Software and Systems Engineering Standards: Verification and Validation of Sy...Software and Systems Engineering Standards: Verification and Validation of Sy...
Software and Systems Engineering Standards: Verification and Validation of Sy...
 
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdfCCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
CCS355 Neural Network & Deep Learning UNIT III notes and Question bank .pdf
 
Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024Architect Hassan Khalil Portfolio for 2024
Architect Hassan Khalil Portfolio for 2024
 
Application of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptxApplication of Residue Theorem to evaluate real integrations.pptx
Application of Residue Theorem to evaluate real integrations.pptx
 

browserCloud.js - David Dias M.Sc Thesis Defense Deck

  • 1. TÉCNICO LISBOA browserCloud.js A federated community cloud using a P2P overlay network on top of the Web Platform Telecommunications and Computer Science Engineering project David Dias - 65963 - mail@daviddias.me - github.com/diasdavid
  • 2. TÉCNICO LISBOA AGENDA • Motivation • Related work • Architecture • Implementation • Evaluation • Conclusions • Publications and Impact 2
  • 3. TÉCNICO LISBOA MOTIVATION • User generated data has been increasing exponentially. • There is a huge demand of large amounts of computing power. • There is no really sustainable and economic solution for the masses. 3 Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications
  • 4. TÉCNICO LISBOA CHALLENGES • Access to large untapped computing infrastructure. • Execute distributed jobs in a heterogeneous environment, not confined to specific non-universal platforms. • Network bottleneck of constant moving data-to computation vs computation-to-data. • Lack of capability to quickly adapt to different types of jobs, as seen in BOINC based systems. • Centralised Computing platforms have scalability problems when usage and users number grows. 4 Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications
  • 5. TÉCNICO LISBOA OBJECTIVES 5 • Have a decentralised infrastructure, virtually self-organised • Enable flexible types of jobs to be executed, defined in pre-execution time. • Efficient decentralised lookup system. • Enable common users to take advantage of free Computing system. • Easy volunteer sign up. Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications
  • 6. TÉCNICO LISBOA CONTRIBUTIONS 6 • We’ve successfully delivered a: • Decentralised communication platform for the browser • WebRTC DHT • Job Distribution and Agnostic Execution System • Case Study: Ray Tracing framework • Validation: Decentralised browser testing framework • Every component is open source, MIT licensed • Contributions can be found at http://github.com/diasdavid/, identified with the INESC ID Lisboa and Técnico Lisboa emblem. Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications
  • 7. TÉCNICO LISBOA RELATED WORK 7 Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications
  • 8. TÉCNICO LISBOA RELATED WORK • Cloud computing and Open Cloud Platforms • Volunteered resource sharing • Web platform 8 Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications
  • 9. TÉCNICO LISBOA CLOUD COMPUTING • Cloud Computing describes a larger number of computers, connected through a network, with special differentiating properties[4]: • Several offers in the Cloud Computing space • It is an old concept, envisioned around 1961 by Prof. John McCarthy, MIT • Created the “lock-in” syndrome 9 IaaS Infrastructure as a Service PaaS Platform as a Service SaaS Software as a Service … … Advantage Cloud Conventional Data Center Virtually infinite computing resources Yes No Freedom from up-front commitment Yes No “Pay-as-you-go” Yes No Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications
  • 10. TÉCNICO LISBOA VOLUNTEERED RESOURCE SHARING • Definition: Sharing idle computing resources on the network, that otherwise would be wasted • Computing Jobs are defined with meta-heuristics, creating a bag-of- tasks, able to be run in parallel • Community Clouds - Known also as “bottom up networking”, where individuals join a community effort to create data-links, community managed, most known are: guifi.net(over 20000 nodes!) , AWMN and funkfeur freenet • CONFINE[34] - Community Cloud federation,
 start of the community-lab test bed and
 Clommunity, a community network in a box.
 
 
 
 10 Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications
  • 11. TÉCNICO LISBOA CYCLE AND STORAGE SHARING • Improve flexibility on the type of jobs you execute with Gridlets, a unit of workload that packs the data and the logic necessary to process it 11 Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications Type System Architecture Organization Storage Sharing Napster P2P Kazaa Freenet BitTorrent Cycle Sharing @HOME projects (BOINC Based) Centralized
  • 12. TÉCNICO LISBOA PEER-TO-PEER ARCHITECTURES 12 Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications
  • 13. TÉCNICO LISBOA WEB PLATFORM • Ubiquitous platform • Highly dynamic runtime for differential updates and code migration for job execution • Recent developments have enabled: • Close to native performance for highly CPU bound jobs • P2P Interconnectivity • Scalable storage and fast indexing 13 Motivation Challenges Objectives Contributions Related work CC and Open Cloud Platforms Volunteered Resource Sharing Web platform Architecture Implementation Evaluation Conclusions Publications
  • 14. TÉCNICO LISBOA Key learnings from the related work 14 Motivation Challenges Objectives Contributions Related work Architecture Implementation Evaluation Conclusions Publications Section Learnings Cloud Computing Interoperability is easier to happen when abstracted Community Clouds Gridlets mitigate the network bottleneck of moving the data to the computation by moving the computation to the data (also seen in Manta) P2P Efficient storage/lookups requires structured organization of the data Web Platform Highly dynamic runtime is possible thanks to Javascript Near native performance is possible in a browser P2P in a browser is now possible with WebRTC
  • 16. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE 16 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 17. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE • Membership management - Peers to join and leave browserCloud.js. A peer only has the knowledge of a small group of other peers. • Message routing - Messages are routed between peers. • Job scheduling and results aggregation - Peers interact between each other to send tasks and retrieve the results to the peer executing the job. • Dynamic runtime support - Job agnostic. • Reduced entrance cost to enable greater adoption - Simple APIs design, abstracting the complexity in favor of greater extendability. • Enable integration and compliance tests - Automate the process of verifying browserCloudjs integrity and functionality. 17 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications Entities: • browser - The points on our network that will be able to issue jobs, execute tasks and route messages. • Rendezvous point - The only centralised component in this architecture, its purpose is for the clients to have a way to connect to and join the overlay network.
  • 18. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE Membership Management: 1 - Registration 2 - New peer available - Finger tables are updated. 3 - Connection establishment between two peers 18 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 19. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE Membership Management: 1 - Registration 2 - New peer available - Finger tables are updated. 3 - Connection establishment between two peers 19 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 20. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE Membership Management: 1 - Registration 2 - New peer available - Finger tables are updated. 3 - Connection establishment between two peers 20 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 21. TÉCNICO LISBOA DISTRIBUTED ARCHITECTURE Message routing: • Chord routing scheme • 48 bit unique identifiers • Flexible finger table 21 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 22. TÉCNICO LISBOA RESOURCE MANAGEMENT Job workflow: 1 - Divide a Job into smaller units (bag of tasks) 2 - Select how many browsers we want to distribute the job to. 3 - Query the network for available browsers. 4 - Compose the several gridlets with task plus a data partition. 5 - Send each gridlets to the network. 6 - Browsers compute the results and send them back to the job issuer. 7 - Browser submitting the job gathers all the tasks results and constructs the job result. 
 22 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 23. TÉCNICO LISBOA SOFTWARE STACK Communication layer: • Holds the connections with other peers. • Performs the necessary logic, using fingers, for efficient routing. • Keeps the peer connected to the network by updating its routing table as necessary. 23 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 24. TÉCNICO LISBOA SOFTWARE STACK Service router: • Plugin like module. • Interface with communication layer. • Enables new services to leverage the P2P browser infrastructure. 24 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 25. TÉCNICO LISBOA SOFTWARE STACK Job Scheduler: • Find browsers available in the network. • Orchestrates the Job over the network. • Creates gridlets from data + tasks to be executed in each browser. 25 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 26. TÉCNICO LISBOA TESTING Testing framework: 1 - A Web Server is started by the Control Center, this endpoint will be serving the necessary static assets (e,g .html, .css and .js files). 2 + 3 - The number of required browsers for the test being executed, are spawned. 4 - The Control Center starts sending commands to each browser to execute. 5 - Browsers report to the Control Center which events were triggered. 6 - Control Center assesses the order in which these events happened and asserts if the behaviour was the expected. 26 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 27. TÉCNICO LISBOA TESTING Testing framework: 1 - A Web Server is started by the Control Center, this endpoint will be serving the necessary static assets (e,g .html, .css and .js files). 2 + 3 - The number of required browsers for the test being executed, are spawned. 4 - The Control Center starts sending commands to each browser to execute. 5 - Browsers report to the Control Center which events were triggered. 6 - Control Center assesses the order in which these events happened and asserts if the behaviour was the expected. 27 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 28. TÉCNICO LISBOA TESTING Testing framework: 1 - A Web Server is started by the Control Center, this endpoint will be serving the necessary static assets (e,g .html, .css and .js files). 2 + 3 - The number of required browsers for the test being executed, are spawned. 4 - The Control Center starts sending commands to each browser to execute. 5 - Browsers report to the Control Center which events were triggered. 6 - Control Center assesses the order in which these events happened and asserts if the behaviour was the expected. 28 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 29. TÉCNICO LISBOA TESTING Testing framework: 1 - A Web Server is started by the Control Center, this endpoint will be serving the necessary static assets (e,g .html, .css and .js files). 2 + 3 - The number of required browsers for the test being executed, are spawned. 4 - The Control Center starts sending commands to each browser to execute. 5 - Browsers report to the Control Center which events were triggered. 6 - Control Center assesses the order in which these events happened and asserts if the behaviour was the expected. 29 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 30. TÉCNICO LISBOA TESTING Testing framework: 1 - A Web Server is started by the Control Center, this endpoint will be serving the necessary static assets (e,g .html, .css and .js files). 2 + 3 - The number of required browsers for the test being executed, are spawned. 4 - The Control Center starts sending commands to each browser to execute. 5 - Browsers report to the Control Center which events were triggered. 6 - Control Center assesses the order in which these events happened and asserts if the behaviour was the expected. 30 Motivation Challenges Objectives Contributions Related Work Architecture Distributed Architecture Resource Management Software Stack Testing Implementation Evaluation Conclusions Publications
  • 32. TÉCNICO LISBOA IMPLEMENTATION • Browser module • Signalling Server • Testing Framework • Visualizer • Simulator • Ray Tracing module 32 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 33. TÉCNICO LISBOA BROWSER MODULE • Establishes the link between the browsers • Holds the intelligence for Chord routing • Capable of distribute jobs over the network 33 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 34. TÉCNICO LISBOA SIGNALLING SERVER • HTTP + WebSockets Server • Rendezvous point for browsers to connect • Has a snapshot of the DHT state 34 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 35. TÉCNICO LISBOA TESTING FRAMEWORK • Spawns browsers • Exposes a HTTP server • Uses WebSockets to interact with the browsers 35 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 36. TÉCNICO LISBOA VISUALIZER • Retrieves DHT information from Signalling Server • Represents the DHT in a graph using D3JS 36 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 37. TÉCNICO LISBOA SIMULATOR • Similar to the Visualizer. • Able to create customised generations of a browserCloudjs network for visualisation. 37 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 38. TÉCNICO LISBOA RAY TRACING • Parse a CSS file describing the scene • Creates individual tasks • Runs tasks • Glues the results • Exports to .png or canvas 38 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Browser module Signalling Server Testing Framework Visualizer Simulator Ray Tracing Evaluation Conclusions Publications
  • 40. TÉCNICO LISBOA QUALITATIVE ASSESSMENT ✓ Efficient resource discovery through peer-to-peer routing over a structured overlay network, using a DHT. ✓ Distribute Jobs through the peers available, submitting, coordinating and aggregating the results. ✓ Remove the need for centralised indexes or points of control. 
 There is still a need of a rendezvous point, however the data transmitted, computed and stored inside the network is peer’s responsibility. ✓ Enable every machine equipped with a WebRTC enabled browser to be part of a browserCloudjs instance. 40 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications
  • 41. TÉCNICO LISBOA QUALITATIVE ASSESSMENT ✓ Enable peers to both participate and contribute to a job and at the same time submitting and requesting the network to process their own. ✓ Enable browsers to be part of one or more browserCloudjs instance. ✓ browserCloudjs’ Job Scheduler is job agnostic, this means that different types of jobs can be executed on demand without any previous configuration or preparation. ✓ browserCloudjs solves the decentralised communication problem between browsers in a scalable way.
 Enable new scenarios to be developed on top of it through its modular and pluggable approach. 41 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications
  • 42. TÉCNICO LISBOA QUANTITATIVE ASSESSMENT • Setup: Core i7 with 16Gb of RAM running Chrome v39 • Goals • Measuring the time lapsed for a single browser to compute a CPU bound job and several browsers to compute that same job, but in parallel. • Measuring the RTT time between any of two browsers in the network and evaluate as routing efficiency evolves with the increase in number of browsers. • Assessing if there are significant speedups. 42 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications
  • 43. TÉCNICO LISBOA QUANTITATIVE ASSESSMENT Ideal scenario(1): jobTime = DelayFromResourceDiscovery + timeOfExecutingSlowestTask + slowestDelayFromResultReply Using shared resources(2): jobTime = ︎ DelayFromResourceDiscovery + (timeOfExecuting N Tasks on M Resources)+ ︎ DelayFromResultReply 43 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications RTT between 2 browsers in a 10 browser network
  • 44. TÉCNICO LISBOA QUANTITATIVE ASSESSMENT • Length of running the Ray Tracing Job in one machine - 24ms 44 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications
  • 46. TÉCNICO LISBOA QUANTITATIVE ASSESSMENT • Identified performance bottlenecks • Logging - Since V8 runs in a single thread, any synchronous operation will block the event loop and add delay to the whole processing; although these logs are essential for us to assess the efficiency of the infrastructure, they are not vital to the job. • Delay added - One technique we used to simulate the network delay is to use the ‘set-Timeout’ native function of V8’s JavaScript implementation, however it does not necessarily guarantee that the function will be executed in X amount of milliseconds. • Tasks can not be sent in parallel - A node has to send each individual computing unit sequentially and independently, meaning that if we divide a job into 2000 tasks for e.g, each task will have to wait for the previous to be sent. • We have noticed a considerable machine resources bottle neck when we had more than 40 data channels present in same machine - This means that is is unviable to leverage the full 48 fingers space our finger table has the capacity, since one fingers equals 2 data channels (one in and one out). 46 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Qualitative Quantitative Conclusions Publications
  • 48. TÉCNICO LISBOA CONCLUSIONS Major Achievements: • The first browser based DHT - browserCloudjs offers for the first time in browser history, a fully functional DHT, performing resource decentralised resource discovery on the browser. • The first peer-to-peer browser computing platform - the research of using browsers to leverage the idle computer cycles have been in the literature for a while, however, always following the centralised/BOINC model. browserCloudjs offers the first peer-2- peer browser computing framework with proven speedups. 48 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Conclusions Publications
  • 49. TÉCNICO LISBOA CONCLUSIONS Future Work: • New job strategies - Currently browserCloudjs only supports mapping jobs, however, there is no practical limitation to execute full map/reduce jobs and/or streaming functions (for realtime data scenarios). • Hybrid peers - peers that live inside of a server that can both act as a signalling server and stabilising the network when the churn rate is high. • Geographic distribution awareness - Select finger tables based in optimal RTT distribution and geographic positioning of peers. • Optimisation of the JavaScript code developed - Increasing the performance of crucial functions. • Continuous upgrade of browserCloudjs platform as the Web Platform APIs evolve and as WebRTC moves from the draft state to finished spec. 49 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Conclusions Publications
  • 50. TÉCNICO LISBOA PUBLICATIONS AND IMPACT • Delivered a talk at Data Terra Nemo, P2P Conf, Berlin, May 23rd & 24th. http://dtn.is/ • Delivered a talk at OpoJS, Oporto, Feb 26th. http://www.opojs.com • Delivered a talk at require(‘lx’), Lisbon, Jun 3rd. • Invited to deliver a talk a LNUG in the end of June. http://lnug.org/ • WebRTC Weekly Issue #60 mention, the number one WebRTC newsletter with more than 1000 subscribers (https:// webrtcweekly.com/issue/webrtc-weekly-issue-60/) • Number one Top article in EchoJS for 3 days in a row and Top-5 for 7 days. (http://www.echojs.com/news/14009) • browserCloud.js demo video - Over 220 impressions. (https:// www.youtube.com/watch?v=kjwIjoENCRE). • Started a collaboration with Karma to join forces on the browser testing framework https://github.com/diasdavid/piri-piri/issues/6 50 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Conclusions Publications
  • 51. TÉCNICO LISBOA PUBLICATIONS AND IMPACT 51 Motivation Challenges Objectives Contributions Related Work Architecture Implementation Evaluation Conclusions Publications Name Downloads Github Stars webrtc-explorer 615 63 webrtc-explorer-signalling-server NA 11 webrtc-explorer-visualizer NA 10 webrtc-explorer-simulator NA 9 webrtc-explorer-browser-process NA 8 piri-piri
 444 14 piri-piri.client
 343 9 simple-raytracer
 973 18 dht-id
 201 11 canela
 648 9 raytracer-browser-p2p NA 8 webrtc-chord 668 8 webrtc-chord-signalling-server NA 9 webrtc-chord-uui
 138 9 webrtc-ring 408 15 webrtc-ring-signalling-server NA 8 Totals 4438 219