SlideShare a Scribd company logo
1 of 1
Download to read offline
Implementa)on	
  
Step1:	
  	
  
•  Master	
  performs	
  handshake	
  with	
  all	
  
the	
  nearby	
  peers	
  (iDevices)	
  which	
  
offer	
  the	
  service	
  of	
  MapReduce	
  using	
  
MulAPeer	
  ConnecAvity.	
  
•  Job	
  is	
  to	
  count	
  the	
  occurrences	
  of	
  
alphabet	
  (A,	
  E,	
  W,	
  Z)	
  in	
  a	
  data	
  file.	
  
	
  
•  The	
  data	
  file	
  is	
  a	
  dicAonary	
  of	
  words	
  
with	
  more	
  than	
  5	
  million	
  characters.	
  
•  Avg	
  Speedup:	
  2.56	
  
Introduc)on	
  
•  A	
  single	
  iDevice	
  like	
  iPhone	
  or	
  iPad	
  does	
  not	
  
have	
  enough	
  processing	
  power	
  to	
  process	
  
computaAonally	
  complex	
  funcAons	
  on	
  large	
  
data	
  sets.	
  	
  
•  This	
  project	
  envisions	
  a	
  mesh	
  of	
  inter-­‐connected	
  
iDevices	
  to	
  accomplish	
  a	
  task	
  like	
  that.	
  
•  How?	
  By	
  working	
  together	
  on	
  a	
  single	
  data	
  set.	
  
Background	
  
Conclusions	
  
•  It	
  can	
  be	
  noAced	
  that	
  the	
  Ame	
  required	
  to	
  perform	
  
the	
  job	
  is	
  inversely	
  proporAonal	
  to	
  the	
  number	
  of	
  
peers	
  used.	
  
	
  
•  Using	
  this	
  methodology	
  many	
  client-­‐server	
  
systems	
  can	
  be	
  transformed	
  into	
  clustered	
  client	
  
systems,	
  such	
  as	
  face-­‐recogniAon.	
  
Implemen'ng	
  MapReduce	
  framework	
  on	
  iOS	
  devices	
  using	
  Mul'Peer	
  Connec'vity	
  
Student:	
  Varun	
  Goyal	
  	
  	
  	
  	
  	
  	
  	
  	
  Advisor:	
  Dr.	
  Peizhao	
  Hu	
  
Rochester	
  InsAtute	
  Of	
  Technology,	
  One	
  Lomb	
  Memorial	
  Drive,	
  Rochester,	
  NY	
  14623	
  -­‐	
  5603	
  
0	
  
10	
  
20	
  
30	
  
40	
  
50	
  
60	
  
1	
  Peer	
   2	
  Peers	
   3	
  Peers	
  
Count	
  of	
  A	
  
Count	
  of	
  E	
  
Count	
  of	
  W	
  
Count	
  of	
  Z	
  
Results	
  
•  This	
  was	
  recently	
  made	
  possible	
  when	
  Apple	
  
released	
  Mul'Peer	
  Connec'vity(MPC)	
  as	
  a	
  part	
  
of	
  iOS	
  7	
  release.	
  
•  MPC	
  makes	
  it	
  possible	
  for	
  iDevices	
  to	
  discover	
  
services	
  provided	
  by	
  other	
  iDevices	
  near	
  by.	
  
•  Uses	
  basic	
  infrastructure	
  like	
  WiFi	
  or	
  bluetooth.	
  
•  MapReduce	
  with	
  word	
  count	
  example:	
  
1.  Master	
  splits	
  the	
  input	
  data	
  into	
  chunks	
  and	
  
sends	
  it	
  to	
  each	
  worker	
  node	
  
2.  Map:	
  counts	
  the	
  number	
  of	
  occurances	
  for	
  
each	
  key	
  
3.  Shuffle:	
  data	
  is	
  shuffled	
  based	
  on	
  key	
  
4.  Reduce:	
  adds	
  up	
  the	
  count	
  for	
  each	
  key	
  
Step	
  2:	
  	
  
•  User	
  selects	
  the	
  job	
  to	
  be	
  performed	
  
•  Master	
  divides	
  the	
  data	
  according	
  to	
  
number	
  of	
  peers.	
  
•  Master	
  sends	
  data,	
  job,	
  and	
  
command	
  to	
  run	
  Map	
  funcAon	
  to	
  
peers	
  in	
  a	
  JSON.	
  	
  
Step	
  3:	
  
•  Peers	
  perform	
  Map	
  funcAon	
  on	
  their	
  
data	
  set	
  
•  Peers	
  return	
  the	
  map	
  funcAon’s	
  
output	
  to	
  Master	
  to	
  be	
  shuffled.	
  
Step	
  4:	
  	
  
•  Depending	
  the	
  number	
  of	
  unique	
  keys	
  
and	
  the	
  number	
  of	
  peers	
  shuffled	
  data	
  
from	
  step	
  3	
  is	
  divided	
  in	
  different	
  parts	
  
and	
  sent	
  back	
  to	
  the	
  peers	
  to	
  run	
  
reduce	
  funcAon	
  on	
  it.	
  
Step	
  5:	
  
•  Peers	
  perform	
  reduce	
  funcAon	
  on	
  
their	
  data	
  set.	
  
•  Peers	
  return	
  the	
  reduced	
  data	
  back	
  to	
  
master.	
  
•  Master	
  displays	
  the	
  final	
  result.	
  
References	
  
•  MulA-­‐peer	
  framework	
  by	
  Apple:	
  hgps://developer.apple.com/library/
prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/
MulApeerConnecAvityFramework/index.html	
  
	
  
•  MulAPeer	
  ConnecAvity,	
  Apple	
  Docs.	
  hgps://developer.apple.com/library/
prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/
MulApeerConnecAvityFramework/index.html	
  
	
  
•  Jeffrey	
  Dean	
  and	
  Sanjay	
  Ghemawat.	
  2008.	
  MapReduce:	
  simplified	
  data	
  
processing	
  on	
  large	
  clusters.	
  Commun.	
  ACM	
  51,	
  1	
  (January	
  2008),	
  107-­‐113.	
  
DOI=10.1145/1327452.1327492	
  hgp://doi.acm.org/
10.1145/1327452.1327492	
  
Contact	
  Info	
  
Name:	
  Varun	
  Goyal	
  
Email:	
  varungoyal44@gmail.com	
  
Phone:	
  +1	
  585	
  743	
  8102	
  
Future	
  Work	
  
Technical	
  Challenges	
  
•  Cannot	
  inject	
  code	
  yet	
  hence	
  cannot	
  supply	
  user	
  defined	
  job.	
  
	
  
•  Need	
  individual	
  device’s	
  permission	
  to	
  join	
  the	
  network	
  every	
  Ame	
  a	
  peer	
  
needs	
  to	
  join	
  the	
  system	
  which	
  provides	
  added	
  security	
  from	
  Apple	
  but	
  
reduces	
  the	
  technical	
  enhancements	
  and	
  new	
  possibiliAes.	
  
	
  
•  Face	
  RecogniAon	
  job	
  was	
  implemented	
  using	
  this	
  system,	
  however	
  due	
  to	
  
shortage	
  of	
  Ame	
  the	
  actual	
  task	
  of	
  performing	
  face-­‐recogniAon	
  is	
  performed	
  
using	
  Kairos	
  API.	
  
•  Once	
  a	
  soluAon	
  is	
  found	
  to	
  inject	
  code	
  into	
  the	
  
system,	
  which	
  Apple	
  can	
  approve,	
  the	
  framework	
  
could	
  be	
  enhanced	
  to	
  perform	
  user	
  defined	
  jobs.	
  
•  Face-­‐RecogniAon	
  algorithm	
  could	
  be	
  implemented	
  
on	
  iOS	
  and	
  provided	
  as	
  a	
  job	
  to	
  this	
  framework	
  to	
  
determine	
  the	
  effecAveness	
  of	
  the	
  framework	
  and	
  
viability	
  of	
  the	
  algorithm	
  to	
  perform	
  face	
  
recogniAon	
  in	
  fixed	
  amount	
  of	
  Ame.	
  
•  If	
  Apple	
  updates	
  the	
  MulAPeer	
  ConnecAvity	
  
framework	
  to	
  be	
  able	
  to	
  use	
  system	
  resources	
  in	
  
the	
  background	
  (like	
  an	
  Ideal	
  Mode)	
  this	
  
framework	
  can	
  be	
  used	
  with	
  minimum	
  user	
  
interacAon.	
  
0	
  
20	
  
40	
  
60	
  
80	
  
100	
  
120	
  
140	
  
160	
  
180	
  
200	
  
1	
  Peer	
   2	
  Peers	
   3	
  Peers	
  
Dear	
  
Bear	
  
Car	
  
Tree	
  
•  Job	
  is	
  to	
  count	
  the	
  occurrences	
  of	
  
words	
  (Dear,	
  Bear,	
  Car,	
  Tree)	
  in	
  a	
  data	
  
file.	
  
	
  
•  The	
  data	
  file	
  is	
  a	
  dicAonary	
  of	
  words	
  
with	
  more	
  than	
  83	
  million	
  characters.	
  
•  Avg	
  Speedup:	
  2.97	
  

More Related Content

Similar to PDF-Poster- Varun

Data Parallel and Object Oriented Model
Data Parallel and Object Oriented ModelData Parallel and Object Oriented Model
Data Parallel and Object Oriented ModelNikhil Sharma
 
Presentation 7.pptx
Presentation 7.pptxPresentation 7.pptx
Presentation 7.pptxShivam327815
 
Maha Mahmoud Fawzy Resume
Maha Mahmoud Fawzy ResumeMaha Mahmoud Fawzy Resume
Maha Mahmoud Fawzy Resumemaha fawzy
 
Cloud Computing - Geektalk
Cloud Computing - GeektalkCloud Computing - Geektalk
Cloud Computing - GeektalkMalisa Ncube
 
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTION
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTIONMOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTION
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTIONI Ruby
 
Performance Evaluation of a Network Using Simulation Tools or Packet Tracer
Performance Evaluation of a Network Using Simulation Tools or Packet TracerPerformance Evaluation of a Network Using Simulation Tools or Packet Tracer
Performance Evaluation of a Network Using Simulation Tools or Packet TracerIOSRjournaljce
 
Crime File System
Crime File SystemCrime File System
Crime File SystemIJARIIT
 
World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018Adam Gibson
 
Multi Layer Federated Learning.pptx
Multi Layer Federated Learning.pptxMulti Layer Federated Learning.pptx
Multi Layer Federated Learning.pptxTimePass43152
 
A two stage feature selection method for text categorization
A two stage feature selection method for text categorizationA two stage feature selection method for text categorization
A two stage feature selection method for text categorizationParag Tamhane
 
System models for distributed and cloud computing
System models for distributed and cloud computingSystem models for distributed and cloud computing
System models for distributed and cloud computingpurplesea
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011marpierc
 
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...In-Memory Computing Summit
 
K anonymity for crowdsourcing database
K anonymity for crowdsourcing databaseK anonymity for crowdsourcing database
K anonymity for crowdsourcing databaseLeMeniz Infotech
 

Similar to PDF-Poster- Varun (20)

Data Parallel and Object Oriented Model
Data Parallel and Object Oriented ModelData Parallel and Object Oriented Model
Data Parallel and Object Oriented Model
 
YouSz_20210421.pptx
YouSz_20210421.pptxYouSz_20210421.pptx
YouSz_20210421.pptx
 
Presentation 7.pptx
Presentation 7.pptxPresentation 7.pptx
Presentation 7.pptx
 
Maha Mahmoud Fawzy Resume
Maha Mahmoud Fawzy ResumeMaha Mahmoud Fawzy Resume
Maha Mahmoud Fawzy Resume
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Cloud Computing - Geektalk
Cloud Computing - GeektalkCloud Computing - Geektalk
Cloud Computing - Geektalk
 
Hadoop
HadoopHadoop
Hadoop
 
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTION
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTIONMOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTION
MOBILE CLOUD COMPUTING USING CRYPTOGRAPHIC HASH FUNCTION
 
Performance Evaluation of a Network Using Simulation Tools or Packet Tracer
Performance Evaluation of a Network Using Simulation Tools or Packet TracerPerformance Evaluation of a Network Using Simulation Tools or Packet Tracer
Performance Evaluation of a Network Using Simulation Tools or Packet Tracer
 
Crime File System
Crime File SystemCrime File System
Crime File System
 
50120140505006
5012014050500650120140505006
50120140505006
 
World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018
 
Multi Layer Federated Learning.pptx
Multi Layer Federated Learning.pptxMulti Layer Federated Learning.pptx
Multi Layer Federated Learning.pptx
 
A two stage feature selection method for text categorization
A two stage feature selection method for text categorizationA two stage feature selection method for text categorization
A two stage feature selection method for text categorization
 
System models for distributed and cloud computing
System models for distributed and cloud computingSystem models for distributed and cloud computing
System models for distributed and cloud computing
 
Deep Learning at Scale
Deep Learning at ScaleDeep Learning at Scale
Deep Learning at Scale
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011
 
E-AEGIS
E-AEGISE-AEGIS
E-AEGIS
 
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
IMCSummit 2015 - Day 1 Developer Track - In-memory Computing for Iterative CP...
 
K anonymity for crowdsourcing database
K anonymity for crowdsourcing databaseK anonymity for crowdsourcing database
K anonymity for crowdsourcing database
 

PDF-Poster- Varun

  • 1. Implementa)on   Step1:     •  Master  performs  handshake  with  all   the  nearby  peers  (iDevices)  which   offer  the  service  of  MapReduce  using   MulAPeer  ConnecAvity.   •  Job  is  to  count  the  occurrences  of   alphabet  (A,  E,  W,  Z)  in  a  data  file.     •  The  data  file  is  a  dicAonary  of  words   with  more  than  5  million  characters.   •  Avg  Speedup:  2.56   Introduc)on   •  A  single  iDevice  like  iPhone  or  iPad  does  not   have  enough  processing  power  to  process   computaAonally  complex  funcAons  on  large   data  sets.     •  This  project  envisions  a  mesh  of  inter-­‐connected   iDevices  to  accomplish  a  task  like  that.   •  How?  By  working  together  on  a  single  data  set.   Background   Conclusions   •  It  can  be  noAced  that  the  Ame  required  to  perform   the  job  is  inversely  proporAonal  to  the  number  of   peers  used.     •  Using  this  methodology  many  client-­‐server   systems  can  be  transformed  into  clustered  client   systems,  such  as  face-­‐recogniAon.   Implemen'ng  MapReduce  framework  on  iOS  devices  using  Mul'Peer  Connec'vity   Student:  Varun  Goyal                  Advisor:  Dr.  Peizhao  Hu   Rochester  InsAtute  Of  Technology,  One  Lomb  Memorial  Drive,  Rochester,  NY  14623  -­‐  5603   0   10   20   30   40   50   60   1  Peer   2  Peers   3  Peers   Count  of  A   Count  of  E   Count  of  W   Count  of  Z   Results   •  This  was  recently  made  possible  when  Apple   released  Mul'Peer  Connec'vity(MPC)  as  a  part   of  iOS  7  release.   •  MPC  makes  it  possible  for  iDevices  to  discover   services  provided  by  other  iDevices  near  by.   •  Uses  basic  infrastructure  like  WiFi  or  bluetooth.   •  MapReduce  with  word  count  example:   1.  Master  splits  the  input  data  into  chunks  and   sends  it  to  each  worker  node   2.  Map:  counts  the  number  of  occurances  for   each  key   3.  Shuffle:  data  is  shuffled  based  on  key   4.  Reduce:  adds  up  the  count  for  each  key   Step  2:     •  User  selects  the  job  to  be  performed   •  Master  divides  the  data  according  to   number  of  peers.   •  Master  sends  data,  job,  and   command  to  run  Map  funcAon  to   peers  in  a  JSON.     Step  3:   •  Peers  perform  Map  funcAon  on  their   data  set   •  Peers  return  the  map  funcAon’s   output  to  Master  to  be  shuffled.   Step  4:     •  Depending  the  number  of  unique  keys   and  the  number  of  peers  shuffled  data   from  step  3  is  divided  in  different  parts   and  sent  back  to  the  peers  to  run   reduce  funcAon  on  it.   Step  5:   •  Peers  perform  reduce  funcAon  on   their  data  set.   •  Peers  return  the  reduced  data  back  to   master.   •  Master  displays  the  final  result.   References   •  MulA-­‐peer  framework  by  Apple:  hgps://developer.apple.com/library/ prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/ MulApeerConnecAvityFramework/index.html     •  MulAPeer  ConnecAvity,  Apple  Docs.  hgps://developer.apple.com/library/ prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/ MulApeerConnecAvityFramework/index.html     •  Jeffrey  Dean  and  Sanjay  Ghemawat.  2008.  MapReduce:  simplified  data   processing  on  large  clusters.  Commun.  ACM  51,  1  (January  2008),  107-­‐113.   DOI=10.1145/1327452.1327492  hgp://doi.acm.org/ 10.1145/1327452.1327492   Contact  Info   Name:  Varun  Goyal   Email:  varungoyal44@gmail.com   Phone:  +1  585  743  8102   Future  Work   Technical  Challenges   •  Cannot  inject  code  yet  hence  cannot  supply  user  defined  job.     •  Need  individual  device’s  permission  to  join  the  network  every  Ame  a  peer   needs  to  join  the  system  which  provides  added  security  from  Apple  but   reduces  the  technical  enhancements  and  new  possibiliAes.     •  Face  RecogniAon  job  was  implemented  using  this  system,  however  due  to   shortage  of  Ame  the  actual  task  of  performing  face-­‐recogniAon  is  performed   using  Kairos  API.   •  Once  a  soluAon  is  found  to  inject  code  into  the   system,  which  Apple  can  approve,  the  framework   could  be  enhanced  to  perform  user  defined  jobs.   •  Face-­‐RecogniAon  algorithm  could  be  implemented   on  iOS  and  provided  as  a  job  to  this  framework  to   determine  the  effecAveness  of  the  framework  and   viability  of  the  algorithm  to  perform  face   recogniAon  in  fixed  amount  of  Ame.   •  If  Apple  updates  the  MulAPeer  ConnecAvity   framework  to  be  able  to  use  system  resources  in   the  background  (like  an  Ideal  Mode)  this   framework  can  be  used  with  minimum  user   interacAon.   0   20   40   60   80   100   120   140   160   180   200   1  Peer   2  Peers   3  Peers   Dear   Bear   Car   Tree   •  Job  is  to  count  the  occurrences  of   words  (Dear,  Bear,  Car,  Tree)  in  a  data   file.     •  The  data  file  is  a  dicAonary  of  words   with  more  than  83  million  characters.   •  Avg  Speedup:  2.97