Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Javantura v3 - Apache Spark revolution – what’s it all about – Petar Zečević

1,233 views

Published on

Javantura v3 - Apache Spark revolution – what’s it all about – Petar Zečević

Published in: Technology
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Javantura v3 - Apache Spark revolution – what’s it all about – Petar Zečević

  1. 1. Apache Spark Rnvoludon what’: it an about? $ Javant u ra GROUP « I-Ire! El Petar Zeéevié petar. zecevic@svgroup. hr @p_zecevic
  2. 2. Apache Spark Zagreb Mectup 9101-IP http: //wvvw. meetup. com/ Apache- Spark-Zagreb-Meetup
  3. 3. (discouinit code on "Elm last slide 2) ) - If _ ‘ '. '“- l J _| ill http: //bit. |y/ sparkinaction
  4. 4. fitgetiiria iiet today What is Spark? Why Spark? How does it work? What can you do with it? Q & A
  5. 5. Show of heme 0 I've never used Apache Spark 0 I've played around with it 0 I'm planning to or I'm already using Spark in production
  6. 6. Whet is Apeche Spark? It is a distributed general-purpose large-scale data processing engine
  7. 7. Distributed Runs on many machines Runs in a cluster Parallelizes computations Sends program to data
  8. 8. itieenereflapntpsese Used across industries Reads any kind of data Writes any kind of data Does whatever Iava/ Python, /R can do
  9. 9. Big Data: % But also: Banks Healthcare Biology Physics Weather Astrophysics But you should have enough data
  10. 10. Dete proceulng engine It's about processing data It's not your ERP system It's not your web framework It's not for rendering videos
  11. 11. Why Speck? (Why not MapReduce)
  12. 12. sgpiatlt; is itfasii: is Works mostly in memory Especially fast for iterative tasks 100 times faster than MapReduce Broke sorting world record in 2014 Project Tungsten brought additional optimizations in v1.5
  13. 13. Simple end conciee AP! 0 Functional programming 0 Distributed collections feel like local ones 0 APIS in Scala, lava, Python and R
  14. 14. :'1Z‘. f_'_‘s it‘: :. ;.: .t‘. 50.7. ‘is '— .4.‘ _ . ~li. "‘ i, Il Y ,3 an , - I I 34:" K 1 1 l'il': .r": _t5l I 1 ix. -1-- i-« nu». I. ..” ago. .. ~. . noun. a. .. pg. .- sq. .. 7.-. ~ um «um. -e-1 nu. - ---e-- ‘ i. ..—¢—-u , .. . ... .. -.4 . -u-mg” . ..i . ... unen- 1 —. u . .a . - —. .-ipa. .. . .. r . .-. .-—-n. _. $031 ; , . - . --. <u—u. «i—-nu: . . .r . p.. ... .—i--. ... nnrlvnu Yll-Illa: 4l'i, ~'1-#1 i. ..-u-9.-u a-u-om. --i r«, s Duvlxul‘ nan-v-—. -—u-on-uirv. — non-nix; i. &IIln. nn&tu-mi V — — — i-- -g. -—. n-mu-u-o, —u. u-u -. .. u. «—-. .- l avatar‘! bvutlvnwll I n—. u.u. .. .a . — gnu-ii me. --up. .. ..n pp. --a. _ . . . .»_u_. ... .. n -. .-. .. nu . .. -_»-mi--. -.. 1 . i . —u. . ii. ma. ii. .. . . h . ..n-. - . . i. . . ... ... _—-. ... -_. .-¢n. .¢_ -.4.. . . . . —«. Ill . _a——-. -u—. .po-nu. o-. -o. I-Iunnlu . ».. .—. ... _ . _i 1 T -—r-. —n-I-«ii-1‘-n ii w u~-I v. .. . -u. varnueu — - -. u—n. a.ii I : :'_'. .‘: .?; :.: :‘: :.. .- iiitvisi. -i. -—i, ' -ammzuu-r—-. n—nu. . . .a. .w. nun. .. ,.. ..v --. . -4-nu no. .. . .-. . . .». nus-yr». ---u. ».. ..-. .. i. ... . . - -nun: -aux <1! . -.. . uan-a. -uiir - . -ii-. ... -uuaw. -. ..a . .u. m—o. —-ii . ... u re can -4-u. -p-n. ¢e-an. l . ..i y. on-H -. -_ now —. u, 1-" . . gun -1-. ‘-u. —-nun: --. -~-- -. dune -c-. —.—u upcu. i-« -gs-nu-—. -. xnn-nu. ups -u—-an-—. .. um . - i. .i. -. —u. —-am g. -»— naug- u. ... ... .. -on-up-. L-5-nu. vu-v. --new I . .n. - nu-one‘ . ... n-o. --wnpm ; -g. u.n. .-an-e. names. .4- . ... . -: --n. . .. . .. . -. a. .. I. .." and I ) . .—. .. . .u. ... ... , _ n-an-“.4.. .-ii . -.. . . ... mp-ii
  15. 15. spent 1: e unifying pletfonn In the same framework: Batch processing Real-time processing Analytics using SQL Machine leaming Graph algorithms
  16. 16. ,A| c|fl"l~, l , a‘s. .'A_'-iu_--ag _ u I u u ' -' : au———n. /I. uuw aa—. a—‘——'bv: &aIbv. sa—.4 fig Serving layer at Speed layer <: > / Data Queries
  17. 17. e r? » «'51 i@ etparfllt has gene nfreinstreant Large community Used in many companies In all major Hadoop distributions Many packages and connectors Conferences and certifications IBM recently pledged to commit 3000 developers solely to Spark
  18. 18. Ef§®lr""t; l' die yen use it“? *5 Install on commodity hardware e Configure and start the cluster Write an application (or use Spark shell) e Execute application on the cluster 6 But not just Spark cluster: also runs on YARN also runs on Mesos
  19. 19. _ —- _. _ ‘, n . .‘: .I- , . . ,_ ~ . . I — . . , ._v —. 1 . _ - . I ‘ I : - Executor JVM Executor JVM Driver JVM heap Scheduler ’ I I Executor JVM spam application Executor JVM . CID l: ll: l
  20. 20. Resilient Distributed Dataset Represents distributed collections create new RDDS execute RDDS and get results Example: val rdd = sc. textFile(”/ scmexfile. txt”) val cnt = rdd. f1atMap(line = > 1ine. split(” ”)). map(word = > (word, 1)). reduceByKey(_ + _) cnt. co1lect()
  21. 21. .. ; ii can you die ririitii Bpaiit‘?
  22. 22. spark eempeiients _ Spark Spark MLllb '. ~1?i-t-Maillot, -i Hf’ Streaming V """""' I I Spark Graphx Spark Core Spark SQL Graph RDD I ROD I I DataFrame I . . '3“. A I . I '. ‘:! .l. l': ~_~: w". <ta. ~.~, i.~ Q-r; _iih»,1:! :i. ts, <,-4_i. ~‘
  23. 23. Spark Core Work with RDDS Load, save, parse raw data Transform, aggregate, join data sets Communicate with executors
  24. 24. SQL *2 Work with DaiaFIa111eS Handle structured data, organized in columns Load/ save structured data from/ to external data sources re Transform, aggregate, join data using SQL is Query data through a IDBC server
  25. 25. Speck Streaming Handle real—time data Use the same API as batch jobs Latency from half a second and up Connectors to Kaflca, Flume, etc.
  26. 26. Machine learning algorithms Classification, regression, clustering, PCA, TF—IDF, Neural networks under development
  27. 27. Spark Vertices and edges Graph algorithms Page rank, connected components Triangle count, shortest paths
  28. 28. Went to lmow more’? http: //wvvw. meetup. com/ Apache- Spark-Zagreb-Meetup
  29. 29. 39% off I Discount code: zecevic39 http: //bit. |y/ sparkinaction
  30. 30. est is 9 g‘: _J~. .- -.4»-_ -'_~. ../ 8
  31. 31. Tiidliii ‘HIE? iatnirisiiii no r tovnil

×