SlideShare a Scribd company logo
1 of 4
Big Data
Technologies
 Apache HBase
 Apache Hive
 Apache Hadoop
 MapReduce
 Spring XD
 Apache Sqoop
 Apache Spark (Streaming/Mlib) using Java
 Apache Kafka
 Apache Solr
 Unix Shell Scripting
 Apache Zookeeper
 Apache Oozie
 Machine Learning
Projects/Proof of Concepts
 Import of structured big data to HBase using Hadoop MapReduce,SpringXD ,Sqoop etc.
 Import of unstructured big data to HDFS using Hadoop API, Spark Streaming, WebHDFS,SpringXD
etc.
 Realtime ingestion of structured & unstructured data from GreenPlum,Twitter etc onto HDFS using
Spark Streaming,SpringXD,Apache Kafka. Display of realtime ingestion graph using WebSockets API
on springboot application.
 Hive-HBase integration. This feature allows Hive QL statements to access HBase tables for both read
(SELECT) and write (INSERT). It is even possible to combine access to HBase tables with native Hive
tables via joins and unions.
 Big Data indexing & search(structured & unstructured) using Apache Solr
 Big Data Growth Monitoring and Charting using Hadoop API and SpringBoot.
 Big Data Compression and Encryption
 Hadoop MapReduce Job Scheduling and Tracking using Oozie,CronTab.
 Reusable component created for Apache Solr to self-restart and email notification on failure.
 Reusable component created to monitor status of SolrCloud using SpringBoot-AngularJs application
and to start/stop Solr Nodes and ZooKeeper
 Multi-Node SolrCloud creation and administration.
Projects/Proof of Concepts
 Apache Hadoop Installation on Windows 8.1 Operating System by building Hadoop Binaries from
source code using Maven
 Created Hadoop Cluster by integrating different machines running on windows and unix operating
systems and then used this cluster to run map-reduce jobs.
 Apache Spark Installation on top of Hadoop on Windows 8.1 Operating System.
 Machine Learning Algorithms(Linear Regression,Logistic Regression etc) implemented using Apache
Spark MLib

More Related Content

What's hot

Resume_Shivam_08072016
Resume_Shivam_08072016Resume_Shivam_08072016
Resume_Shivam_08072016Shivam Tyagi
 
Big data introduction (HackTM 2016)
Big data introduction (HackTM 2016)Big data introduction (HackTM 2016)
Big data introduction (HackTM 2016)Moldovan Radu Adrian
 
Hunk - Unlocking the Power of Big Data
Hunk - Unlocking the Power of Big DataHunk - Unlocking the Power of Big Data
Hunk - Unlocking the Power of Big DataSplunk
 
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCamp
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCampSteve Woolege Of Aster Data Gives Lightning Talk At BigDataCamp
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCampBigDataCamp
 
Big Data Analytics for Non-Programmers
Big Data Analytics for Non-ProgrammersBig Data Analytics for Non-Programmers
Big Data Analytics for Non-ProgrammersEdureka!
 
Hadoop - A Very Short Introduction
Hadoop - A Very Short IntroductionHadoop - A Very Short Introduction
Hadoop - A Very Short Introductiondewang_mistry
 
Building Scalable Big Data Pipelines
Building Scalable Big Data PipelinesBuilding Scalable Big Data Pipelines
Building Scalable Big Data PipelinesChristian Gügi
 
Streaming API, Spark and Ruby
Streaming API, Spark and RubyStreaming API, Spark and Ruby
Streaming API, Spark and RubyManohar Amrutkar
 
Using HBase for Real Time Access
Using HBase for Real Time AccessUsing HBase for Real Time Access
Using HBase for Real Time AccessRahul Gaikwad
 
Big Data Landscape 2019
Big Data Landscape 2019Big Data Landscape 2019
Big Data Landscape 2019QAware GmbH
 
Big Data Landscape 2019
Big Data Landscape 2019Big Data Landscape 2019
Big Data Landscape 2019QAware GmbH
 
Lighthouse - an open-source library to build data lakes - Kris Peeters
Lighthouse - an open-source library to build data lakes - Kris PeetersLighthouse - an open-source library to build data lakes - Kris Peeters
Lighthouse - an open-source library to build data lakes - Kris PeetersData Science Leuven
 

What's hot (20)

Big data advanced topics - part I
Big data   advanced topics - part IBig data   advanced topics - part I
Big data advanced topics - part I
 
Resume_Shivam_08072016
Resume_Shivam_08072016Resume_Shivam_08072016
Resume_Shivam_08072016
 
Apache spark
Apache sparkApache spark
Apache spark
 
Big data introduction (HackTM 2016)
Big data introduction (HackTM 2016)Big data introduction (HackTM 2016)
Big data introduction (HackTM 2016)
 
Hunk - Unlocking the Power of Big Data
Hunk - Unlocking the Power of Big DataHunk - Unlocking the Power of Big Data
Hunk - Unlocking the Power of Big Data
 
Hadoop An Introduction
Hadoop An IntroductionHadoop An Introduction
Hadoop An Introduction
 
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCamp
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCampSteve Woolege Of Aster Data Gives Lightning Talk At BigDataCamp
Steve Woolege Of Aster Data Gives Lightning Talk At BigDataCamp
 
Big Data Analytics for Non-Programmers
Big Data Analytics for Non-ProgrammersBig Data Analytics for Non-Programmers
Big Data Analytics for Non-Programmers
 
Big data & Hadoop
Big data & HadoopBig data & Hadoop
Big data & Hadoop
 
Hadoop - A Very Short Introduction
Hadoop - A Very Short IntroductionHadoop - A Very Short Introduction
Hadoop - A Very Short Introduction
 
Big Data and Hadoop Training in Chandigarh
Big Data and Hadoop Training in ChandigarhBig Data and Hadoop Training in Chandigarh
Big Data and Hadoop Training in Chandigarh
 
Apache Hadoop
Apache HadoopApache Hadoop
Apache Hadoop
 
Building Scalable Big Data Pipelines
Building Scalable Big Data PipelinesBuilding Scalable Big Data Pipelines
Building Scalable Big Data Pipelines
 
Streaming API, Spark and Ruby
Streaming API, Spark and RubyStreaming API, Spark and Ruby
Streaming API, Spark and Ruby
 
Using HBase for Real Time Access
Using HBase for Real Time AccessUsing HBase for Real Time Access
Using HBase for Real Time Access
 
Big Data - Part IV
Big Data - Part IVBig Data - Part IV
Big Data - Part IV
 
Big data and Hadoop
Big data and HadoopBig data and Hadoop
Big data and Hadoop
 
Big Data Landscape 2019
Big Data Landscape 2019Big Data Landscape 2019
Big Data Landscape 2019
 
Big Data Landscape 2019
Big Data Landscape 2019Big Data Landscape 2019
Big Data Landscape 2019
 
Lighthouse - an open-source library to build data lakes - Kris Peeters
Lighthouse - an open-source library to build data lakes - Kris PeetersLighthouse - an open-source library to build data lakes - Kris Peeters
Lighthouse - an open-source library to build data lakes - Kris Peeters
 

Viewers also liked

SockJS Intro
SockJS IntroSockJS Intro
SockJS IntroNgoc Dao
 
Spring Boot & WebSocket
Spring Boot & WebSocketSpring Boot & WebSocket
Spring Boot & WebSocketMing-Ying Wu
 

Viewers also liked (20)

SockJS Intro
SockJS IntroSockJS Intro
SockJS Intro
 
Javantura v4 - DMN – supplement your BPMN - Željko Šmaguc
Javantura v4 - DMN – supplement your BPMN - Željko ŠmagucJavantura v4 - DMN – supplement your BPMN - Željko Šmaguc
Javantura v4 - DMN – supplement your BPMN - Željko Šmaguc
 
Javantura v4 - CroDuke Indy and the Kingdom of Java Skills - Branko Mihaljevi...
Javantura v4 - CroDuke Indy and the Kingdom of Java Skills - Branko Mihaljevi...Javantura v4 - CroDuke Indy and the Kingdom of Java Skills - Branko Mihaljevi...
Javantura v4 - CroDuke Indy and the Kingdom of Java Skills - Branko Mihaljevi...
 
Javantura v4 - Java or Scala – Web development with Playframework 2.5.x - Kre...
Javantura v4 - Java or Scala – Web development with Playframework 2.5.x - Kre...Javantura v4 - Java or Scala – Web development with Playframework 2.5.x - Kre...
Javantura v4 - Java or Scala – Web development with Playframework 2.5.x - Kre...
 
Javantura v4 - FreeMarker in Spring web - Marin Kalapać
Javantura v4 - FreeMarker in Spring web - Marin KalapaćJavantura v4 - FreeMarker in Spring web - Marin Kalapać
Javantura v4 - FreeMarker in Spring web - Marin Kalapać
 
Javantura v4 - JVM++ The GraalVM - Martin Toshev
Javantura v4 - JVM++ The GraalVM - Martin ToshevJavantura v4 - JVM++ The GraalVM - Martin Toshev
Javantura v4 - JVM++ The GraalVM - Martin Toshev
 
Javantura v4 - Getting started with Apache Spark - Dinko Srkoč
Javantura v4 - Getting started with Apache Spark - Dinko SrkočJavantura v4 - Getting started with Apache Spark - Dinko Srkoč
Javantura v4 - Getting started with Apache Spark - Dinko Srkoč
 
Javantura v4 - Let me tell you a story why Scrum is not for you - Roko Roić
Javantura v4 - Let me tell you a story why Scrum is not for you - Roko RoićJavantura v4 - Let me tell you a story why Scrum is not for you - Roko Roić
Javantura v4 - Let me tell you a story why Scrum is not for you - Roko Roić
 
Javantura v4 - Support SpringBoot application development lifecycle using Ora...
Javantura v4 - Support SpringBoot application development lifecycle using Ora...Javantura v4 - Support SpringBoot application development lifecycle using Ora...
Javantura v4 - Support SpringBoot application development lifecycle using Ora...
 
Javantura v4 - The power of cloud in professional services company - Ivan Krn...
Javantura v4 - The power of cloud in professional services company - Ivan Krn...Javantura v4 - The power of cloud in professional services company - Ivan Krn...
Javantura v4 - The power of cloud in professional services company - Ivan Krn...
 
Javantura v4 - KumuluzEE – Microservices with Java - Matjaž B. Jurič & Tilen ...
Javantura v4 - KumuluzEE – Microservices with Java - Matjaž B. Jurič & Tilen ...Javantura v4 - KumuluzEE – Microservices with Java - Matjaž B. Jurič & Tilen ...
Javantura v4 - KumuluzEE – Microservices with Java - Matjaž B. Jurič & Tilen ...
 
Javantura v4 - Test-driven documentation with Spring REST Docs - Danijel Mitar
Javantura v4 - Test-driven documentation with Spring REST Docs - Danijel MitarJavantura v4 - Test-driven documentation with Spring REST Docs - Danijel Mitar
Javantura v4 - Test-driven documentation with Spring REST Docs - Danijel Mitar
 
Spring Boot & WebSocket
Spring Boot & WebSocketSpring Boot & WebSocket
Spring Boot & WebSocket
 
Javantura v4 - Angular2 - Ionic2 - from birth to stable versions - Hrvoje Pek...
Javantura v4 - Angular2 - Ionic2 - from birth to stable versions - Hrvoje Pek...Javantura v4 - Angular2 - Ionic2 - from birth to stable versions - Hrvoje Pek...
Javantura v4 - Angular2 - Ionic2 - from birth to stable versions - Hrvoje Pek...
 
Javantura v4 - Spring Boot and JavaFX - can they play together - Josip Kovaček
Javantura v4 - Spring Boot and JavaFX - can they play together - Josip KovačekJavantura v4 - Spring Boot and JavaFX - can they play together - Josip Kovaček
Javantura v4 - Spring Boot and JavaFX - can they play together - Josip Kovaček
 
Javantura v4 - True RESTful Java Web Services with JSON API and Katharsis - M...
Javantura v4 - True RESTful Java Web Services with JSON API and Katharsis - M...Javantura v4 - True RESTful Java Web Services with JSON API and Katharsis - M...
Javantura v4 - True RESTful Java Web Services with JSON API and Katharsis - M...
 
Javantura v4 - What’s NOT new in modular Java - Milen Dyankov
Javantura v4 - What’s NOT new in modular Java - Milen DyankovJavantura v4 - What’s NOT new in modular Java - Milen Dyankov
Javantura v4 - What’s NOT new in modular Java - Milen Dyankov
 
Javantura v4 - Security architecture of the Java platform - Martin Toshev
Javantura v4 - Security architecture of the Java platform - Martin ToshevJavantura v4 - Security architecture of the Java platform - Martin Toshev
Javantura v4 - Security architecture of the Java platform - Martin Toshev
 
Javantura v4 - Java and lambdas and streams - are they better than for loops ...
Javantura v4 - Java and lambdas and streams - are they better than for loops ...Javantura v4 - Java and lambdas and streams - are they better than for loops ...
Javantura v4 - Java and lambdas and streams - are they better than for loops ...
 
Javantura v4 - (Spring)Boot your application on Red Hat middleware stack - Al...
Javantura v4 - (Spring)Boot your application on Red Hat middleware stack - Al...Javantura v4 - (Spring)Boot your application on Red Hat middleware stack - Al...
Javantura v4 - (Spring)Boot your application on Red Hat middleware stack - Al...
 

Similar to Big Data - Linked In_DEEPU

12 SQL On-Hadoop Tools
12 SQL On-Hadoop Tools12 SQL On-Hadoop Tools
12 SQL On-Hadoop ToolsXplenty
 
Windows Azure HDInsight Service
Windows Azure HDInsight ServiceWindows Azure HDInsight Service
Windows Azure HDInsight ServiceNeil Mackenzie
 
Overview of big data & hadoop version 1 - Tony Nguyen
Overview of big data & hadoop   version 1 - Tony NguyenOverview of big data & hadoop   version 1 - Tony Nguyen
Overview of big data & hadoop version 1 - Tony NguyenThanh Nguyen
 
Overview of Big data, Hadoop and Microsoft BI - version1
Overview of Big data, Hadoop and Microsoft BI - version1Overview of Big data, Hadoop and Microsoft BI - version1
Overview of Big data, Hadoop and Microsoft BI - version1Thanh Nguyen
 
Intro to Hybrid Data Warehouse
Intro to Hybrid Data WarehouseIntro to Hybrid Data Warehouse
Intro to Hybrid Data WarehouseJonathan Bloom
 
Big Data and Hadoop Components
Big Data and Hadoop ComponentsBig Data and Hadoop Components
Big Data and Hadoop ComponentsDezyreAcademy
 
Big Data Technology Stack : Nutshell
Big Data Technology Stack : NutshellBig Data Technology Stack : Nutshell
Big Data Technology Stack : NutshellKhalid Imran
 
Brief Introduction about Hadoop and Core Services.
Brief Introduction about Hadoop and Core Services.Brief Introduction about Hadoop and Core Services.
Brief Introduction about Hadoop and Core Services.Muthu Natarajan
 
The other Apache technologies your big data solution needs!
The other Apache technologies your big data solution needs!The other Apache technologies your big data solution needs!
The other Apache technologies your big data solution needs!gagravarr
 
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdf
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdfApache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdf
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdfMounikaPolabathina
 
a9TD6cbzTZotpJihekdc+w==.docx
a9TD6cbzTZotpJihekdc+w==.docxa9TD6cbzTZotpJihekdc+w==.docx
a9TD6cbzTZotpJihekdc+w==.docxVasimMemon4
 
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...DataWorks Summit/Hadoop Summit
 
Srikanth hadoop 3.6yrs_hyd
Srikanth hadoop 3.6yrs_hydSrikanth hadoop 3.6yrs_hyd
Srikanth hadoop 3.6yrs_hydsrikanth K
 

Similar to Big Data - Linked In_DEEPU (20)

Intro to Hadoop
Intro to HadoopIntro to Hadoop
Intro to Hadoop
 
12 SQL On-Hadoop Tools
12 SQL On-Hadoop Tools12 SQL On-Hadoop Tools
12 SQL On-Hadoop Tools
 
Windows Azure HDInsight Service
Windows Azure HDInsight ServiceWindows Azure HDInsight Service
Windows Azure HDInsight Service
 
Overview of big data & hadoop version 1 - Tony Nguyen
Overview of big data & hadoop   version 1 - Tony NguyenOverview of big data & hadoop   version 1 - Tony Nguyen
Overview of big data & hadoop version 1 - Tony Nguyen
 
Overview of Big data, Hadoop and Microsoft BI - version1
Overview of Big data, Hadoop and Microsoft BI - version1Overview of Big data, Hadoop and Microsoft BI - version1
Overview of Big data, Hadoop and Microsoft BI - version1
 
Intro to Hybrid Data Warehouse
Intro to Hybrid Data WarehouseIntro to Hybrid Data Warehouse
Intro to Hybrid Data Warehouse
 
Introduction to sqoop
Introduction to sqoopIntroduction to sqoop
Introduction to sqoop
 
Poorna Hadoop
Poorna HadoopPoorna Hadoop
Poorna Hadoop
 
Hive with HDInsight
Hive with HDInsightHive with HDInsight
Hive with HDInsight
 
Big Data and Hadoop Components
Big Data and Hadoop ComponentsBig Data and Hadoop Components
Big Data and Hadoop Components
 
Big Data Technology Stack : Nutshell
Big Data Technology Stack : NutshellBig Data Technology Stack : Nutshell
Big Data Technology Stack : Nutshell
 
Brief Introduction about Hadoop and Core Services.
Brief Introduction about Hadoop and Core Services.Brief Introduction about Hadoop and Core Services.
Brief Introduction about Hadoop and Core Services.
 
The other Apache technologies your big data solution needs!
The other Apache technologies your big data solution needs!The other Apache technologies your big data solution needs!
The other Apache technologies your big data solution needs!
 
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdf
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdfApache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdf
Apache Spark vs. Hadoop Is Spark Set to Replace Hadoop.pdf
 
Introduction to Hive
Introduction to HiveIntroduction to Hive
Introduction to Hive
 
a9TD6cbzTZotpJihekdc+w==.docx
a9TD6cbzTZotpJihekdc+w==.docxa9TD6cbzTZotpJihekdc+w==.docx
a9TD6cbzTZotpJihekdc+w==.docx
 
Hive and querying data
Hive and querying dataHive and querying data
Hive and querying data
 
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...
Apache HBase + Spark: Leveraging your Non-Relational Datastore in Batch and S...
 
Srikanth hadoop 3.6yrs_hyd
Srikanth hadoop 3.6yrs_hydSrikanth hadoop 3.6yrs_hyd
Srikanth hadoop 3.6yrs_hyd
 
Hadoop vs Apache Spark
Hadoop vs Apache SparkHadoop vs Apache Spark
Hadoop vs Apache Spark
 

Big Data - Linked In_DEEPU

  • 2. Technologies  Apache HBase  Apache Hive  Apache Hadoop  MapReduce  Spring XD  Apache Sqoop  Apache Spark (Streaming/Mlib) using Java  Apache Kafka  Apache Solr  Unix Shell Scripting  Apache Zookeeper  Apache Oozie  Machine Learning
  • 3. Projects/Proof of Concepts  Import of structured big data to HBase using Hadoop MapReduce,SpringXD ,Sqoop etc.  Import of unstructured big data to HDFS using Hadoop API, Spark Streaming, WebHDFS,SpringXD etc.  Realtime ingestion of structured & unstructured data from GreenPlum,Twitter etc onto HDFS using Spark Streaming,SpringXD,Apache Kafka. Display of realtime ingestion graph using WebSockets API on springboot application.  Hive-HBase integration. This feature allows Hive QL statements to access HBase tables for both read (SELECT) and write (INSERT). It is even possible to combine access to HBase tables with native Hive tables via joins and unions.  Big Data indexing & search(structured & unstructured) using Apache Solr  Big Data Growth Monitoring and Charting using Hadoop API and SpringBoot.  Big Data Compression and Encryption  Hadoop MapReduce Job Scheduling and Tracking using Oozie,CronTab.  Reusable component created for Apache Solr to self-restart and email notification on failure.  Reusable component created to monitor status of SolrCloud using SpringBoot-AngularJs application and to start/stop Solr Nodes and ZooKeeper  Multi-Node SolrCloud creation and administration.
  • 4. Projects/Proof of Concepts  Apache Hadoop Installation on Windows 8.1 Operating System by building Hadoop Binaries from source code using Maven  Created Hadoop Cluster by integrating different machines running on windows and unix operating systems and then used this cluster to run map-reduce jobs.  Apache Spark Installation on top of Hadoop on Windows 8.1 Operating System.  Machine Learning Algorithms(Linear Regression,Logistic Regression etc) implemented using Apache Spark MLib