The document provides instructions for installing Hadoop on Ubuntu 16. It involves installing OpenJDK, generating SSH keys, downloading and extracting Hadoop, configuring environment variables, and modifying configuration files like core-site.xml and hdfs-site.xml to configure the namenode and datanode. The instructions also cover formatting the namenode, starting the Hadoop processes, and accessing the web interface.
Training on DSpace Institutional Repository
Organized by
BALID Institute of Information Management (BIIM
DSpace Manual for BALID Trainee
Institutional Repository
1-2 May 2014
Venue: CIRDAP
• Installation of DSpace on Debian
• Configuration of DSpace
• Customization of Dspace
• Cron Jobs setup for production system
• MTA Setup for DSpace
• Some Important Commands of PostgreSQL
• DSpace Discovery Setup
Prepared By
Nur Ahammad
Junior Assistant Librarian
Independent University, Bangladesh
linux-commandline-magic-Joomla-World-Conference-2014Peter Martin
The Linux command line is a powerful tool. The majority of webservers run on Linux/Unix. Some hosting companies offer SSH access to their hosting environment. Via SSH you can login and use the Linux command line.
In this presentation, Peter will show you some time-saving command line commands that you can use for certain tasks on your Joomla websites:
some basic SSH command line commands
Analyzing & recovering a hacked website
Backup a website or move it to another server
Finding unused files
Setting up a test environment
Training on DSpace Institutional Repository
Organized by
BALID Institute of Information Management (BIIM
DSpace Manual for BALID Trainee
Institutional Repository
1-2 May 2014
Venue: CIRDAP
• Installation of DSpace on Debian
• Configuration of DSpace
• Customization of Dspace
• Cron Jobs setup for production system
• MTA Setup for DSpace
• Some Important Commands of PostgreSQL
• DSpace Discovery Setup
Prepared By
Nur Ahammad
Junior Assistant Librarian
Independent University, Bangladesh
linux-commandline-magic-Joomla-World-Conference-2014Peter Martin
The Linux command line is a powerful tool. The majority of webservers run on Linux/Unix. Some hosting companies offer SSH access to their hosting environment. Via SSH you can login and use the Linux command line.
In this presentation, Peter will show you some time-saving command line commands that you can use for certain tasks on your Joomla websites:
some basic SSH command line commands
Analyzing & recovering a hacked website
Backup a website or move it to another server
Finding unused files
Setting up a test environment
Huippuvalmennus kasvuhakuisille pk-yrityksilleAava & Bang
Kasvumarkkinointi on Mainostoimisto Aava & Bang Oy:n kehittämä markkinoinnin ajattelu- ja työskentelytapa, joka tähtää aina ensisijaisesti kasvuun kohti selkeästi asetettuja ja mitattavia päämääriä
Hadoop installation on windows using virtual box and also hadoop installation on ubuntu
http://logicallearn2.blogspot.in/2018/01/hadoop-installation-on-ubuntu.html
Hands on Docker - Launch your own LEMP or LAMP stackDana Luther
In this tutorial we will go over setting up a standard LEMP stack for development use and learn how to modify it to mimic your production/pre-production environments as closely as possible. We will go over how to switch from Nginx to Apache, upgrade PHP versions and introduce additional storage engines such as Redis to the equation. We’ll also step through how to run both unit and acceptance suites using headless Selenium images in the stack. Leave here fully confident in knowing that whatever environment you get thrown into, you can replicate it and work in it comfortably.
Installation and setup hadoop publishedDipendra Kusi
Here you will learn how to setup Apache Hadoop for Big Data and learn the basic script like word count in Apache Hadoop in Cloudera environment. Further more you will learn how to create jar file in eclipse that can be run in Hadoop framework and required library for it.
The title "Big Data using Hadoop.pdf" suggests that the document is likely a PDF file that focuses on the utilization of Hadoop technology in the context of Big Data. Hadoop is a popular open-source framework for distributed storage and processing of large datasets. The document is expected to cover various aspects of working with big data, emphasizing the role of Hadoop in managing and analyzing vast amounts of information.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
2. Install Java
Do I have Java? Type on terminal: java -version
If I see the output below, then I don’t have java installed, follow instructions next
slide
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
2
3. Install Java
Type:
sudo apt-get install openjdk-8-jdk
Type Y to continue the installation process (it will take a while to complete the
installation)
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
3
4. Do I have java?
To confirm java ins installed on my Ubuntu system type:
java –version
You will see output below
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
4
5. Install Openssh
Is mandatory to install openssh server:
sudo apt-get install openssh-server
If ssh server is installed then
generate keys, run command below:
ssh-keygen -t rsa
Enter file, press enter
Enter passphrase, press enter
Enter same passphrase again press
enter
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
5
6. SSH Keys
Now we will copy the key to the user and host, in my case my user is hadoop and
host is hadoopdev
ssh-copy-id hadoop@hadoopdev
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
6
8. Download Apache Hadoop
Type in the terminal the following command to create new folder within my home
linux folder, in this case/home/Hadoop/:
mkdir hadoop_install
Then go into this new folder:
cd hadoop_install
And copy the command below:
wget http://www-eu.apache.org/dist/hadoop/common/hadoop-2.7.3/hadoop-
2.7.3.tar.gz
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
8
9. Download Apache Hadoop
You will see windows reflecting the progress of the download
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
9
10. Unzip Hadoop folder
Once download is complete
Type the following command:
tar -xvf hadoop-2.7.3.tar.gz
Now you will see 2 folders, the new directory is called hadoop-2.7.3:
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
10
11. Setup bashrc
This is the java location (very important for next steps):
Edit bashrc
Type:
Sudo gedit ~/.bashrc
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
11
12. Setup ~/.bashrc
Add this lines to the .bashrc
Pls note on previous slide the java path is displayed, need to point bashrc to the
actual java path
#HADOOP VARIABLES START
export JAVA_HOME=/usr/lib/jvm/ java-1.8.0-openjdk-amd64
export HADOOP_INSTALL=/home/hadoop/hadoop_install
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
12
13. Testing hadoop installation
Type the following command to refresh ~/.bashrc changes (no need to restart)
source ~/.basrch
Type the command below (if at this point you see an output like this you’re
doing well)
hadoop version
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
13
15. Point your java to hadoop conf file
Go to the path:
/home/hadoop/hadoop_install/hadoop-2.7.3/etc/hadoop
Edit the file:
sudo gedit Hadoop-env.sh
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
15
16. Modifying hadoop-env.sh
Modify the value for Java Home in the file: hadoop-env.sh
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
16
17. Modify core-site.xml
Create a folder called tmp in /home/hadoop/hadoop_install
Add the following text to the core-site.xml , file is on the path:
/home/hadoop/hadoop_install/hadoop-2.7.3/etc/hadoop
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hadoop_install/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
<description>The name of the default file system.</description>
</property>
</configuration>
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
17
18. Modify mapred-site.xml
By default there is a file called: mapred-site.xml.template, needs to be renamed to
mapred-site.xml and then add the code below:
File is on path: /home/hadoop/hadoop_install/hadoop-2.7.3/etc/hadoop
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:54311</value>
<description>The host and port that the MapReduce job tracker runs at. </description>
</property>
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
18
19. Modify hdfs-site.xml
We need to créate 2 new folders which will contain name node and data node:
I placed these 2 folders on: /home/hadoop/hadoop_install/
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
19
20. Modify hdfs-site.xml
Add the code below in the file hdfs-site.xml, the paths for namnode and datanode are the 2 new folders
you just created on previous slide.
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///home/hadoop/hadoop_install/namenode</value>
</property>
<property>
<name>dfs.data.node.name.dir</name>
<value>file:///home/hadoop/hadoop_install/datanode</value>
</property>
</configuration>
#hdfs-site.xml is located on the path: /home/hadoop/hadoop_install/hadoop-2.7.3/etc/hadoop
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
20
21. Format the namenode
Run the following command:
hadoop namenode –format
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
21
22. Format the namenode part 2
If everything is ok you will see message below:
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
22
23. Running Hadoop Single node
Run the command:
startall.sh
Then execute the command:
jps, you will see the following output
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
23
24. Stop Cluster
We run stop-all.sh
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
24
25. Web Interface: localhost:50070
In the browser go to: localhost:50070
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
25
26. Applies for:
This installation runs under:
Ubuntu 16
Hadoop 2.7.3
Virtual Machine:
2 Processors
2 Gb Ram
2 Network Interface, 1 as Bridge, 2nd as Nat
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
26
27. You need help?
Contact name:
Enrique Davila Gutierrez
Enrique.davila@Gmail.com
10/24/2016Enrique Davila Big Data Instructor enrique.davila@gmail.com
27
