Nodejs.meetup

NYC Data Science Academy
Node.JS in Action
(Copyright materials, all right reserved)
Sign up for Node.js beginner class at www.nycdatascience.com
or email vivian.zhang@supstat.com

Overview
 Deployment of Node.js on AWS EC2 Instance
 Use Bootstrap Framework for our page
 Rules of Routes, Session and MongoDB, Error Handling and Visiting Control
 Web Scrapper
 Save JSON data into MongoDB

AWS Security Group
 You should Add HTTP Rules
 For convenience, you could allow All TCP rules

Installation on Ubuntu
 Our machine is Ubuntu 12.04 LTS, 64 bits
 We install node and express from apt-get
 Check if we are having the right configuration
$ sudo add-apt-repository ppa:chris-lea/node.js
$ sudo apt-get update
$ sudo apt-get install nodejs
$ sudo npm install express@3 -g
$ sudo apt-get install unzip
$ sudo apt-get install build-essential
$ node -v
v0.10.26
$ npm -v
1.4.3
$ express -V
3.5.1

Start Our first Project
 In ~/ directory, we can start our first express project
 If you see the notification, then it is OK!
 Visit your public address on port 3000 and you will see
$ express -e nodejs-demo
$ cd nodejs-demo
$ sudo npm install
$ node app.js

Supervisor
 If we modify our code, we need to restart our server. That is inconvenient.
 By supervisor, we can modify our code and the server will restart automatically.
$ sudo npm install supervisor -g
$ supervisor app.js
Running node-supervisor with
program 'app.js'
--watch '.'
--extensions 'node,js'
--exec 'node'
Starting child process with 'node app.js'
Watching directory '/home/ubuntu/nodejs-demo' for changes.
Express server listening on port 3000

Intro to the working directory
 node_modules: Store all the dependent libraries(every project manage its own
dependencies).
 package.json: Project dependencies configuration and developer information
 app.js: Application starting file
 public: Static files(css,js,img)
 routes: Routes files(C in MVC, controller)
 Views: Page files(Ejs template)

Support html files
 And rename views/index.ejs to views/index.html
 Add the following lines in app.js, see next page for codes
 Visit your website
$ mv views/index.ejs views/index.html

Support html files
var express = require('express');
var routes = require('./routes');
var user = require('./routes/user');
var http = require('http');
var path = require('path');
var ejs = require('ejs');
var app = express();
// all environments
app.set('port', process.env.PORT || 3000);
app.set('views', path.join(__dirname, 'views'));
// app.set('view engine', ‟ejs');
app.engine('.html', ejs.__express);
app.set('view engine', 'html');
app.use(express.favicon());
app.use(express.logger('dev'));

Use Bootstrap Framework
 Bootstrap Framework is one of the most popular frameworks for front-end.
 Download it from http://getbootstrap.com/ and unzip.
 And we also need jQuery.
 Check whether your “~/nodejs-demo/public/stylesheets” has three .css files and
whether „~/nodejs-demo/public/javascripts‟ has two .js files.
$ cd ~/
$ wget https://github.com/twbs/bootstrap/releases/download/v3.1.1/bootstrap-
3.1.1-dist.zip
$ unzip bootstrap-3.1.1-dist.zip
$ cd ~/bootstrap-3.1.1-dist
$ cp css/*.min.css ~/nodejs-demo/public/stylesheets/
$ cd js/
$ wget http://code.jquery.com/jquery-1.9.1.min.js
$ cp *.min.js ~/nodejs-demo/public/javascripts
$ cd ~/nodejs-demo
$ ls public/javascripts
$ ls public/stylesheets

Split index.html
 We want a common header and footer by spliting index.html into header.html,
index.html and footer.html
 First, we add these two news files in foldr views/

header.html
 We edit the header.html like this
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title><%=: title %></title>

<link href="/stylesheets/bootstrap.min.css" rel="stylesheet" media="screen">
<!-- <link href="css/bootstrap-responsive.min.css" rel="stylesheet" media="screen"> --
>
</head>
<body screen_capture_injected="true">
$ vi ~/nodejs-demo/views/header.html

footer.html
 We edit the footer.html like this
<script src="/javascripts/jquery-1.9.1.min.js">
</script>
<script src="/javascripts/bootstrap.min.js">
</script>
</body>
</html>
$ vi ~/nodejs-demo/views/footer.html

index.html
 Through ejs(emmeded javascript), we can include our header and footer in
index.html
 modify index.html
<% include header.html %>
<h1><%= title %>
</h1>
<p>Welcome to <%= title %>
</p>
<% include footer.html %>
$ rm ~/nodejs-demo/views/index.html
$ vi ~/nodejs-demo/views/index.html

Routes of our demo website
 Routes: pages, authorization and action
 / : index.html, No need to login to visit.
 /home : home.html, Need to login to visit.
 /login : login.html, redirect to home.html with correct username and password
 /logout : No corresponding html file
 redirect back to index.html after logout
 We can add routes to enable these features.

Add routes
 In app.js, we edit it by “vi ~/nodejs-demo/app.js”
 Note: app.get is get request, app.post is post request, get.all is request for this
path.
app.get('/', routes.index);
//app.get('/users', user.list);
app.get('/login', routes.login);
app.post('/login', routes.doLogin);
app.get('/logout', routes.logout);
app.get('/home', routes.home);

Add routes
 Then we edit routes/index.js by by “vi ~/nodejs-demo/routes/index.js”

Add routes
exports.index = function(req, res){
res.render('index', { title: 'Express' });
};
exports.login = function(req, res){
res.render('login', { title: 'User Login'});
};
exports.doLogin = function(req, res){
var user={
username:'admin',
password:'admin'
}
if(req.body.username===user.username && req.body.password===user.password){
res.redirect('/home');
}
res.redirect('/login');
};
exports.logout = function(req, res){
res.redirect('/');
};
exports.home = function(req, res){
var user={
username:'admin',
password:'admin'
}
res.render('home', { title: 'Home',user: user});
};

Add pages
 Create new home page by “vi ~/nodejs-demo/views/home.html” , see page 20
 Create new login page by “vi ~/nodejs-demo/views/login.html”, see page 21

Create home.html
<h1>Welcome <%= user.username %>!</h1>
<a class="btn" href="/logout">Logout</a>

Create login.html
<link href="/stylesheets/signin.css" rel="stylesheet">
<div class="container">
<form class="form-signin" method="post">
<h2 class="form-signin-heading">Please sign in</h2>
<input type="text" class="form-control" id="username" name="username"
required autofocus>
<input type="password" class="form-control" id="password"
name="password" required>
<button class="btn btn-lg btn-primary btn-block" type="submit">Sign
in</button>
</form>
</div>

Add pages
 And we edit index.html by „vi ~/nodejs-demo/views/index.html‟
<h1><%= title %>
</h1>
<p>Welcome to <%= title %>
<a href="/login">Signin</a>
</p>

Improve the look
 This is what we see at /login.html:
 Is there anyway we can make it better?

Edit the Bootstrap Style
 We create signin.css file to public/stylesheets/ by
 “vi ~/nodejs-demo/public/stylesheets/signin.css”
 Copy/past left side and right side into the file
body {
padding-top: 40px;
padding-bottom: 40px;
background-color: #eee;
}
.form-signin {
max-width: 330px;
padding: 15px;
margin: 0 auto;
}
.form-signin .form-signin-heading,
.form-signin .checkbox {
margin-bottom: 10px;
}
.form-signin .checkbox {
font-weight: normal;
}
.form-signin .form-control {
position: relative;
font-size: 16px;
height: auto;
padding: 10px;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;
}
.form-signin .form-control:focus {
z-index: 2;
}
.form-signin input[type="text"] {
margin-bottom: -1px;
border-bottom-left-radius: 0;
border-bottom-right-radius: 0;
}
.form-signin input[type="password"] {
margin-bottom: 10px;
border-top-left-radius: 0;
border-top-right-radius: 0;
}

Edit the Bootstrap Style
 Then the website looks like:
 Use admin and admin to login!

Session
 In “routes/index.js” file , when we call function “exports.doLogin”, if username and
password are correct, we will redirect to home.
 In the same file, when we call function “exports.home”, we render the page and pass
username to the home.html by the following specific:
 Why can't we assign the username to session so that we do not need pass to each
page again?
res.redirect('/home');
res.render('home', { title: 'Home',user: user});

Storage of data
 On local drive, we store user credentials into cookie files.
 On server side, we also store session and user credentials in database, like Redis,
MongoDB.
 MongoDB pass the username object to each page instead of asking node.js passing
the username object between pages.

Edit app.js
 The Order of Commands Matters !!!

Edit app.js
app.set('view engine', 'html');
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(express.json());
app.use(express.urlencoded());
app.use(express.methodOverride());
app.use(express.cookieParser());
app.use(express.cookieSession({secret :
'nycdatascience.com'}));
app.use(express.session({
secret : 'nycdatascience.com',
store: store,
cookie: { maxAge: 900000 }
}));
app.use(function(req, res, next){
res.locals.user = req.session.user;
next();
});
app.use(app.router);
app.use(express.static(path.join(__dirname, 'public')));
var express = require('express');
var routes = require('./routes');
var user = require('./routes/user');
var path = require('path');
var ejs = require('ejs');
var SessionStore = require("session-
mongoose")(express);
var store = new SessionStore({
url: "mongodb://localhost/session",
interval: 120000
});
var app = express();

MongoDB on Ubuntu
 Following the official document, we successfully installed MongoDB
 http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
$ echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' |
sudo tee /etc/apt/sources.list.d/mongodb.list
$ sudo apt-get update
$ sudo apt-get install mongodb-org
$ sudo service mongod start

Installing Mongoose
 We need to install Mongoose: how node.js connect with MongoDB
 Then /login page is accessible
$ sudo npm install session-mongoose
$ npm install mongoose

Edit index.js
 We need to edit index.js to support session by „vi ~/nodejs-demo/routes/index.js‟

Edit index.js
var user={
username:'admin',
password:'admin'
}
req.session.user=user;
return res.redirect('/home');
}
else{
return res.redirect('/login');
}
};
exports.logout = function(req, res){
req.session.user=null;
res.redirect('/');
};
exports.home = function(req, res){
//delete first 5 lines, change last line
res.render('home', { title: 'Home'});
}

Error Handling
 We want to tell user if they have entered the wrong information
 Edit app.js by „vi ~/nodejs-demo/app.js‟
app.use(function(req, res, next){
res.locals.user = req.session.user;
var err = req.session.error;
delete req.session.error;
res.locals.message = '';
if (err) res.locals.message = '<div class="alert alert-error">' + err +'</div>';
next();
});
app.use(express.static(path.join(__dirname, 'public')));

Error Handling
 Edit login.html by „vi ~/nodejs-demo/views/login.html‟
<link href="/stylesheets/signin.css" rel="stylesheet">
<div class="container">
<form class="form-signin" method="post">
<%- message %>
<h2 class="form-signin-heading">Please sign in</h2>
<input type="text" class="form-control" id="username" name="username" required autofocus>
<input type="password" class="form-control" id="password" name="password" required>
<button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button>
</form>
</div>

Error Handling
 Edit index.js by by „vi ~/nodejs-demo/routes/index.js‟
var user={
username:'admin',
password:'admin'
}
req.session.user=user;
return res.redirect('/home');
} else {
req.session.error='Wrong Username or Password!';
}
};

Error Handling
 Let's have a try:

Website safety
 Our website is almost ready, but it is not safe yet.
 We can access /home without login.

Refine model of our demo website
 /: anybody could visit
 Still remember app.get, app.post and app.all?
 /login: use app.all to take care of the requests of visiting /login. We call
notAuthentication function(If session.user is not null, then the user has logined in.)
first to check whether user's been logged in
 /logout: use app.get to take care of the requests of visiting /logout. We call
authentication function(If session.user is null, it will hint “please sign in”).
 /home: use app.get to take care of the requests of visiting /home. We call
authentication function(If session.user is null, it will hint “please sign in”).

Edit app.js
 We add two functions and make use of them.

Edit app.js
app.use(express.static(path.join(__dirname,
'public')));
function authentication(req, res, next) {
if (!req.session.user) {
req.session.error='Please Sign In to
Visit';
}
next();
}
function notAuthentication(req, res, next) {
if (req.session.user) {
req.session.error='Logged in';
return res.redirect(‟/home‟);
}
next();
}
// development only
if ('development' == app.get('env')) {
app.use(express.errorHandler());
}
app.get('/', routes.index);
//app.get('/users', user.list);
app.all('/login', notAuthentication);
app.get('/login', routes.login);
app.post('/login', routes.doLogin);
app.get('/logout', authentication);
app.get('/logout', routes.logout);
app.get('/home', authentication);
app.get('/home', routes.home);

Safety check
 Let's check if it works.
 We have done it!
 Go to /home and /logout page while you are not logined in
 Go to login page while you are logined in

Add some content?
 So, how to add content to our database?
 Why not use informations from other websites?

Install libraries
 We need to install jsdom, jQuery, xmlhttprequest, request, htmlparser
$ sudo npm install jsdom
$ sudo npm install jQuery
$ sudo npm install xmlhttprequest
$ sudo npm install request
$ sudo npm install htmlparser

What is valuable?
 Our event webpage!
 http://www.meetup.com/NYC-Open-Data/events/163300552/

Add codes
 Add myUtil.js by „vi ~/nodejs_demo/myUtil.js‟
 It is used as getter function to scrap certain web page.
var MyUtil = function () {
};
var request = require('request');
MyUtil.prototype.get=function(url,callback){
request(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
callback(body,response.statusCode);
}
})
}
module.exports = new MyUtil();

Add codes
 And edit index.js
 Add the following codes
in the end.
var myUtil = require('../myUtil.js');
var $ = require('jQuery');
var url="http://www.meetup.com/NYC-Open-Data/events/163300552/";
console.log(url);
myUtil.get(url,function(content,status){
console.log("status:="+status);
res.send(content);
});
};

Content Downloaded
 Then we visit the index page:

Extract useful information
 We keep modify routes/index.js and use XPath command to extract event name and
event start time:
 Refresh the page, http://ec2-54-86-42-76.compute-1.amazonaws.com:3000/
 In the console, we can see that the name and time are correct

Extract useful information
console.log(url);
var events={}
events.name = $(content).find('h1[itemprop="name"]').text();
events.time = $(content).find('time[id="event-start-time"]').text();
console.log(events);
res.send(content);
});
};

Use MongoDB to save JSON data
 We first create a unique folder for our database operations
 Then we create a file models/Event.js:
$ mkdir models

 Then we add the model for events: Event.js
var mongoose = require('mongoose')
mongoose.connect('mongodb://localhost/nodejs');
exports.mongoose = mongoose;
var Schema = mongoose.Schema;
var EventSchema = new Schema({
name : String,
time : String,
});
var Event = mongoose.model("Event", EventSchema);
var EventExport = function(){};
module.exports = new EventExport();
EventExport.prototype.save = function(obj, callback) {
var instance = new Event(obj);
instance.save(function(err){
callback(err);
});
};

 Finally, we edit index.js to call the save method:

 Refresh our page, the content on the page and console is still the same.How about
the database?
 We use
 to interact with MongoDB
 Then type the following command:
 That is our data!
$ mongo
> use nodejs
switched to db nodejs
> show collections
events
system.indexes
> db.events.find()
{ "_id" : ObjectId("535526ee43059424331b16a9"), "name" : "Node.js Workshop II:
toolkit to make your work efficiently", "time" : "Monday, April 21, 2014 7:00 PM",
"__v" : 0 }

Nodejs.meetup

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Nodejs.meetup

Similar to Nodejs.meetup (20)

More from Vivian S. Zhang

More from Vivian S. Zhang (20)

Recently uploaded

Recently uploaded (20)

Nodejs.meetup