• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Nodejs.meetup
 

Nodejs.meetup

on

  • 2,762 views

RSVP Node.js class at www.nycdatascience.com

RSVP Node.js class at www.nycdatascience.com

NYC data science academy's free workshop, given at NYC Open Data Meetup, http://www.meetup.com/NYC-Open-Data/events/163300552/

Statistics

Views

Total Views
2,762
Views on SlideShare
2,743
Embed Views
19

Actions

Likes
1
Downloads
13
Comments
2

3 Embeds 19

http://nycdatascience.com 17
https://www.linkedin.com 1
http://www.slideee.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Mindey, I got Bachelor and Master in Computer Science and second Master in Statistics. I am a more CS person indeed. :)
    Are you sure you want to
    Your message goes here
    Processing…
  • Thank you, Vivian! I had recently wanted to look into Node.js (cause it really supports real time apps well), really cool intro! (love MongoDB, etc.) You seems to be interested not only in statistics, but in computer science too!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Nodejs.meetup Nodejs.meetup Presentation Transcript

    • NYC Data Science Academy Node.JS in Action (Copyright materials, all right reserved) Sign up for Node.js beginner class at www.nycdatascience.com or email vivian.zhang@supstat.com
    • NYC Data Science Academy Overview  Deployment of Node.js on AWS EC2 Instance  Use Bootstrap Framework for our page  Rules of Routes, Session and MongoDB, Error Handling and Visiting Control  Web Scrapper  Save JSON data into MongoDB
    • NYC Data Science Academy AWS Security Group  You should Add HTTP Rules  For convenience, you could allow All TCP rules
    • NYC Data Science Academy Installation on Ubuntu  Our machine is Ubuntu 12.04 LTS, 64 bits  We install node and express from apt-get  Check if we are having the right configuration $ sudo add-apt-repository ppa:chris-lea/node.js $ sudo apt-get update $ sudo apt-get install nodejs $ sudo npm install express@3 -g $ sudo apt-get install unzip $ sudo apt-get install build-essential $ node -v v0.10.26 $ npm -v 1.4.3 $ express -V 3.5.1
    • NYC Data Science Academy Start Our first Project  In ~/ directory, we can start our first express project  If you see the notification, then it is OK!  Visit your public address on port 3000 and you will see $ express -e nodejs-demo $ cd nodejs-demo $ sudo npm install $ node app.js
    • NYC Data Science Academy Supervisor  If we modify our code, we need to restart our server. That is inconvenient.  By supervisor, we can modify our code and the server will restart automatically. $ sudo npm install supervisor -g $ supervisor app.js Running node-supervisor with program 'app.js' --watch '.' --extensions 'node,js' --exec 'node' Starting child process with 'node app.js' Watching directory '/home/ubuntu/nodejs-demo' for changes. Express server listening on port 3000
    • NYC Data Science Academy Intro to the working directory  node_modules: Store all the dependent libraries(every project manage its own dependencies).  package.json: Project dependencies configuration and developer information  app.js: Application starting file  public: Static files(css,js,img)  routes: Routes files(C in MVC, controller)  Views: Page files(Ejs template)
    • NYC Data Science Academy Support html files  And rename views/index.ejs to views/index.html  Add the following lines in app.js, see next page for codes  Visit your website $ mv views/index.ejs views/index.html
    • NYC Data Science Academy Support html files var express = require('express'); var routes = require('./routes'); var user = require('./routes/user'); var http = require('http'); var path = require('path'); var ejs = require('ejs'); var app = express(); // all environments app.set('port', process.env.PORT || 3000); app.set('views', path.join(__dirname, 'views')); // app.set('view engine', ‟ejs'); app.engine('.html', ejs.__express); app.set('view engine', 'html'); app.use(express.favicon()); app.use(express.logger('dev'));
    • NYC Data Science Academy Use Bootstrap Framework  Bootstrap Framework is one of the most popular frameworks for front-end.  Download it from http://getbootstrap.com/ and unzip.  And we also need jQuery.  Check whether your “~/nodejs-demo/public/stylesheets” has three .css files and whether „~/nodejs-demo/public/javascripts‟ has two .js files. $ cd ~/ $ wget https://github.com/twbs/bootstrap/releases/download/v3.1.1/bootstrap- 3.1.1-dist.zip $ unzip bootstrap-3.1.1-dist.zip $ cd ~/bootstrap-3.1.1-dist $ cp css/*.min.css ~/nodejs-demo/public/stylesheets/ $ cd js/ $ wget http://code.jquery.com/jquery-1.9.1.min.js $ cp *.min.js ~/nodejs-demo/public/javascripts $ cd ~/nodejs-demo $ ls public/javascripts $ ls public/stylesheets
    • NYC Data Science Academy Split index.html  We want a common header and footer by spliting index.html into header.html, index.html and footer.html  First, we add these two news files in foldr views/
    • NYC Data Science Academy header.html  We edit the header.html like this <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title><%=: title %></title> <!-- Bootstrap --> <link href="/stylesheets/bootstrap.min.css" rel="stylesheet" media="screen"> <!-- <link href="css/bootstrap-responsive.min.css" rel="stylesheet" media="screen"> -- > </head> <body screen_capture_injected="true"> $ vi ~/nodejs-demo/views/header.html
    • NYC Data Science Academy footer.html  We edit the footer.html like this <script src="/javascripts/jquery-1.9.1.min.js"> </script> <script src="/javascripts/bootstrap.min.js"> </script> </body> </html> $ vi ~/nodejs-demo/views/footer.html
    • NYC Data Science Academy index.html  Through ejs(emmeded javascript), we can include our header and footer in index.html  modify index.html <% include header.html %> <h1><%= title %> </h1> <p>Welcome to <%= title %> </p> <% include footer.html %> $ rm ~/nodejs-demo/views/index.html $ vi ~/nodejs-demo/views/index.html
    • NYC Data Science Academy Routes of our demo website  Routes: pages, authorization and action  / : index.html, No need to login to visit.  /home : home.html, Need to login to visit.  /login : login.html, redirect to home.html with correct username and password  /logout : No corresponding html file  redirect back to index.html after logout  We can add routes to enable these features.
    • NYC Data Science Academy Add routes  In app.js, we edit it by “vi ~/nodejs-demo/app.js”  Note: app.get is get request, app.post is post request, get.all is request for this path. app.get('/', routes.index); //app.get('/users', user.list); app.get('/login', routes.login); app.post('/login', routes.doLogin); app.get('/logout', routes.logout); app.get('/home', routes.home);
    • NYC Data Science Academy Add routes  Then we edit routes/index.js by by “vi ~/nodejs-demo/routes/index.js”
    • NYC Data Science Academy Add routes exports.index = function(req, res){ res.render('index', { title: 'Express' }); }; exports.login = function(req, res){ res.render('login', { title: 'User Login'}); }; exports.doLogin = function(req, res){ var user={ username:'admin', password:'admin' } if(req.body.username===user.username && req.body.password===user.password){ res.redirect('/home'); } res.redirect('/login'); }; exports.logout = function(req, res){ res.redirect('/'); }; exports.home = function(req, res){ var user={ username:'admin', password:'admin' } res.render('home', { title: 'Home',user: user}); };
    • NYC Data Science Academy Add pages  Create new home page by “vi ~/nodejs-demo/views/home.html” , see page 20  Create new login page by “vi ~/nodejs-demo/views/login.html”, see page 21
    • NYC Data Science Academy Create home.html <% include header.html %> <h1>Welcome <%= user.username %>!</h1> <a class="btn" href="/logout">Logout</a> <% include footer.html %>
    • NYC Data Science Academy Create login.html <% include header.html %> <link href="/stylesheets/signin.css" rel="stylesheet"> <div class="container"> <form class="form-signin" method="post"> <h2 class="form-signin-heading">Please sign in</h2> <input type="text" class="form-control" id="username" name="username" required autofocus> <input type="password" class="form-control" id="password" name="password" required> <button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button> </form> </div> <% include footer.html %>
    • NYC Data Science Academy Add pages  And we edit index.html by „vi ~/nodejs-demo/views/index.html‟ <% include header.html %> <h1><%= title %> </h1> <p>Welcome to <%= title %> <a href="/login">Signin</a> </p> <% include footer.html %>
    • NYC Data Science Academy Improve the look  This is what we see at /login.html:  Is there anyway we can make it better?
    • NYC Data Science Academy Edit the Bootstrap Style  We create signin.css file to public/stylesheets/ by  “vi ~/nodejs-demo/public/stylesheets/signin.css”  Copy/past left side and right side into the file body { padding-top: 40px; padding-bottom: 40px; background-color: #eee; } .form-signin { max-width: 330px; padding: 15px; margin: 0 auto; } .form-signin .form-signin-heading, .form-signin .checkbox { margin-bottom: 10px; } .form-signin .checkbox { font-weight: normal; } .form-signin .form-control { position: relative; font-size: 16px; height: auto; padding: 10px; -webkit-box-sizing: border-box; -moz-box-sizing: border-box; box-sizing: border-box; } .form-signin .form-control:focus { z-index: 2; } .form-signin input[type="text"] { margin-bottom: -1px; border-bottom-left-radius: 0; border-bottom-right-radius: 0; } .form-signin input[type="password"] { margin-bottom: 10px; border-top-left-radius: 0; border-top-right-radius: 0; }
    • NYC Data Science Academy Edit the Bootstrap Style  Then the website looks like:  Use admin and admin to login!
    • NYC Data Science Academy Session  In “routes/index.js” file , when we call function “exports.doLogin”, if username and password are correct, we will redirect to home.  In the same file, when we call function “exports.home”, we render the page and pass username to the home.html by the following specific:  Why can't we assign the username to session so that we do not need pass to each page again? res.redirect('/home'); res.render('home', { title: 'Home',user: user});
    • NYC Data Science Academy Storage of data  On local drive, we store user credentials into cookie files.  On server side, we also store session and user credentials in database, like Redis, MongoDB.  MongoDB pass the username object to each page instead of asking node.js passing the username object between pages.
    • NYC Data Science Academy Edit app.js  The Order of Commands Matters !!!
    • NYC Data Science Academy Edit app.js app.set('view engine', 'html'); app.use(express.favicon()); app.use(express.logger('dev')); app.use(express.bodyParser()); app.use(express.json()); app.use(express.urlencoded()); app.use(express.methodOverride()); app.use(express.cookieParser()); app.use(express.cookieSession({secret : 'nycdatascience.com'})); app.use(express.session({ secret : 'nycdatascience.com', store: store, cookie: { maxAge: 900000 } })); app.use(function(req, res, next){ res.locals.user = req.session.user; next(); }); app.use(app.router); app.use(express.static(path.join(__dirname, 'public'))); var express = require('express'); var routes = require('./routes'); var user = require('./routes/user'); var http = require('http'); var path = require('path'); var ejs = require('ejs'); var SessionStore = require("session- mongoose")(express); var store = new SessionStore({ url: "mongodb://localhost/session", interval: 120000 }); var app = express();
    • NYC Data Science Academy MongoDB on Ubuntu  Following the official document, we successfully installed MongoDB  http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 $ echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list $ sudo apt-get update $ sudo apt-get install mongodb-org $ sudo service mongod start
    • NYC Data Science Academy Installing Mongoose  We need to install Mongoose: how node.js connect with MongoDB  Then /login page is accessible $ sudo npm install session-mongoose $ npm install mongoose
    • NYC Data Science Academy Edit index.js  We need to edit index.js to support session by „vi ~/nodejs-demo/routes/index.js‟
    • NYC Data Science Academy Edit index.js exports.doLogin = function(req, res){ var user={ username:'admin', password:'admin' } if(req.body.username===user.username && req.body.password===user.password){ req.session.user=user; return res.redirect('/home'); } else{ return res.redirect('/login'); } }; exports.logout = function(req, res){ req.session.user=null; res.redirect('/'); }; exports.home = function(req, res){ //delete first 5 lines, change last line res.render('home', { title: 'Home'}); }
    • NYC Data Science Academy Error Handling  We want to tell user if they have entered the wrong information  Edit app.js by „vi ~/nodejs-demo/app.js‟ app.use(function(req, res, next){ res.locals.user = req.session.user; var err = req.session.error; delete req.session.error; res.locals.message = ''; if (err) res.locals.message = '<div class="alert alert-error">' + err +'</div>'; next(); }); app.use(app.router); app.use(express.static(path.join(__dirname, 'public')));
    • NYC Data Science Academy Error Handling  Edit login.html by „vi ~/nodejs-demo/views/login.html‟ <% include header.html %> <link href="/stylesheets/signin.css" rel="stylesheet"> <div class="container"> <form class="form-signin" method="post"> <%- message %> <h2 class="form-signin-heading">Please sign in</h2> <input type="text" class="form-control" id="username" name="username" required autofocus> <input type="password" class="form-control" id="password" name="password" required> <button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button> </form> </div> <% include footer.html %>
    • NYC Data Science Academy Error Handling  Edit index.js by by „vi ~/nodejs-demo/routes/index.js‟ exports.doLogin = function(req, res){ var user={ username:'admin', password:'admin' } if(req.body.username===user.username && req.body.password===user.password){ req.session.user=user; return res.redirect('/home'); } else { req.session.error='Wrong Username or Password!'; return res.redirect('/login'); } };
    • NYC Data Science Academy Error Handling  Let's have a try:
    • NYC Data Science Academy Website safety  Our website is almost ready, but it is not safe yet.  We can access /home without login.
    • NYC Data Science Academy Refine model of our demo website  /: anybody could visit  Still remember app.get, app.post and app.all?  /login: use app.all to take care of the requests of visiting /login. We call notAuthentication function(If session.user is not null, then the user has logined in.) first to check whether user's been logged in  /logout: use app.get to take care of the requests of visiting /logout. We call authentication function(If session.user is null, it will hint “please sign in”).  /home: use app.get to take care of the requests of visiting /home. We call authentication function(If session.user is null, it will hint “please sign in”).
    • NYC Data Science Academy Edit app.js  We add two functions and make use of them.
    • NYC Data Science Academy Edit app.js app.use(app.router); app.use(express.static(path.join(__dirname, 'public'))); function authentication(req, res, next) { if (!req.session.user) { req.session.error='Please Sign In to Visit'; return res.redirect('/login'); } next(); } function notAuthentication(req, res, next) { if (req.session.user) { req.session.error='Logged in'; return res.redirect(‟/home‟); } next(); } // development only if ('development' == app.get('env')) { app.use(express.errorHandler()); } app.get('/', routes.index); //app.get('/users', user.list); app.all('/login', notAuthentication); app.get('/login', routes.login); app.post('/login', routes.doLogin); app.get('/logout', authentication); app.get('/logout', routes.logout); app.get('/home', authentication); app.get('/home', routes.home);
    • NYC Data Science Academy Safety check  Let's check if it works.  We have done it!  Go to /home and /logout page while you are not logined in  Go to login page while you are logined in
    • NYC Data Science Academy Add some content?  So, how to add content to our database?  Why not use informations from other websites?
    • NYC Data Science Academy Install libraries  We need to install jsdom, jQuery, xmlhttprequest, request, htmlparser $ sudo npm install jsdom $ sudo npm install jQuery $ sudo npm install xmlhttprequest $ sudo npm install request $ sudo npm install htmlparser
    • NYC Data Science Academy What is valuable?  Our event webpage!  http://www.meetup.com/NYC-Open-Data/events/163300552/
    • NYC Data Science Academy Add codes  Add myUtil.js by „vi ~/nodejs_demo/myUtil.js‟  It is used as getter function to scrap certain web page. var MyUtil = function () { }; var http = require('http'); var request = require('request'); MyUtil.prototype.get=function(url,callback){ request(url, function (error, response, body) { if (!error && response.statusCode == 200) { callback(body,response.statusCode); } }) } module.exports = new MyUtil();
    • NYC Data Science Academy Add codes  And edit index.js  Add the following codes in the end. var myUtil = require('../myUtil.js'); var $ = require('jQuery'); exports.index = function(req, res){ var url="http://www.meetup.com/NYC-Open-Data/events/163300552/"; console.log(url); myUtil.get(url,function(content,status){ console.log("status:="+status); res.send(content); }); };
    • NYC Data Science Academy Content Downloaded  Then we visit the index page:
    • NYC Data Science Academy Extract useful information  We keep modify routes/index.js and use XPath command to extract event name and event start time:  Refresh the page, http://ec2-54-86-42-76.compute-1.amazonaws.com:3000/  In the console, we can see that the name and time are correct
    • NYC Data Science Academy Extract useful information var myUtil = require('../myUtil.js'); var $ = require('jQuery'); exports.index = function(req, res){ var url="http://www.meetup.com/NYC-Open-Data/events/163300552/"; console.log(url); myUtil.get(url,function(content,status){ console.log("status:="+status); var events={} events.name = $(content).find('h1[itemprop="name"]').text(); events.time = $(content).find('time[id="event-start-time"]').text(); console.log(events); res.send(content); }); };
    • NYC Data Science Academy Use MongoDB to save JSON data  We first create a unique folder for our database operations  Then we create a file models/Event.js: $ mkdir models
    • NYC Data Science Academy Use MongoDB to save JSON data  Then we add the model for events: Event.js var mongoose = require('mongoose') mongoose.connect('mongodb://localhost/nodejs'); exports.mongoose = mongoose; var Schema = mongoose.Schema; var EventSchema = new Schema({ name : String, time : String, }); var Event = mongoose.model("Event", EventSchema); var EventExport = function(){}; module.exports = new EventExport(); EventExport.prototype.save = function(obj, callback) { var instance = new Event(obj); instance.save(function(err){ callback(err); }); };
    • NYC Data Science Academy Use MongoDB to save JSON data  Finally, we edit index.js to call the save method:
    • NYC Data Science Academy Use MongoDB to save JSON data var myUtil = require('../myUtil.js'); var Event = require('../models/Event.js') var $ = require('jQuery'); exports.index = function(req, res){ var url="http://www.meetup.com/NYC-Open-Data/events/163300552/"; console.log(url); myUtil.get(url,function(content,status){ console.log("status:="+status); var events={} events.name = $(content).find('h1[itemprop="name"]').text(); events.time = $(content).find('time[id="event-start-time"]').text(); console.log(events); var json = events; Event.save(json, function(err){ if (err){ res.send({'success':false,'err':err}); } else { res.send({'success':true}); } }); res.send(content); }); };
    • NYC Data Science Academy Use MongoDB to save JSON data  Refresh our page, the content on the page and console is still the same.How about the database?  We use  to interact with MongoDB  Then type the following command:  That is our data! $ mongo > use nodejs switched to db nodejs > show collections events system.indexes > db.events.find() { "_id" : ObjectId("535526ee43059424331b16a9"), "name" : "Node.js Workshop II: toolkit to make your work efficiently", "time" : "Monday, April 21, 2014 7:00 PM", "__v" : 0 }
    • NYC Data Science Academy FAQ