Utah Code Camp 2015 presentation on making mashups with NodeJS.

1. Richard Thomson
Principal Architect for Modeling
DAZ 3D
@LegalizeAdulthood
legalize@xmission.com
http://LegalizeAdulthood.wordpress.com
http://github.com/LegalizeAdulthood

2. Wikipedia
"A mashup is a web page that uses content
from more than one source to create a single
new service displayed in a single graphical
interface.
[...]
The term implies easy, fast integration,
using APIs and data sources to produce
enriched results [...]."

3. What is NodeJS?
"Node.js is a platform built on Chrome's
JavaScript runtime for easily building fast,
scalable network applications. Node.js uses an
event-driven, non-blocking I/O model that
makes it lightweight and efficient, perfect for
data-intensive real-time applications that run
across distributed devices."
http://nodejs.org

4. http://nodejs.org

5. What is a Mashup?
 Web server application
 Aggregates data from multiple sources
 Renders your interpretation of the data
 Data gathering is inherently asynchronous
 Good time to use NodeJS!

6. A Software Team's Mashup
 Sprint dates
 Bug tracker queries
 Builds
 TODO Comments
 Cross-repository synchronization
 Copyright notices

7. http://sonarqube.org

8. Sources of Internal Data
 Source code (local / version control)
 Bug tracker
 Wiki
 QA/build servers
 build logs
 static analysis reports
 code reviews (review board, gerrit, etc.)

9. Sources of External Data
 REST apis
 RSS feeds
 HTML pages
 Email
 Serial port

10. Simple Mashup Server

11. Mashup Server with
Persistence

12. Mashup Agent

13. Exploring Available Data
 Use a REST api exploration tool (not curl)
 Advanced REST Client for Chrome is great
 Explore other data sources from Node.js
command line

14. Handy npm Modules
 request
 Simplifies making HTTP requests
 moment
 Date/Time manipulation
 async
 Orchestrate data sources and updates
 underscore / lodash
 General utility
 cheerio
 Lax HTML parsing for screen scraping

15. http://npmjs.com
https://github.com/caolan/async
http://momentjs.com/
http://underscorejs.org/
https://github.com/cheeriojs/cheerio

16. Web Server
 http.createServer
 express + jade
 MEAN stack
 (MongoDB, Express, AngularJS, NodeJS)
 Others

17. https://npmjs.com/package/express
http://jade-lang.com/
http://en.wikipedia.org/wiki/MEAN

18. Data Gathering
 Use NodeJS style callbacks
 Periodic updates
 Use an interval timer and a cache
 Use an interval timer and events
 Use async to get parallelism
 One NodeJS module per data source

19. Screen Scraping with Cheerio
 Cheerio is a lax HTML parser
 Forgiving of crappy HTML all over the web
 Provides a jQuery-like API for accessing the
parsed document

20. Data Rendering
 Server side:
 Templates with data model substitution
 Client side:
 Server static client app (AngularJS, etc.)
 Client app pulls model data from REST api

21. Advice
 Start simple and evolve
 Separate responsibilities into Node modules
 Inject dependent modules for unit testing
 Look for npm modules for your data source
 For large data models, consider rendering a
REST api and serve client-side app