Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EWD 3 Training Course Part 42: The QEWD Docker Appliance

521 views

Published on

This is part 42 of the EWD 3 Training Course. In this presentation, you'll learn how to use the Docker Container version of QEWD

Published in: Software
  • Be the first to comment

EWD 3 Training Course Part 42: The QEWD Docker Appliance

  1. 1. Copyright © 2016 M/Gateway Developments Ltd EWD 3 Training Course Part 42 The QEWD Docker Appliance Rob Tweed Director, M/Gateway Developments Ltd Twitter: @rtweed
  2. 2. Copyright © 2016 M/Gateway Developments Ltd Docker? • A technology that allows software to be encapsulated inside an isolated container • According to Docker themselves: – "Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment." – See https://www.docker.com/what-docker
  3. 3. Copyright © 2016 M/Gateway Developments Ltd QEWD Docker Container • QEWD is available from the main docker.io repository as a pre-built Docker container – rtweed/qewd • You can also build your own version using the Dockerfile that is included in the QEWD repository – eg, if you wanted to apply your own customisations – See https://github.com/robtweed/qewd/blob/master/docker/Dockerfile
  4. 4. Copyright © 2016 M/Gateway Developments Ltd Benefits of the QEWD Container • Provided you've installed Docker, you don't need to do any further installation – Just run the container • First time you do this, it will automatically download it from docker.io
  5. 5. Copyright © 2016 M/Gateway Developments Ltd Benefits of the QEWD Container • QEWD functionality without any "moving parts" on your system – eg no Node.js dependency clashes • You don't even need Node.js installed on your system – You just define your handler modules • Write JavaScript files and let the QEWD container handle them – Without Node.js having to even be present on your system
  6. 6. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • Just the core QEWD engine – Master process – Worker processes • Doesn't include the embedded database – Currently designed to work with Redis only • Using mapped volumes – You can use Redis locally, in another container or remotely
  7. 7. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • Normal QEWD Session management will take place provided a Redis database is available
  8. 8. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • Nothing you can do in a standard local QEWD installation that can't also be achieved by using the Dockerised version – There's even a version for the Raspberry Pi! • rtweed/rpi-qewd
  9. 9. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • "Out of the box" defaults – Management password: 'keepThisSecret!' – Worker pool size: 1 – Database: Redis listening on port 6379 • qewd-monitor application is included and ready to run – No other applications or APIs pre-defined
  10. 10. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • You can customise the configuration: – Management password – Server name (what appears in qewd-monitor's banner) – Worker pool size
  11. 11. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • By using Docker volume mapping, you can make specific directories in your host system available to the QEWD Docker container • In your host system you can therefore define interactive QEWD applications: – Static browser-side resources (HTML, JS, CSS files) – Back-end handler modules
  12. 12. Copyright © 2016 M/Gateway Developments Ltd QEWD Appliance • You can also define routes within your host system, to be used by the QEWD Appliance – For Web / REST service APIs
  13. 13. Copyright © 2016 M/Gateway Developments Ltd So let's try it out • I'm going to use a Ubuntu Linux system • However, Docker can be installed on most platforms – Windows – Linux (all versions/flavours) – Unix – MacOS – Even the Raspberry Pi!
  14. 14. Copyright © 2016 M/Gateway Developments Ltd Installing Docker on Ubuntu 16.04 • sudo apt-get update • sudo apt-get install docker.io • You can now use Docker – By default, you'll need to run Docker commands as sudo – This can be modified. See: – https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04
  15. 15. Copyright © 2016 M/Gateway Developments Ltd Installing Docker on Raspberry Pi • Just invoke the command: curl -sSL https://get.docker.com | sh • You can now use Docker on your Raspberry Pi –By default, you'll need to run Docker commands as sudo
  16. 16. Copyright © 2016 M/Gateway Developments Ltd Some Quick Docker Tests • Try running these to confirm that Docker is properly installed and working on your system: – sudo docker version – sudo docker info
  17. 17. Copyright © 2016 M/Gateway Developments Ltd Let's start a Redis Container On Linux: •sudo docker run -d --name redis -p 6379:6379 redis •Runs as a daemon (-d) •Exposes internal port 6379 (the default Redis port) externally to the host as port 6379 (-p) •First time you invoke this command, it will download and install the Redis Docker container
  18. 18. Copyright © 2016 M/Gateway Developments Ltd Let's start a Redis Container On Raspberry Pi: •sudo docker run -d --name redis -p 6379:6379 hypriot/rpi-redis •Runs as a daemon (-d) •Exposes internal port 6379 (the default Redis port) externally to the host as port 6379 (-p) •First time you invoke this command, it will download and install the Redis Docker container
  19. 19. Copyright © 2016 M/Gateway Developments Ltd Test Redis • First install the Redis command line tool: – sudo apt-get install redis-tools • Now start the Redis command line: redis-cli • It should respond with 127.0.0.1:6379> – it's working and you're connected to the Redis container! • Try the command: info • Exit redis-cli by typing CTRL&C
  20. 20. Copyright © 2016 M/Gateway Developments Ltd Prepare for the QEWD Appliance • First important step: – Create a directory for mapping the QEWD working directory – It can be any directory you like. I'll use: cd ~ mkdir qewd – Now make a sub-directory named modules cd qewd mkdir modules • You don't need to put anything into this directory yet, but it should exist
  21. 21. Copyright © 2016 M/Gateway Developments Ltd Start the QEWD Appliance sudo docker run -d -p 8080:8080 --link redis:redis -v /home/ubuntu/qewd/modules:/opt/qewd/mapped rtweed/qewd First time you invoke this, it will download the QEWD Container from Docker.io -p 8080:8080 maps port 8080 in the QEWD container to your host's port 8080 -d run as a daemon. You can also try -it for interactive display -link redis:redis links to the Redis container -v maps the QEWD working directory to the one we created QEWD will now be running! On Linux:
  22. 22. Copyright © 2016 M/Gateway Developments Ltd Start the QEWD Appliance sudo docker run -d -p 8080:8080 --link redis:redis -v /home/pi/qewd/modules:/opt/qewd/mapped rtweed/rpi-qewd First time you invoke this, it will download the QEWD Container from Docker.io -p 8080:8080 maps port 8080 in the QEWD container to your host's port 8080 -d run as a daemon. You can also try -it for interactive display -link redis:redis links to the Redis container -v maps the QEWD working directory to the one we created QEWD will now be running! On Raspberry Pi:
  23. 23. Copyright © 2016 M/Gateway Developments Ltd Try QEWD http://192.168.1.123:8080/qewd-monitor/index.html Change the IP address / host name as appropriate for your host machine You should see the qewd-monitor application! Log in using the default password: keepThisSecret!
  24. 24. Copyright © 2016 M/Gateway Developments Ltd Let's check Redis again • Start redis-cli again • This time enter the command: – keys * • You should see a load of Redis key/value pairs, representing the QEWD Session global storage used by your qewd-monitor session
  25. 25. Copyright © 2016 M/Gateway Developments Ltd Let's build a quick application • We'll create the files in the ~/qewd directory that we previously made
  26. 26. Copyright © 2016 M/Gateway Developments Ltd First, stop the QEWD Container • Use the qewd-monitor; or • Use the docker stop command – To do that we need to first find out the QEWD container's unique Id – sudo docker ps -a CONTAINER ID IMAGE d810c520894e rtweed/qewd 7d0c18b8cec5 redis
  27. 27. Copyright © 2016 M/Gateway Developments Ltd First, stop the QEWD Container • Use the qewd-monitor; or • Use the docker stop command – To do that we need to first find out the QEWD container's unique Id – sudo docker ps -a CONTAINER ID IMAGE d810c520894e rtweed/qewd 7d0c18b8cec5 redis Usually the first 3 characters are sufficient
  28. 28. Copyright © 2016 M/Gateway Developments Ltd Stop the QEWD Container • sudo docker stop d81
  29. 29. Copyright © 2016 M/Gateway Developments Ltd Create a test application • We'll name it testapp – It will have a button which, when clicked, sends a message to the QEWD back-end which returns an acknowledgement response • First create a directory for the browser- side static files: cd ~/qewd mkdir www cd www
  30. 30. Copyright © 2016 M/Gateway Developments Ltd Create the index.html <html> <head> <title>Demo QEWD application</title> </head> <body> <script src="//ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script> <script src="/socket.io/socket.io.js"></script> <script src="/ewd-client.js"></script> <script src="app.js"></script> <button id="testBtn">Click Me</button> <div id="content"> Content goes here </div> </body> </html> Save as ~/qewd/www/testapp/index.html
  31. 31. Copyright © 2016 M/Gateway Developments Ltd Create the app.js Save as ~/qewd/www/testapp/app.js $(document).ready(function() { EWD.log = true; EWD.on('ewd-registered', function() { $('#testBtn').on('click', function(e) { var message = {type: 'testButton'}; EWD.send(message, function(messageObj) { $('#content').text(messageObj.message.ok); }); }); }); EWD.start('testapp', $, io); });
  32. 32. Copyright © 2016 M/Gateway Developments Ltd Create the back-end module • This needs to go into the directory we created earlier: ~/qewd/modules • We'll name the module testapp.js
  33. 33. Copyright © 2016 M/Gateway Developments Ltd Create testapp.js module.exports = { handlers: { testButton: function(messageObj, session, send, finished) { console.log('*** handling the button click message!'); session.data.$('foo').value = 'bar'; finished({ ok: 'testButton message was processed successfully!' }); } } }; Save as ~/qewd/modules/testapp.js
  34. 34. Copyright © 2016 M/Gateway Developments Ltd How will QEWD find the module? • The QEWD Docker Appliance will expect to find it in its node_modules directory • We don't have access to that from the host • But we've mapped ~/qewd/modules to a mapped volume in the Docker container: ~/opt/qewd/mapped
  35. 35. Copyright © 2016 M/Gateway Developments Ltd How will QEWD find the module? • We can use a QEWD module map to tell QEWD where to find the back-end module for our testapp application • The QEWD Docker appliance allows us to define this in the mapped volume directory using the reserved file name: – moduleMap.js
  36. 36. Copyright © 2016 M/Gateway Developments Ltd Create moduleMap.js module.exports = { testapp: '/opt/qewd/mapped/testapp' } Save as ~/qewd/modules/moduleMap.js
  37. 37. Copyright © 2016 M/Gateway Developments Ltd Create moduleMap.js module.exports = { testapp: '/opt/qewd/mapped/testapp' } The module map exports as many mappings as you want Save as ~/qewd/modules/moduleMap.js
  38. 38. Copyright © 2016 M/Gateway Developments Ltd Create moduleMap.js module.exports = { testapp: '/opt/qewd/mapped/testapp' } The module map exports as many mappings as you want We just need this one. Save as ~/qewd/modules/moduleMap.js
  39. 39. Copyright © 2016 M/Gateway Developments Ltd Create moduleMap.js module.exports = { testapp: '/opt/qewd/mapped/testapp' } Note: the mapped module path is to the internal path within the container Save as ~/qewd/modules/moduleMap.js
  40. 40. Copyright © 2016 M/Gateway Developments Ltd Create moduleMap.js module.exports = { testapp: '/opt/qewd/mapped/testapp' } Note: the mapped module path is to the internal path within the container, not the path we created on the host ~/qewd/modules/testapp.js
  41. 41. Copyright © 2016 M/Gateway Developments Ltd That should be all we need • When we start the QEWD Appliance again, we must also map the directory containing the browser-side files, ie – ~/qewd/www/testapp
  42. 42. Copyright © 2016 M/Gateway Developments Ltd Start the QEWD Appliance • On Linux: sudo docker run -d -p 8080:8080 --link redis:redis ↩ -v /home/ubuntu/qewd/modules:/opt/qewd/mapped ↩ -v /home/ubuntu/qewd/www/testapp:/opt/qewd/www/testapp ↩ rtweed/qewd
  43. 43. Copyright © 2016 M/Gateway Developments Ltd Start the QEWD Appliance • On Raspberry Pi: sudo docker run -d -p 8080:8080 --link redis:redis ↩ -v /home/pi/qewd/modules:/opt/qewd/mapped ↩ -v /home/pi/qewd/www/testapp:/opt/qewd/www/testapp ↩ rtweed/rpi-qewd
  44. 44. Copyright © 2016 M/Gateway Developments Ltd Try running testapp It works The console show that testapp correctly registered itself on QEWD Try clicking the button..
  45. 45. Copyright © 2016 M/Gateway Developments Ltd Try running testapp It works too! A message was sent to the QEWD back-end which returned the expected response
  46. 46. Copyright © 2016 M/Gateway Developments Ltd Check the Session Start the qewd-monitor application and click the Session tab:
  47. 47. Copyright © 2016 M/Gateway Developments Ltd Check the Session Start the qewd-monitor application and click the Session tab: Where did this come from?
  48. 48. Copyright © 2016 M/Gateway Developments Ltd It came from testapp.js module.exports = { handlers: { testButton: function(messageObj, session, send, finished) { console.log('*** handling the button click message!'); session.data.$('foo').value = 'bar'; finished({ ok: 'testButton message was processed successfully!' }); } } }; Save as ~/qewd/modules/testapp.js
  49. 49. Copyright © 2016 M/Gateway Developments Ltd QEWD Apps with no moving parts! • We just created a QEWD app without having anything installed other than Docker • We just defined: – the HTML and JavaScript files for the browser – The back-end JavaScript handler module • And mapped their directories as Docker volumes
  50. 50. Copyright © 2016 M/Gateway Developments Ltd Adding more apps • Each app you create will need its own subdirectory under the ~/qewd/www directory – For its browser-side HTML, JavaScript, etc files – Each of these will have to be mapped to the corresponding volume in the QEWD container • Their back-end modules, however, just go into your mapped ~/qewd/modules folder – And then define their module mapping in moduleMap.js
  51. 51. Copyright © 2016 M/Gateway Developments Ltd Mapping Multiple Apps module.exports = { testapp: '/opt/qewd/mapped/testapp', mySecondApp: '/opt/qewd/mapped/mySecondApp', myThirdApp: '/opt/qewd/mapped/myThirdApp' } ~/qewd/modules/moduleMap.js
  52. 52. Copyright © 2016 M/Gateway Developments Ltd Mapping Multiple Apps • eg, On Linux: sudo docker run -d -p 8080:8080 --link redis:redis ↩ -v /home/ubuntu/qewd/modules:/opt/qewd/mapped ↩ -v /home/ubuntu/qewd/www/testapp:/opt/qewd/www/testapp ↩ -v /home/ubuntu/qewd/www/mySecondApp:/opt/qewd/www/mySecondApp ↩ -v /home/ubuntu/qewd/www/myThirdApp:/opt/qewd/www/myThirdApp ↩ rtweed/qewd
  53. 53. Copyright © 2016 M/Gateway Developments Ltd What about REST APIs? • We can use the QEWD Docker Appliance to very quickly create REST APIs, again without any moving parts other than Docker itself • Even simpler to create than applications
  54. 54. Copyright © 2016 M/Gateway Developments Ltd Let's create an API • We'll define a URL path prefix: – /api • And handle any lower-level URLs, eg: – /api/test • To do this we just need to create a handler module for the /api path. We'll call it api.js. It must be saved into the ~/qewd/modules directory
  55. 55. Copyright © 2016 M/Gateway Developments Ltd Create api.js module.exports = { restModule: true, handlers: { test: function(messageObj, finished) { finished({foo: 'bar'}); } } }; Save as ~/qewd/modules/api.js
  56. 56. Copyright © 2016 M/Gateway Developments Ltd Create api.js module.exports = { restModule: true, handlers: { test: function(messageObj, finished) { finished({foo: 'bar'}); } } }; Save as ~/qewd/modules/api.js It's just a standard QEWD REST/Web Service Module
  57. 57. Copyright © 2016 M/Gateway Developments Ltd Create api.js module.exports = { restModule: true, handlers: { test: function(messageObj, finished) { finished({foo: 'bar'}); } } }; This function, test, will handle any incoming /api/test requests
  58. 58. Copyright © 2016 M/Gateway Developments Ltd QEWD needs to know how to find it • As we're defining the handler module in a mapped directory, we need to tell QEWD where to find it and how to define it as a route. • For this we define a reserved named file in our ~/qewd/modules directory: – routes.js
  59. 59. Copyright © 2016 M/Gateway Developments Ltd Create routes.js module.exports = [ { path: '/api', module: '/opt/qewd/mapped/api' } ]; Save as ~/qewd/modules/routes.js
  60. 60. Copyright © 2016 M/Gateway Developments Ltd Create routes.js module.exports = [ { path: '/api', module: '/opt/qewd/mapped/api' } ]; Save as ~/qewd/modules/routes.js You export an array of route objects We'll just define one for /api in this example, but you can define as many routes as you wish
  61. 61. Copyright © 2016 M/Gateway Developments Ltd Create routes.js module.exports = [ { path: '/api', module: '/opt/qewd/mapped/api' } ]; Note that the route must map to the container's internal mapped volume path So any incoming requests with a URL starting /api will be handled by the api.js file we've created in the host's mapped directory
  62. 62. Copyright © 2016 M/Gateway Developments Ltd Let's try it • Our handler function isn't making any distinctions about the HTTP method used, so we can test it just using a browser.
  63. 63. Copyright © 2016 M/Gateway Developments Ltd Let's try it It worked!
  64. 64. Copyright © 2016 M/Gateway Developments Ltd REST/Web Service APIs • Of course this was a very simple example • But you can write as complex APIs as you wish. • All the functionality described in parts 31 and 36 of this course are available in the QEWD Docker Appliance
  65. 65. Copyright © 2016 M/Gateway Developments Ltd REST APIs without moving parts • You can now define Web/REST APIs by writing JavaScript handler module files • No need to install Node.js
  66. 66. Copyright © 2016 M/Gateway Developments Ltd Customising QEWD Appliance • QEWD Docker Appliance defaults: – Management password: keepThisSecret! • Always a good idea to change this! – Server name: QEWD Docker Server • Used by the qewd-monitor application – Worker pool size: 1 • You can change these
  67. 67. Copyright © 2016 M/Gateway Developments Ltd Customising QEWD Appliance • Create a file named custom.js in your mapped directory (eg ~/qewd/mapped) – custom.js is a reserved name that the QEWD Docker Appliance will look for and use for customising its configuration
  68. 68. Copyright © 2016 M/Gateway Developments Ltd Create custom.js module.exports = { config: { managementPassword: 'mySecret!', serverName: 'Robs QEWD Docker Appliance', poolSize: 2 } }; Save as ~/qewd/modules/custom.js
  69. 69. Copyright © 2016 M/Gateway Developments Ltd Restart the Appliance • eg: – sudo docker restart d81 • Try running qewd-monitor – You'll need to use your new custom login password – It will display the new name for your server – It will show 2 worker processes
  70. 70. Copyright © 2016 M/Gateway Developments Ltd Further Customisation • You can get access to the master process objects: – Create user defined data / objects that are passed to worker processes – Access the ewd-qoper8 object – Access the Express object • Define a run function. This is invoked when QEWD starts.
  71. 71. Copyright © 2016 M/Gateway Developments Ltd Create custom.js function customise(config, q, intercept) { console.log('*** This is the custom function calling!'); } module.exports = { run: customise, config: { managementPassword: 'mySecret!', serverName: 'Robs QEWD Docker Appliance', poolSize: 2 } }; Save as ~/qewd/modules/custom.js
  72. 72. Copyright © 2016 M/Gateway Developments Ltd Create custom.js function customise(config, q, intercept) { console.log('*** This is the custom function calling!'); } module.exports = { run: customise, config: { managementPassword: 'mySecret!', serverName: 'Robs QEWD Docker Appliance', poolSize: 2 } }; q is the main ewd-qoper8 object intercept.app is the Express object allowing you to add custom middleware
  73. 73. Copyright © 2016 M/Gateway Developments Ltd That's the QEWD Docker Appliance • A handy and simple, low-impact way of using QEWD without worrying about interactions with your existing set-up – And without having to install Node.js or a database • All the functionality of a standard local implementation of QEWD is still available to you

×