Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Passenger 6 generic language support presentation

48 views

Published on

YouTube video: https://www.youtube.com/watch?v=QyMQSYdctv0
Blog: https://blog.phusion.nl/2018/11/06/how-passenger-6-generic-language-support-is-implemented/

Introductory presentation to the Passenger 6 generic language support coding session.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Passenger 6 generic language support presentation

  1. 1. PASSENGER
 GENERIC LANGUAGE SUPPORT 2018 live stream
  2. 2. TABLE OF CONTENTS 1. Why? 2. UX goal 3. Passenger architecture 4. Implementation plan 5. Coding!
  3. 3. Questions?
 Comments? Enter them in the chat
 (or post on Github)
  4. 4. ➤ Bring all the good stuff of Passenger to the whole world ➤ Standardization ➤ Onboard team members faster ➤ Streamline procedures ➤ Reduce inconsistencies ➤ Share and retain operational knowledge ➤ Focus ➤ Write less code ➤ Reduce infrastructure complexity ➤ Automate the boring details ➤ Focus on your core business Why GLS?
  5. 5. UX?
  6. 6. Ask for a
 command string! Like Heroku's Procfile
  7. 7. USER EXPERIENCE (1/2) server { listen 80; server_name foo.com; root /webapps/fooapp/public; passenger_enabled on; passenger_app_start_command './fooexe --port=$PORT'; }
  8. 8. USER EXPERIENCE (2/2) # Passengerfile.json { "app_start_command": "./fooexe --port=$PORT" } # Nginx config server { listen 80; server_name foo.com; root /webapps/fooapp/public; passenger_enabled on; # Look, no passenger_app_start_command! }
  9. 9. Passenger architecture 101
  10. 10. We need to
 modify these!
  11. 11. Passenger agent Config Core ApiServer Controller main request handling server Security Update Checker Telemetry Collector Contains most Passenger business logic. Manages processes, routes requests. Config ApiServerCoreWatcher InstanceDir Toucher Watchdog Starts and supervises the Core. ApplicationPool Pool main subsystem interface Options collection of app config; collection of parameters for a get() call Session represents a single request/response session with a specific process Group represents an app; a collection of processes Process represents a single app process Socket represents a server socket endpoint that a process listens on Context global subsystem configuration Public interface Internals Abstracts application process spawning and management logic into a simple API. Keeps track of processes and related statistics. SpawnEnvSetupper Exec wrapper used by SpawningKit. Sets up the process environment, then executes the actual application command. ExecHelper executes a command as a specific user, similar to sudo Utilities SystemMetrics prints various system metrics TempDirToucher touches everything in a dir periodically to prevent deletion by /tmp cleaners Context global subsystem configuration SpawningKit Low-level subsystem specialized in spawning application processes. Factory given a spawn method and language name, creates appropriate Spawner object Direct Spawner implements "direct" spawn method Smart Spawner implements "smart" spawn method <<abstract>> Spawner main subsystem interface Result info about a spawned process; returned by Spawner Error Renderer given a spawn exception, generates HTML error page Exceptions relevant exception classes User Switching Rules figures out which user an app should run as HandshakePrepare implements the preparation phase of the spawn flow Config config of a single spawn operation BackgroundIO Capturer runs in a thread & captures all subprocess output in a string Session working state of a single spawn operation Public interface Internals Spawning protocol logic Utilities
  12. 12. Passenger agent Config Core ApiServer Controller main request handling server Security Update Checker Telemetry Collector Contains most Passenger business logic. Manages processes, routes requests. Config ApiServerCoreWatcher InstanceDir Toucher Watchdog Starts and supervises the Core. ApplicationPool Pool main subsystem interface Options collection of app config; collection of parameters for a get() call Session represents a single request/response session with a specific process Group represents an app; a collection of processes Process represents a single app process Socket represents a server socket endpoint that a process listens on Context global subsystem configuration Public interface Internals Abstracts application process spawning and management logic into a simple API. Keeps track of processes and related statistics. SpawnEnvSetupper Exec wrapper used by SpawningKit. Sets up the process environment, then executes the actual application command. ExecHelper executes a command as a specific user, similar to sudo Utilities SystemMetrics prints various system metrics TempDirToucher touches everything in a dir periodically to prevent deletion by /tmp cleaners Context global subsystem configuration SpawningKit Low-level subsystem specialized in spawning application processes. Factory given a spawn method and language name, creates appropriate Spawner object Direct Spawner implements "direct" spawn method Smart Spawner implements "smart" spawn method <<abstract>> Spawner main subsystem interface Result info about a spawned process; returned by Spawner Error Renderer given a spawn exception, generates HTML error page Exceptions relevant exception classes User Switching Rules figures out which user an app should run as HandshakePrepare implements the preparation phase of the spawn flow Config config of a single spawn operation BackgroundIO Capturer runs in a thread & captures all subprocess output in a string Session working state of a single spawn operation Public interface Internals Spawning protocol logic Utilities Three
 layers!
  13. 13. Config Core ApiServer Controller main request handling server Security Update Checker Telemetry Collector Contains most Passenger business logic. Manages processes, routes requests. ApplicationPool Pool main subsystem interface Options collection of app config; collection of parameters for a get() call Session represents a single request/response session with a specific process Group represents an app; a collection of processes Process represents a single app process Socket represents a server socket endpoint that a process listens on Context global subsystem configuration Public interface Internals Abstracts application process spawning and management logic into a simple API. Keeps track of processes and related statistics. Passenger agent
  14. 14. Config Core ApiServer Controller main request handling server Security Update Checker Telemetry Collector Contains most Passenger business logic. Manages processes, routes requests. ApplicationPool Pool main subsystem interface Options collection of app config; collection of parameters for a get() call Session represents a single request/response session with a specific process Group represents an app; a collection of processes Process represents a single app process Socket represents a server socket endpoint that a process listens on Context global subsystem configuration Public interface Internals Abstracts application process spawning and management logic into a simple API. Keeps track of processes and related statistics. Passenger agent
  15. 15. Context global subsystem configuration SpawningKit Low-level subsystem specialized in spawning application processes. Factory given a spawn method and language name, creates appropriate Spawner object Direct Spawner implements "direct" spawn method Smart Spawner implements "smart" spawn method <<abstract>> Spawner main subsystem interface Result info about a spawned process; returned by Spawner Error Renderer given a spawn exception, generates HTML error page Exceptions relevant exception classes User Switching Rules figures out which user an app should run as HandshakePrepare implements the preparation phase of the spawn flow HandshakePerform implements the waiting & response phase of the spawn flow Journey keeps track of spawn operation progress Config config of a single spawn operation PipeWatcher runs in a thread & logs all subprocess output BackgroundIO Capturer runs in a thread & captures all subprocess output in a string Session working state of a single spawn operation WorkDir temp dir for a single spawn operation, used to communicate with the subprocess Public interface Internals Spawning protocol logic Utilities
  16. 16. Detailed
 implementation
 plan
  17. 17. 5.3.0 Foundation already laid in Implemented generic spawn flow
  18. 18. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  19. 19. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  20. 20. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  21. 21. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  22. 22. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  23. 23. SpawningKit:: DirectSpawner Language wrapper Ruby: rack-loader Python: wsgi-loader Node.js: nodejs-loader HandshakePrepare: - create work dir (temp dir) - dump config into a file in work dir (JSON) create new process - load config from work dir - load app (e.g. config.ru, index.js) - setup other behavior (e.g. stats collection) - setup internal server finish signal, internal server address return process info to caller e.g. socket address Passenger application spawn flow (simplified): auto-supported apps
  24. 24. Normal flow [DONE] Generic flow Up next:
  25. 25. SpawningKit:: DirectSpawner Application HandshakePrepare: - create work dir (temp dir) - find free port - dump config into a file in work dir (JSON) create new process return process info to caller e.g. socket address Passenger application spawn flow (simplified): generic apps - initialization - setup server socket ... ... SpawningKit notices that port is now in use Mechanism introduced in 5.3.0!
 But not exposed to users.
  26. 26. SpawningKit:: DirectSpawner Application HandshakePrepare: - create work dir (temp dir) - find free port - dump config into a file in work dir (JSON) create new process return process info to caller e.g. socket address Passenger application spawn flow (simplified): generic apps - initialization - setup server socket ... ... SpawningKit notices that port is now in use
  27. 27. SpawningKit:: DirectSpawner Application HandshakePrepare: - create work dir (temp dir) - find free port - dump config into a file in work dir (JSON) create new process return process info to caller e.g. socket address Passenger application spawn flow (simplified): generic apps - initialization - setup server socket ... ... SpawningKit notices that port is now in use
  28. 28. SpawningKit:: DirectSpawner Application HandshakePrepare: - create work dir (temp dir) - find free port - dump config into a file in work dir (JSON) create new process return process info to caller e.g. socket address Passenger application spawn flow (simplified): generic apps - initialization - setup server socket ... ... SpawningKit notices that port is now in use
  29. 29. Goal Build the mechanisms needed to activate generic spawn flow
  30. 30. PLAN Integration mode frontend Core Controller Core ApplicationPool • Add user config (e.g. passenger_app_start_command) • Process info from web server
 -OR-
 Perform our own autodetection (Standalone+builtin) • Pass necessary info to ApplicationPool Web server • Autodetect whether auto-supported or generic app • Pass necessary info to Core Controller • Modify Options class to support the new config options • Properly configure SpawningKit • Kill generic process upon shutdown
  31. 31. PLAN Integration mode frontend Core Controller Core ApplicationPool • Add user config (e.g. passenger_app_start_command) • Process info from web server
 -OR-
 Perform our own autodetection (Standalone+builtin) • Pass necessary info to ApplicationPool Web server • Autodetect whether auto-supported or generic app • Pass necessary info to Core Controller • Modify Options class to support the new config options • Properly configure SpawningKit • Kill generic process upon shutdown
  32. 32. PLAN Integration mode frontend Core Controller Core ApplicationPool • Add user config (e.g. passenger_app_start_command) • Process info from web server
 -OR-
 Perform our own autodetection (Standalone+builtin) • Pass necessary info to ApplicationPool Web server • Autodetect whether auto-supported or generic app • Pass necessary info to Core Controller • Modify Options class to support the new config options • Properly configure SpawningKit • Kill generic process upon shutdown
  33. 33. PLAN Integration mode frontend Core Controller Core ApplicationPool • Add user config (e.g. passenger_app_start_command) • Process info from web server
 -OR-
 Perform our own autodetection (Standalone+builtin) • Pass necessary info to ApplicationPool Web server • Autodetect whether auto-supported or generic app • Pass necessary info to Core Controller • Modify Options class to support the new config options • Properly configure SpawningKit • Kill generic process upon shutdown
  34. 34. PLAN Integration mode frontend Core Controller Core ApplicationPool • Add user config (e.g. passenger_app_start_command) • Process info from web server
 -OR-
 Perform our own autodetection (Standalone+builtin) • Pass necessary info to ApplicationPool Web server • Autodetect whether auto-supported or generic app • Pass necessary info to Core Controller • Modify Options class to support the new config options • Properly configure SpawningKit • Kill generic process upon shutdown
  35. 35. CONFIG TO ACTIVATE EACH SPAWNINGKIT FLOW Generic Auto-supported genericApp = true startsUsingWrapper irrelevant appType irrelevant startupFile irrelevant appStartCommand = [user-provided] genericApp = false, startsUsingWrapper = true, appType = (ruby,python,nodejs), startupFile = (autodetected/user-provided), appStartCommand = (path to interpreter) (path to wrapper)
  36. 36. Time to code!
  37. 37. Join the discussion on Github! Thanks for watching! https://github.com/phusion/passenger/issues/2135

×