Process Control DaemonFor Embedded Linux PlatformsSpeaker: Hai Shalomrt-embedded.com/pcd
Background review: What were the reasons that led to the development of PCD.PCD project review: Features and high level overview of the project.Live demonstration.Q & A.Agenda
Does your product have a process controller?Does your product automatically recover after a crash?Do you think your product’s boot time is fast enough?Are you using methods other than printf  to debug a crashed application?Are you familiar with all the processes which are running in your product and their dependencies?Some questions
Most of you probably answered “No” to at least one question.People who answered “Yes” to all questions are probably using PCD already!Let’s review some facts about Embedded Linux based products…What were your answers?
Done by scripts (rcS, rc.*). These are great, but might be:Not optimal for embedded / not deterministic:
Limited ways to synchronize depended processes(delay).
Limited ways to verify successful start of a process
No error checking (usually).
No formal way to define dependencies.
Difficult to start processes in parallel.
Not trivial to understand, maintain and extend:
Require additional shell scripting expertise.
Tend to be long and unreadable.
Plenty of commented code, old remarks, different code styles.System start up
Done by scripts (rcS, rc.*). These are great, but might be:Not optimal for embedded / not deterministic:
Limited ways to synchronize depended processes (delay).
Limited ways to verify successful start of a process
No error checking (usually).
No formal way to define dependencies.
Difficult to start processes in parallel.
Not trivial to understand, maintain and extend:
Require additional shell scripting expertise.
Tend to be long and unreadable.
Plenty of commented code, old remarks, different code styles.System start upLooks familiar?
A crashed program just terminates, usually after printing “Segmentation Fault”.Now what?
Where is the debug information?
Kernel crashes are assumed to be handled by the system’s watchdog.Signal Handlers not always implemented correctly.Unsafe to use printf, and many other functions.The system remains unstable and unusable. End user must power-cycle (again?).Crash handling and recovery
A crashed program just terminates, usually after printing “Segmentation Fault”.Now what?
Where is the debug information?
Kernel crashes are assumed to be handled by the system’s watchdog.Signal Handlers not always implemented correctly.Unsafe to use printf, and many other functions.The system remains unstable and unusable. End user must power-cycle (again?).Crash handling and recovery
No central management entity.init is the parent of all processes.Must know process’ pid in order to signal or kill.Each process must manage his own children.Child process inherits his father’s priority.
Parents must retrieve child’sexit status, or else we end upwith Zombies…Process management
A customer reports a crash in the field or in his lab tests:There is no standard method for generating and collecting remote debug information.
When a process abnormally terminates, all its information goes away and no log is saved.
You might be on the next flight to the customer’s lab.Field/Remote debugging
A great (and free) solution: PCDWhat is PCD?
What is PCD?PCD – Process Control Daemon, is an open source, light-weight system level process manager for Embedded-Linux based products (consumer electronics, network devices, etc).The PCD provides a complementary service for any Embedded Linux driven product. Designed and implemented by Hai Shalom during employment at Texas Instruments for Next-Gen Puma5 Cable chipset.Released to open source as part of his M.Sc. Degree research.PCD is a proven solution that already drives millions of devices in the world.
System startup: PCD starts up the system in an efficient, synchronized and deterministic manner.Process management: a centralized entity that controls and monitors all processes, and provides API to manage them. System recovery: Configurable per process recovery action is taken in case of a crash.Debug information: PCD provides a detailed crash log in case of a program error.PCD Features in high-level
How does it work?What are the advantages of products with PCD?
Rule blocks replace/extend traditional shell scripts.Each rule defines a single process.Rule inter-dependency is well defined.PCD Scripts: Rule blocksProcess 1Rule 1PCDScriptFileProcess 2Rule 2Process 3Rule 3
Very simple and readable syntax.Easy to extend and maintain.Each Rule block is based on the same template and contains the following details:What is the process name and parameters?
When to start it (depends on event…)?
What is the required priority?
What is the completion event?
How much time to wait for it to complete?
What to do in case of a crash?PCD Scripts: Rule blocks
Very simple and readable syntax.Easy to extend and maintain.Each Rule block is based on the same template and contains the following details:What is the process name and parameters?
When to start it (depends on event…)?
What is the required priority?

Process control daemon

  • 1.
    Process Control DaemonForEmbedded Linux PlatformsSpeaker: Hai Shalomrt-embedded.com/pcd
  • 2.
    Background review: Whatwere the reasons that led to the development of PCD.PCD project review: Features and high level overview of the project.Live demonstration.Q & A.Agenda
  • 3.
    Does your producthave a process controller?Does your product automatically recover after a crash?Do you think your product’s boot time is fast enough?Are you using methods other than printf to debug a crashed application?Are you familiar with all the processes which are running in your product and their dependencies?Some questions
  • 4.
    Most of youprobably answered “No” to at least one question.People who answered “Yes” to all questions are probably using PCD already!Let’s review some facts about Embedded Linux based products…What were your answers?
  • 5.
    Done by scripts(rcS, rc.*). These are great, but might be:Not optimal for embedded / not deterministic:
  • 6.
    Limited ways tosynchronize depended processes(delay).
  • 7.
    Limited ways toverify successful start of a process
  • 8.
  • 9.
    No formal wayto define dependencies.
  • 10.
    Difficult to startprocesses in parallel.
  • 11.
    Not trivial tounderstand, maintain and extend:
  • 12.
    Require additional shellscripting expertise.
  • 13.
    Tend to belong and unreadable.
  • 14.
    Plenty of commentedcode, old remarks, different code styles.System start up
  • 15.
    Done by scripts(rcS, rc.*). These are great, but might be:Not optimal for embedded / not deterministic:
  • 16.
    Limited ways tosynchronize depended processes (delay).
  • 17.
    Limited ways toverify successful start of a process
  • 18.
  • 19.
    No formal wayto define dependencies.
  • 20.
    Difficult to startprocesses in parallel.
  • 21.
    Not trivial tounderstand, maintain and extend:
  • 22.
    Require additional shellscripting expertise.
  • 23.
    Tend to belong and unreadable.
  • 24.
    Plenty of commentedcode, old remarks, different code styles.System start upLooks familiar?
  • 25.
    A crashed programjust terminates, usually after printing “Segmentation Fault”.Now what?
  • 26.
    Where is thedebug information?
  • 27.
    Kernel crashes areassumed to be handled by the system’s watchdog.Signal Handlers not always implemented correctly.Unsafe to use printf, and many other functions.The system remains unstable and unusable. End user must power-cycle (again?).Crash handling and recovery
  • 28.
    A crashed programjust terminates, usually after printing “Segmentation Fault”.Now what?
  • 29.
    Where is thedebug information?
  • 30.
    Kernel crashes areassumed to be handled by the system’s watchdog.Signal Handlers not always implemented correctly.Unsafe to use printf, and many other functions.The system remains unstable and unusable. End user must power-cycle (again?).Crash handling and recovery
  • 31.
    No central managemententity.init is the parent of all processes.Must know process’ pid in order to signal or kill.Each process must manage his own children.Child process inherits his father’s priority.
  • 32.
    Parents must retrievechild’sexit status, or else we end upwith Zombies…Process management
  • 33.
    A customer reportsa crash in the field or in his lab tests:There is no standard method for generating and collecting remote debug information.
  • 34.
    When a processabnormally terminates, all its information goes away and no log is saved.
  • 35.
    You might beon the next flight to the customer’s lab.Field/Remote debugging
  • 36.
    A great (andfree) solution: PCDWhat is PCD?
  • 37.
    What is PCD?PCD– Process Control Daemon, is an open source, light-weight system level process manager for Embedded-Linux based products (consumer electronics, network devices, etc).The PCD provides a complementary service for any Embedded Linux driven product. Designed and implemented by Hai Shalom during employment at Texas Instruments for Next-Gen Puma5 Cable chipset.Released to open source as part of his M.Sc. Degree research.PCD is a proven solution that already drives millions of devices in the world.
  • 38.
    System startup: PCDstarts up the system in an efficient, synchronized and deterministic manner.Process management: a centralized entity that controls and monitors all processes, and provides API to manage them. System recovery: Configurable per process recovery action is taken in case of a crash.Debug information: PCD provides a detailed crash log in case of a program error.PCD Features in high-level
  • 39.
    How does itwork?What are the advantages of products with PCD?
  • 40.
    Rule blocks replace/extendtraditional shell scripts.Each rule defines a single process.Rule inter-dependency is well defined.PCD Scripts: Rule blocksProcess 1Rule 1PCDScriptFileProcess 2Rule 2Process 3Rule 3
  • 41.
    Very simple andreadable syntax.Easy to extend and maintain.Each Rule block is based on the same template and contains the following details:What is the process name and parameters?
  • 42.
    When to startit (depends on event…)?
  • 43.
    What is therequired priority?
  • 44.
    What is thecompletion event?
  • 45.
    How much timeto wait for it to complete?
  • 46.
    What to doin case of a crash?PCD Scripts: Rule blocks
  • 47.
    Very simple andreadable syntax.Easy to extend and maintain.Each Rule block is based on the same template and contains the following details:What is the process name and parameters?
  • 48.
    When to startit (depends on event…)?
  • 49.
    What is therequired priority?