Linux Internals - Part 3


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Linux Internals - Part 3

  1. 1. Linux Internals (Day 3) Pradeep D. Tewani ©2012 Pradeep Tewani <> All Rights Reserved.
  2. 2. What to Expect?Debugging with GdbBuilding Statically Linked ModulesCompiling and Building Complex applicationusing makeInter Process Communication Mechanisms ©2012 Pradeep Tewani <> 2 All Rights Reserved.
  3. 3. Debugging With GDB ©2012 Pradeep Tewani <> All Rights Reserved.
  4. 4. Ways of DebuggingPrinting The most basic type of debuggingUsing GUI Various IDEs provide this facility ddd in Linux, VDDText mode debugger gdb ©2012 Pradeep Tewani <> All Rights Reserved.
  5. 5. Debugger: gdbText mode debugger Repetition of previous commandVery powerfulAll kind of options: Breakpoint, Watch, ...Extensive HelpGUI interfaces: ddd, ... ©2012 Pradeep Tewani <> All Rights Reserved.
  6. 6. Gdb: Debugging StrategiesOh no!!! Another tool to learn Worth learning a new toolSegmentation faults & Core dumps Execution intercepted by gdb Allows examining the state, backtrace, etcDebugging is Narrowing Down Reaching close & then stepping ©2012 Pradeep Tewani <> All Rights Reserved.
  7. 7. gdb UsageCompile with -g option gcc -g file.c -o fileRun gdb (with desired options) gdb [options] ./file [core file]Pass command line arguments by setting args set args <cmd_line_args>Run the program by Typing c, or run <cmd_line_args>For debugging: break, watch, backtrace, …For help: help <command> ©2012 Pradeep Tewani <> 7 All Rights Reserved.
  8. 8. gdb: Other ways of Runningstep [ count ] – step into sub-routinesnext – run over sub-routines in a gofinish – run till current function returnsreturn – make selected stack frame return to itscallerjump <address> – continue program at specifiedline or address ©2012 Pradeep Tewani <> 8 All Rights Reserved.
  9. 9. gdb: Breakpointsinfo break – Show list & status of breakpointsSetting breakpoints (returns its number) [t]break <function> [ if <expr> ] [t]break [<file>:]<line>Breakpoint Operations disable <break_no> enable <break_no> delete <break_no> commands <break_no> Execute gdb commands on reaching <break_no> ©2012 Pradeep Tewani <> 9 All Rights Reserved.
  10. 10. gdb: Examining Dataprint[/xduotcf] <expr> Evaluate the valid C <expr> within current frameset <var> = <expr> Assign <expr> to <var> in the current scope Variables starting with $ are local to gdbdisplay <expr> Print <expr> whenever program stopsundisplay Cancels all previous display requests ©2012 Pradeep Tewani <> 10 All Rights Reserved.
  11. 11. gdb: ShortcutsJust enough of command To make it unique Added with tab completionFirst letter only for break, delete, run, continue, step, next, printRepeat last by <Enter>Executing the same set of initial commands Put them in .gdbinit -x <gdb command file> ©2012 Pradeep Tewani <> 11 All Rights Reserved.
  12. 12. gdb: MiscellaneousSet editmode for gdb command line editmode [ emacs | vi | dumb ]Execute a shell command shell <cmd>Print command history historySet logging set logging <on | off> set logging file <log_file> (Default: gdb.txt) ©2012 Pradeep Tewani <> 12 All Rights Reserved.
  13. 13. Statically Linked ModulesStatic Modules are simply a collection of ordinary object filesLinked with and becomes the part of the applicationConventionally ends with “.a” suffixAdvantages Simplifies the reusability Allows the application vendors a way to simply release an API ©2012 Pradeep Tewani <> 13 All Rights Reserved.
  14. 14. Building Statically Linked ModulesCompile and get the object codes gcc -c <file1.o> <file2.o> <file3.o> ...Create the static library using ar utility ar rcu libmylib <file1.o> <file2.o> <file3.o> ...Compile a C program to be linked with Libraries gcc <filename.c> -L<path> -lmylib -o <executable> ©2012 Pradeep Tewani <> 14 All Rights Reserved.
  15. 15. Makefile RevisitedSpecial Variables $@ - Target $^ - Dependency %.o – All the .o files in the directory %.c – All the .c files in the Directory %.o: %.c gcc -c $^ -o $@Normal Variables Conventionally declared in CapitalsFunctions Written in Brackets SRCS:= $(wildcard *.c) ©2012 Pradeep Tewani <> 15 All Rights Reserved.
  16. 16. Makefile Revisited...Standard Variables CC - c RM – rm -rfFlags CFLAGS, LDFLAGS, ASF, ARFImplicit Commands %.o: %.c cc -c $^ -o $@make looks for Makefiles in this sequence GNUMakefile, makefile, Makefile Non Standard Makefile using -f option ©2012 Pradeep Tewani <> 16 All Rights Reserved.
  17. 17. Inter Process Communication ©2012 Pradeep Tewani <> All Rights Reserved.
  18. 18. Ws of IPCMechanism by which one process communicatei.e. exchange the data with another process.Require Special IPC mechanism because Processes are non-sharing entities They need to coordinate They need to exchange dataSeparate mechanisms are answers to this ©2012 Pradeep Tewani <> 18 All Rights Reserved.
  19. 19. Example of IPCPrint the filenames in a directoryls outputs the filenameslpr prints its inputBut ls & lpr are separate processesWhat are the solutions? Put into a file -> ls > file; lpr < file Set up an IPC, say a pipe (I) -> ls | lpr ©2012 Pradeep Tewani <> 19 All Rights Reserved.
  20. 20. Types of IPC mechanismsPipe: Allows the flow of data in one direction onlyName Pipe: Pipe with a specific nameShared Memory & Semaphore Allows a interchange of data through a defined area of Memory Semaphore is use for synchronizationMapped Memory: Similar to shared memory, but use fileinstead of memoryWhy so many? Based on requirements, flexibility & benefits ©2012 Pradeep Tewani <> 20 All Rights Reserved.
  21. 21. Pipes©2012 Pradeep Tewani <> All Rights Reserved.
  22. 22. Ws of PipeTwo ends – Read & Write to allow the transferUnidirectional and serial communicationLimited CapacityProvides automatic synchronization Read blocks on no data Write blocks on full data ©2012 Pradeep Tewani <> 22 All Rights Reserved.
  23. 23. Pipe UsageOn shell Using |In Programming / C Creation Using system call pipe()Communicate as with other fd Using the system calls read(), write(), ... ©2012 Pradeep Tewani <> 23 All Rights Reserved.
  24. 24. Creating a PipeInvoke the pipe() system callSupply an integer array of 2The call to a pipe stores the reading filedescriptor at array index 0 and writing filedescriptor at index 1.Lets try pipe.c ©2012 Pradeep Tewani <> 24 All Rights Reserved.
  25. 25. Limitations of Pipepipe() call must precede a call to fork()Pipes are only for related processes That is processes with related ancestoryWhat about unrelated processes?There are special types of pipes called... ©2012 Pradeep Tewani <> 25 All Rights Reserved.
  26. 26. FIFO©2012 Pradeep Tewani <> All Rights Reserved.
  27. 27. Ws of FIFOA Pipe with a name in the FilesystemAlso called the Named PipeProperties identical to those of pipe including the serial communication which is now termed as First-In-First-OutExcept that now the processes can be unrelated ©2012 Pradeep Tewani <> 27 All Rights Reserved.
  28. 28. FIFO UsageOn Shell Creation: mknod, or mkfifo Communication: cat, echo..In programming / C Creation System Call: mknod(), open(), close() Library function: mkfifo(), fopen(), fopen(), fclose() Communication System calls : read, write,... Library Functions: fread(), fwrite(), fputs() ©2012 Pradeep Tewani <> 28 All Rights Reserved.
  29. 29. Experiments on ShellMkfifo /tmp/fifo Open another shellls -l /tmp/fifo cat > /tmp/fifocat < /temp/fifo type some text<See the text> rm /tmp/fifo ©2012 Pradeep Tewani <> 29 All Rights Reserved.
  30. 30. Using System call to create FIFOmknod(const char *filename, mode_t mode, dev_t dev) filename – name for the FIFO Mode Permission to use and type of node to be created The node type must be S_IFIFO for FIFO dev_t specifies the major and minor number of device special file Not used for FIFO, so make it 0. Lets try an example ©2012 Pradeep Tewani <> 30 All Rights Reserved.
  31. 31. Shared Memory©2012 Pradeep Tewani <> All Rights Reserved.
  32. 32. Ws of Shared MemoryRawest communication mechanismPiece of Memory which is shared Between two or more processesFastest form of IPC As no unnecessary copying of data ©2012 Pradeep Tewani <> 32 All Rights Reserved.
  33. 33. Linux Memory ModelAll accesses in Linux are through virtual addressesEach Process has its “own” virtual address space which is split into Virtual PagesEach Process maintains a mapping table From its Physical pages to these virtual pages For access to its actual dataAnd mappings of multiple processes can point to the samephysical page Enabling the sharing of “actual” memory ©2012 Pradeep Tewani <> 33 All Rights Reserved.
  34. 34. Shared Memory UsageShell Way Create: ipcmk -M <size> [-p <mode> ] Get the information: ipcs [-m [ -i -id]] Remove: ipcrm [-M <shmkey> | -m <shmid>]Programming way in Linux Physical Memory Allocation: shmget() Mapping / Attaching to the Shared Memory: shmat() Access through usual (Virtual) Address Dereferencing Unmapping / Detaching from Virtual Memory – shmdt() Physical Memory Deallocation & other shared memory Control Operations: Shmctl() ©2012 Pradeep Tewani <> 34 All Rights Reserved.
  35. 35. Shared Memory AllocationDone by only one of the sharing process int shmget(key_t key, size_t size, int shm_flg);Returns Shared Memory Id on success -1 on errorkey: System-wide unique id for a shared Memory Process gets a particular shared memory id using creation keySize: Number of bytes to allocate or get for the shared memoryshm_flg: operation | permissions IPC_CREAT, IPC_EXCL ©2012 Pradeep Tewani <> 35 All Rights Reserved.
  36. 36. Shared Memory AttachmentDone by all the sharing Processes to access the Shared Memory void *shmat(int shm_id, const void *shm_addr, int shm_flg);Returns Virtual start address of the attached shared memory on success MAP_FAILED i.e (void *)(-1) on error (and sets errno)shm_id: Shared Memory Id returned by shmget()shm_addr Requested process vitual start address NULL enables the Kernel to choose oneshm_flg SHM_RND: round down shm_addr to a multiple of the page size SHM_RDONLY: shared memory will only be read, not writtenChildren created by fork() inherit attached shared segments ©2012 Pradeep Tewani <> 36 All Rights Reserved.
  37. 37. Shared Memory DetachmentDone by all the process which did attach or inherit int shmdt(const void *shm_addr)Returns : 0 on success; -1 on error (and sets errno)shm_addr: Process Virtual address map returned by shmatCall to _exit() or exec*() automatically detachesDetachment may deallocates If done by last process And already marked to deallocate ©2012 Pradeep Tewani <> 37 All Rights Reserved.
  38. 38. Shared Memory Control OpsControl Operations on any Shared memory like Physical Memory Deallocation for Shared Memory Any otherint shmctl(int shm_id, int cmd, struct shmid_ds *buf)Retuns 0 on success, -1 on error (and set errno)shm_id: Shared Memory id returned by shmget()Cmd IPC_STAT - Get the shared memory information into buf IPC_SET - Set the various shared memory info specified by buf IPC_RMID – (Mark) Deallocate the physical memory (buf is ignored) ©2012 Pradeep Tewani <> 38 All Rights Reserved.
  39. 39. Need for SynchronizationShared memory has no Access CoordinationIt has to be handled by us – the UserMoreover, for communication among theunrelated processes, you need a synchronizationmechanismSo, now its a time to look into resolving itThats where we go to the next topic.... ©2012 Pradeep Tewani <> 39 All Rights Reserved.
  40. 40. Process Semaphores ©2012 Pradeep Tewani <> All Rights Reserved.
  41. 41. Ws of Process SemaphoresFirst, it isnt any form of IPC But a mechanism for synchronization used by IPCsCounter used to provide the synchronized access To a shared data object for multiple processesVariable protected by kernel No direct access by the userCounter > 0 -> Resource is available == 0 -> Resource is busy / in use < 0 -> Should never happen ©2012 Pradeep Tewani <> 41 All Rights Reserved.
  42. 42. Semaphore TypesCounting Semaphores Semaphores controlling a multiple resources availability Possible Values >= 1 -> Available 0 -> UnavailableBinary Semaphores Semaphores controlling a single resource availability Possible Values 1 -> Available 0 – Unavailable Typically initialized to 0 ©2012 Pradeep Tewani <> 42 All Rights Reserved.
  43. 43. Process Semaphore UsageShell Way Create : ipcmk -S <nsems> [-p <mode>] Get information: ipcs [-s [-i id]] Remove: ipcrm [-S <semkey> | -s <semid>]Programming way in C (using sys calls) Semaphore Allocation/Access: semget() Semaphore Deallocation & other Semaphore control operations: semctl() Semaphore usage Operations: semop() ©2012 Pradeep Tewani <> 43 All Rights Reserved.
  44. 44. Process Semaphore AllocationDone by only one of the sharing process int semget(key_t key, int nsems, int sem_flg);Returns Semaphore array Id on success -1 on error (and sets errno)key: System-wide unique id for a semaphore array Process gets a particular semaphore array id using creation keynsems: Number of semaphores to create or access in the Arraysem_flg: operation | permissions IPC_CREAT, IPC_EXCL ©2012 Pradeep Tewani <> 44 All Rights Reserved.
  45. 45. Semaphore Control OpsControl operations on semaphore like Deleting a Semaphore Array And many moreint semctl(int sem_id, int sem_num, int cmd, ...)Returns 0 on success, -1 on error (set errno)sem_id: Semaphore array id returned by semgetsem_num: Index of the semaphore in an arryCmd IPC_STAT - Get the semaphore information array into optional arg (sem_num is ignored) IPC_SET – Set the various semaphore array info specified by optional arg (sem_num is ignored) IPC_RMID – Remove the semaphore array (sem_num is ignored) SETVAL – Set sem_numth semaphore value as specified by optional arg ... ©2012 Pradeep Tewani <> 45 All Rights Reserved.
  46. 46. The Optional ArgUnion semun{ int val; //Value for set val struct semid_ds *buf; //Buffer for IPC_STAT & IPC_SET unsigned short *array; //Array for GETALL, SETALL .... } struct semid_ds { struct ipc_perm sem_perm; //Ownership & Permissions time_t sem_otime; // Time of last semop time_t sem_ctime; //Time of last change unsigned short sem_nsems; //No. Of semaphores in the array } ©2012 Pradeep Tewani <> 46 All Rights Reserved.
  47. 47. Caution On Deleting Process SemaphoresInvoking _exit() or exec* doesnt do itNeeds explicit removal Using IPC_RMID cmd of semctlOtherwise, may violate the system limit Of total number of process semaphores ©2012 Pradeep Tewani <> 47 All Rights Reserved.
  48. 48. Initializing the Process Semaphores Two ways though both using semctl Initialize all in the array cmd = SETALL arg.array = Array of values to be set Initializing one at a time sem_num = Semaphore number to be initialized cmd = SETVAL arg.val = Value to be set ©2012 Pradeep Tewani <> 48 All Rights Reserved.
  49. 49. Semaphore Usage OpsUsage operations on the Semaphore like Increment, Decrement, ..Int semop(sem_id, struct sembuf *sops, unsigned intnsops)Returns: 0 on success; -1 on error (and sets errno)semid: Semaphore array id returned by sem_getsops: Array of operations to be performedNsops: Number of elements n sops ©2012 Pradeep Tewani <> 49 All Rights Reserved.
  50. 50. Operation Structure Decodedstruct sembuf{ unsigned short sem_num; // Index of semaphore to be operated on short sem_op; //Value to added to the current semaphore short sem_flg; //Operation flags}Operation Specifies sem_op > 0 -> value added & Processes waiting on it are signalled sem_op < 0 and current semaphore value >= |sem_op| -> value added sem_op < 0 and current semaphore value < | sem_op| It would block/wait until the semaphore value becomes >= |sem_op| Operation Flags IPC_NOWAIT – Prevents the operation from blocking Return -1 with EAGAIN in the above blocking case Kernel automatically undos the semaphore operations on the process exit ©2012 Pradeep Tewani <> 50 All Rights Reserved.
  51. 51. What all have we learnt?Debugging with GdbBuilding Statically Linked ModulesCompiling and Building Complex applicationusing makeInter Process Communication Mechanisms ©2012 Pradeep Tewani <> 51 All Rights Reserved.
  52. 52. Any Queries?©2012 Pradeep Tewani <> 52 All Rights Reserved.