Introduction to GNU Make Programming Language


Published on

The digest of "Software Build System" Chap6

Published in: Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Introduction to GNU Make Programming Language

  1. 1. Make@brownylin
  2. 2. Outline• The GNU Make Programming Lang.• Real-World Build System Scenarios
  3. 3. Introduction• A rule (providing all the inter-file dependency information)• Time-stamp driven commands rule target sources myprog:  prog.c  lib.c          gcc  –o  myprog  prog.c  lib.c commands
  4. 4. The GNU Make Prog. Lang.• 3 components ‣ File  dependencies:  GNU-­‐Make  performs  a   pa1ern-­‐matching  opera8on  to  decide  which   rule  to  evaluate  next ‣ Shell  commands:  A  list  of  shell  commands   encapsulated  within  each  rule,  to  be  executed   if  the  target  of  the  rule  is  out-­‐of-­‐date ‣ String  processing:  A  language  for  manipula8ng   GNU  Make  variables  (Func8onal  programming   paradigm)
  5. 5. 3 Sublanguages• File dependencies myprog:  prog.c  lib.c  • Shell commands cp  myfile  yourfile  &&  cp  myfile1  yourfile1   md5  <  myfile  >>yourfile   touch  yourfile.done• String processing VARS  :=  $(sort  $(filter  srcs-­‐%  cflags-­‐%,  $(.VARIABLES)))
  6. 6. Programming Paradigm• Imperative programming (C++, Java, OOP) ‣ Computa8on:  Execu8ng  statements  to  change   states ‣ Program:  Consists  of  a  sequence  of  commands• Functional programming (Lisp, Haskell, Erlang) ‣ Computa8on:  Evalua8on  of  expression ‣ Expression:  Formed  by  using  func8on  to  combine   basic  values
  7. 7. Create Sandwich• Imperative 1. Take  a  bread 2. Spread  bread  with  bu1er 3. Put  cheese  on  the  bread 4. return  sandwich• Functional ‣ return  put(  cheese,                                            spread(bu1er,  bread)  )
  8. 8. Simple Makefikecalculator:  add.o  calc.o  mult.o  sub.o          gcc  -­‐g  -­‐o  calculator  add.o  calc.o  mult.o  sub.oadd.o:  add.c  numbers.h          gcc  -­‐g  -­‐c  add.ccalc.o:  calc.c  numbers.h          gcc  -­‐g  -­‐c  calc.cmult.o:  mult.c  numbers.h          gcc  -­‐g  -­‐c  mult.csub.o:  sub.c  numbers.h          gcc  -­‐g  -­‐c  sub.c
  9. 9. Makefile Rule Types (1/3) • Rules with multiple targets file1.o  file2.o:  source1.c  source2.c  source3.c          ...  commands  go  here  ... • Rules with no prerequisites .PHONY:  help   help: @echo  “Usage:  make  all  ARCH=[i386|mips]” @echo  “              make  clean”the PHONY directive make GNU Make always execute the rule(even a file named help exists)
  10. 10. Makefile Rule Types (2/3)• Rules with patterns in the filename %.o:  %.c   ...  commands  go  here  ...• Rules that apply only to certain files a.o  b.o:  %.o:  %.c          echo  This  rule  is  for  a.o  and  b.o c.o  d.o:  %.o:  %.c          echo  This  rule  is  for  c.o  and  d.o
  11. 11. Makefile Rule Types (3/3)• Multiple rules with the same target ‣ Only  one  of  these  rules  can  contain  a  set  of   shell  commands chunk.o:  chunk.c          gcc  –c  chunk.c chunk.o:  chunk.h  list.h  data.h
  12. 12. Makefile Variables• The rules ‣ Variables  are  given  a  value  by  an  assignment   statement X  :=  5 ‣ Variable  values  are  referenced  using  the  syntax $(X) ‣ All  variables  are  of  string  type.  No  mechanism   exists  for  declaring  variables  before  they’re   used ‣ Variables  have  global  scope  (within  a  single   makefile)
  13. 13. Immediate Evaluation• Using the := operator FIRST  :=  Hello  there   SECOND  :=  World  #  comments  go  here   MESSAGE  :=  $(FIRST)  $(SECOND)   FILES  :=  add.c  sub.c  mult.c   $(info  $(MESSAGE)  –  The  files  are  $(FILES))• Output Hello  there  World  –  The  files  are  add.c  sub.c  mult.c
  14. 14. Deferred Evaluation• Using = instead of := CC  :=  gcc   CFLAGS  :=  -­‐g   CCOMP  =  $(CC)  $(CFLAGS)     $(info  Compiler  is  $(CCOMP))   CC  :=  i386-­‐linux-­‐gcc   $(info  Compiler  is  $(CCOMP))• CCOMP variable isn’t evaluated until the variable is actually used $  gmake   Compiler  is  gcc  –g   Compiler  is  i386-­‐linux-­‐gcc  -­‐g
  15. 15. Conditional Assignment• Assign a value if the variable doesn’t already have one CFLAGS  :=  -­‐g   CFLAGS  ?=  -­‐O   $(info  CFLAGS  is  $(CFLAGS))• Useful when you include one makefile from within another (default value)
  16. 16. Built-In Variables and Rules (1/2)• automatic variables ‣ $@:  Contains  the  filename  of  the  current  rule’s   target ‣ $<:  Represents  the  first  prerequisite  of  a  rule ‣ $^:  Similar  to  $<,  but  it  evaluates  to  the  complete   list  of  prerequisites  in  the  rule,  with  spaces  between   them ‣ If  the  target  is  /home/john/work/src.c,  $(@D)   evaluates  to  /home/john/work;  $(@F)evaluates  to   src.c
  17. 17. Built-In Variables and Rules (2/2) • build-in rules ‣ show  it  use   gmake  –p • built-in rule for C compilationCOMPILE.c  =  $(CC)  $(CFLAGS)  $(CPPFLAGS)  $(TARGET_ARCH)  –c  OUTPUT_OPTION  =  -­‐o  $@  %.o:  %.c          $(COMPILE.c)  $(OUTPUT_OPTION)  $<
  18. 18. Calculator Example (1/3)calculator:  add.o  calc.o  mult.o  sub.o          gcc  -­‐g  -­‐o  calculator  add.o  calc.o  mult.o  sub.oadd.o:  add.c  numbers.h          gcc  -­‐g  -­‐c  add.ccalc.o:  calc.c  numbers.h          gcc  -­‐g  -­‐c  calc.cmult.o:  mult.c  numbers.h          gcc  -­‐g  -­‐c  mult.csub.o:  sub.c  numbers.h          gcc  -­‐g  -­‐c  sub.c
  19. 19. Calculator Example (2/3) • Use build-in C compilation rulecalculator:  add.o  calc.o  mult.o  sub.o          gcc  -­‐g  -­‐o  calculator  add.o  calc.o  mult.o  sub.oadd.o  calc.o  mult.o  sub.o:  numbers.h
  20. 20. Calculator Example (3/3)SRCS  =  add.c  calc.c  mult.c  sub.cPROG  =  calculator  CC  =  gcc  CFLAGS  =  -­‐gOBJS  =  $(SRCS:.c=.o)  $(PROG):  $(OBJS)          $(CC)  $(CFLAGS)  -­‐o  $@  $^  $(OBJS):  numbers.h
  21. 21. Data Structures and Functions • All of GNU Make’s variables are of string type • How to store complex data? ‣ As  long  as  you  have  a  mechanism  for   extrac8ng  specific  items  out  of  a  list,  you  can   treat  this  variable  like  a  structured  data  typePROG_NAME  :=  my-­‐calculator  LIST_OF_SRCS  :=  calc.c  main.c  math.h  lib.c  COLORS  :=  red  FF0000  green  00FF00  blue  0000FF  purple  FF00FF  ORDERS  :=  100  green  cups  200  blue  plates
  22. 22. Common functions (1/4)• words: Given a list as input, returns the number of space-separated words in that list NUM_FILES  :=  $(words  $(LIST_OF_SRCS)) #  $(NUM_FILES)  evaluates  to  4• word: Given a list, extracts the nth word from that list SECOND_FILE  :=  $(word  2,  $(LIST_OF_SRCS)) #  $(SECOND_FILE)  evaluates  to  main.c
  23. 23. Common functions (2/4)• filter: Returns the words from a list, which match a specific pattern C_SRCS  :=  $(filter  %.c,  $(LIST_OF_SRCS)) #  all  C  source  files• patsubst: For each word in a list, replaces any that match a specific pat- tern with a replacement pattern OBJECTS  :=  $(patsubst  %.c,%.o,  $(C_SRCS)) #  similar  to  the  $(C_SRCS:.c=.o)
  24. 24. Common functions (3/4)• addprefix: For each word in a list, prepends an additional string OBJ_LIST  :=  $(addprefix  objs/,  $(OBJECTS)) #  $(OBJ_LIST)  evaluates  to   objs/calc.o  objs/main.o  objs/lib.o• foreach: Visits each word in a list and constructs a new list containing the “mapped” valuesOBJ_LIST_2  :=  $(foreach  file,  $(OBJECTS),objs/$(file))#  $(OBJ_LIST_2)  evaluates  to  objs/calc.o  objs/main.o  objs/lib.o
  25. 25. Common functions (4/4) • dir/notdir: Given a file’s pathname, returns the directory name component or the filename componentDEFN_PATH  :=  src/headers/idl/interface.idl  DEFN_DIR  :=  $(dir  $(DEFN_PATH))  #  src/headers/idl/DEFN_BASENAME  :=  $(notdir  $(DEFN_PATH))  #  interface.idl • shell: Executes a shell command and returns the command’s output as a string PASSWD_OWNER  :=  $(word  3,  $(shell  ls  -­‐l  /etc/passwd))
  26. 26. Concept of a macro (1/2) • Associate a name with a complex GNU Make expression and to pass arguments into that expressionfile_size  =  $(word  5,  $(shell  ls  -­‐l  $(1)))  PASSWD_SIZE  :=  $(call  file_size,/etc/passwd)#  use  the  $(1)  syntax  to  reference  the  first  parameter  of  the  $(call)  expression
  27. 27. Concept of a macro (2/2)• Define a canned sequence of shell commands by using the define directive define  start-­‐banner          @echo  ==============          @echo  Starting  build        @echo  ==============   endef .PHONY:  all   all:        $(start-­‐banner)          $(MAKE)  -­‐C  lib1
  28. 28. Understanding Program Flow• Parsing a makefile ‣ Reading  the  makefile  to  build  the  dependency  graph ‣ Execu8ng  the  compila8on  commands• Controlling the parsing process ‣ Include  a  submakefile,  or  condi8onally  compile  parts   of  the  makefile• Executing the rules ‣ The  order  in  which  rules  are  applied  and  the   corresponding  shell  commands  are  executed
  29. 29. Parsing a makefile (1/2)1. Parsing phase • All  rules  are  scanned,  all  variable  assignments  are   performed,  and  all  variables  and  func8ons  are   evaluated2. Execution phase • GNU  Make  examines  the  8me  stamps  on  all  the   files  to  determine  which  files  (if  any)  are  out  of   date • The  appropriate  shell  commands  are  executed  to   bring  those  targets  up-­‐to-­‐date
  30. 30. Parsing a makefile (2/2) X  :=  Hello  World print:  echo  X  is  $(X) X  :=  Goodbye   $  gmake  print   X  is  Goodbye• Build system’s functionality can be implemented by ‣ Using  GNU  Make  func8ons  (processed  during   the  first  phase) ‣ As  part  of  a  shell  script  (processed  during  the   second  phase)
  31. 31. Controlling the Parsing Process • File inclusionFILES  :=  src1.c  src2.cinclude   #  content  of  textually                                      #  inserted  heresrc1.o  src2.o:  src.h • Conditional compilation CFLAGS  :=  -­‐DPATH="/usr/local"   ifdef  DEBUG        CFLAGS  +=  -­‐g  #  debug  case  if  DEBUG  is  defined else        CFLAGS  +=  -­‐O  #  non-­‐debug  case  if  DEBUG  not  defined endif
  32. 32. Executing the Rules (1/3)1. Invokes GNU Make (gmaike) must specify which target to build (default: first target listed in the makefile)2. Locates a rule to generate the target file, it examines each of the prerequisites listed in that rule and treats them recursively as targetsExample: Before linking add.o and calc.o into the calculator executable program, GNU Make recursively searches for rules that have add.o or calc.o on the left side
  33. 33. Executing the Rules (2/3)3. If a rule is found for the target you’re trying to satisfy a. If  the  target  file  for  the  rule  doesn’t  yet  exist,   the  rule’s  shell  command  sequence  is   executed  and  the  file  is  created  for  the  first   8me b. If  the  target  file  already  exists  on  the  disk,  the   8me  stamp  on  each  of  the  prerequisite  files  is   examined  to  see  if  any  are  newer  than  the   target  file
  34. 34. Executing the Rules (3/3)4. If step 3 fails, meaning that the makefile doesn’t contain a suitable rule to generate a target file a. If  the  target  file  exists  on  the  disk  (but  there’s   no  rule  to  regenerate  it),  GNU  Make  can  only   assume  that  this  is  a  source  file  that  was   handwri1en  by  the  developer b. If  the  target  file  doesn’t  exist  on  the  disk,  GNU   Make  aborts  with  an  error  and  the  build  fails
  35. 35. Build System Scenarios1. Source Code in a Single Directory2. Multiple Directories a. Source  Code  in  Mul8ple  Directories b. Recursive  Make  over  Mul8ple  Directories c. Inclusive  Make  over  Mul8ple  Directories3. Defining New Compilation Tools4. Building with Multiple Variants5. Cleaning a Build Tree6. Debugging Incorrect Builds
  36. 36. 1. Source Code in a Single Directory SRCS  =  add.c  calc.c  mult.c  sub.c   PROG  =  calculator   CC  =  gcc   CFLAGS  =  -­‐g OBJS  =  $(SRCS:.c=.o)   $(PROG):  $(OBJS)   $(CC)  $(CFLAGS)  -­‐o  $@  $^   $(OBJS):  numbers.h• Newly added source file didn’t actually include numbers.h?• Additional header files were added?
  37. 37. 1. Source Code in a Single Directory 1. Automatically generate a new dependency information file (with .d suffix)-­‐include  $(SRCS:.c=.d)%.d:  %.c          @$(CC)  -­‐MM  $(CPPFLAGS)  $<  |  sed  s#(.*).o:  #1.o          1.d:  #g  >  $@ 2. Uses the makedepend command.
  38. 38. these cases, the so 2-a. Source Code in Multiple but are instead sp Directories shows the tree for • Problem ‣ Harder  dependency  genera8on ‣ Developer  conten8on  on  the  single   makefile ‣ Inability  to  subdivide  the  program • How? Divide the build description across more than one makefile Figure 6.1 TheSRCS  =  libmath/clock.c  libmath/letter.c  libmath/number.c     sou        libprint/banner.c  libprint/center.c  libprint/normal.c              calc/calc.c  ... For the first at gram, but the SRC
  39. 39. source directory 2-b. Recursive Make over cursively invokin Multiple Directories tory tree, with ea• Recursive Make ‣ A  different  makefile  in  each  source   directory,  with  the  high-­‐level  makefileSRCS  =  clock.c  letter.c  number.c  LIB  =  libmath.a  CC  =  gcc  CFLAGS  =  -­‐gOBJS  =  $(SRCS:.c=.o)  $(LIB):  $(OBJS)          $(AR)  cr  $(LIB)  $(OBJS)$(OBJS):  math.h Figure 6.2 Multid
  40. 40. 2-b. Recursive Make over Multiple Directories libmath/Makefile libprint/MakefileSRCS  =  clock.c  letter.c  number.c   SRCS  =  banner.c  center.c  normal.c  LIB  =  libmath.a   LIB  =  libprint.a  CC  =  gcc   CC  =  gcc  CFLAGS  =  -­‐g CFLAGS  =  -­‐gOBJS  =  $(SRCS:.c=.o)   OBJS  =  $(SRCS:.c=.o)  $(LIB):  $(OBJS)   $(LIB):  $(OBJS)          $(AR)  cr  $(LIB)  $(OBJS)        $(AR)  cr  $(LIB)  $(OBJS)$(OBJS):  math.h $(OBJS):  printers.h SRCS  =  banner.c  center.c  normal.c   LIB  =  libprint.a   include   $(OBJS):  printers.h
  41. 41. 2-b. Recursive Make over Multiple Directories calc/MakefileSRCS  =  calc.c  PROG  =  calculator  LIBS  =  ../libmath/libmath.a  ../libprint/libprint.a  CC  =  gcc  CFLAGS  =  -­‐g  OBJS  =  $(SRCS:.c=.o)  $(PROG):  $(OBJS)  $(LIBS)          $(CC)  -­‐o  $@  $^
  42. 42. 2-b. Recursive Make over Multiple Directories src/Makefile .PHONY:  all   all:        $(MAKE)  -­‐C  libmath          $(MAKE)  -­‐C  libprint          $(MAKE)  -­‐C  calc• all target has no prerequisites, each of the recursive calls to $(MAKE) happens every time• Not the most efficient solution available
  43. 43. 2-b. Recursive Make over Multiple Directories• Problem? ‣ Hundred  files?  Build  everything  in  the  correct  order   becomes  an  impossible  task ‣ If  the  source  code  in  the  libmath  directory  started  to   use  the  libprint.a  ?  (libmath.a  compiled  first,  so  it   uses  old  libprint.a) ‣ If  you  want  to  build  only  part  of  the  program • If you started in the calc subdirectory and typed gmake, it doesn’t attempt to rebuild any of those files even if they are out-of-date (calc/Makefile doesn’t know how to build libprint.a)
  44. 44. Figure 6.3 illustr 2-c. Inclusive Make over ple because a two-l solution. Multiple Directories• One main makefile, at the top of the source tree (make/• Software developers are only encouraged to view and edit files• GNU Make complexity is hidden inside the make/ file (non-guru software engineers don’t attempt to change the build mechanism by mistake)
  45. 45. 2-c. Inclusive Make over Multiple Directories • files ‣ Designed  to  be  readable  and  editable  by   so`ware  developers ‣ Contain  only  variables  that  developers  care   about src/ src/libraries/math/ SUBDIRS  :=  libraries  application  SRC  :=  add.c  mult.c  sub.c SRC  :=  main.c  CFLAGS  :=  -­‐DBIG_MATH CFLAGS  :=  -­‐g src/libraries/Files.mkSUBDIRS  :=  math  protocols  sql  widgets#  none  of  the  source  files  from  that  directory  would  be  included
  46. 46. 2-c. Inclusive Make over Multiple Directories src/Makefile_subdirs  :=  _curdir  :=  FRAMEWORK  :=  $(CURDIR)/make/  include  include  $(FRAMEWORK)VARS  :=  $(sort  $(filter  srcs-­‐%  cflags-­‐%,  $(.VARIABLES)))  $(foreach  var,  $(VARS),  $(info  $(var)  =  $($(var)))).PHONY:  all  all:          @#  do  nothing
  47. 47. 2-c. Inclusive Make over Multiple Directories _subdirs src/ libraries applications src/libraries/Files.mkapplications libraries/math libraries/protocols libraries/sql libraries/widgets
  48. 48. 2-c. Inclusive Make over Multiple Directories • Main algorithm for traversing the build tree and collecting the values from each fragmentsrcs-­‐$(_curdir)  :=  $(addprefix  $(_curdir),$(SRC))  cflags-­‐$(_curdir)  :=  $(CFLAGS)  _subdirs  :=  $(_subdirs)  $(addprefix  $(_curdir),  $(SUBDIRS))  ifneq  ($(words  $(_subdirs)),0)          _curdir  :=  $(firstword  $(_subdirs))/          _subdirs  :=  $(wordlist  2,  $(words  $(_subdirs)),        $(_subdirs))          SUBDIRS  :=        SRC  :=          CFLAGS  :=          include  $(_curdir)          include  $(FRAMEWORK)  endif make/
  49. 49. 2-c. Inclusive Make over Multiple Directories• Summary ‣ Using  one  instance  of  the  GNU  Make  process   enables  you  to  have  a  single  unified   dependency  graph ‣ Not  an  easy  system  to  create
  50. 50. Pros & Cons• Pros • Cons ‣ Wide  support ‣ Inconsistent  language   design   ‣ Vary  fast  tool ‣ No  standard   ‣ Portable  syntax framework ‣ Fully  featured ‣ Lack  of  portability ‣ The  first  tool ‣ Challenging  debugging ‣ Not  easy  to  use
  51. 51. Evaluation• Convenience (Poor)• Correctness (Poor)• Performance (Excellent)• Scalability (Excellent)• General Rule ‣ Using  GNU  Make  for  legacy  so`ware  that  already   uses  a  Make-­‐based  build  system ‣ First  consider  using  SCons  or  CMake  when   wri8ng  a  new  build  system
  52. 52. Thank you :)