Practical C++ Generative Programming


Published on

Presentation done at the historic 20 yeras of C++ conference in Las Vegas 2005. This is also the first time I ever spoke on the topic of combing generative programming and C++ template metaprogramming

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Practical C++ Generative Programming

  1. 1. Practical Generative ProgrammingSchalk W. Cronjé
  2. 2. Even in this new millennium, many engineers willstill build components that have very little reusepotential due to the inflexible way that they wereconstructed.This leads to excessive time required to adapt acomponent for usage in another system.
  3. 3. Welcome to the world ofGenerative Programming
  4. 4. Themes• GP 101• Building a team• Building a single system• Technical footwork• Building multiple systems• Integration & maintenance
  5. 5. DefinitionIt is a software engineering paradigm where theaim is to automatically manufacture highlycustomised and optimised intermediate or end-products from elementary, reusable componentsby means of configuration knowledge.
  6. 6. Automatic programming• Generative programming is not automaticprogramming.• AP aims for highest level of automation• GP acknowledges that there are potentialydifferent levels of automation possible in acomplete system.• AP usually involves some form of AI and highamounts of domain knowledge.• GP provides practical leverage of state-of-the artsoftware engineering practices.
  7. 7. Elements of Generative ProgrammingProblem space Solution spaceConfigurationKnowledge•Illegal featurecombinations•Default settings &dependencies•Construction rules•Optimisations•ConfiguredComponents•Domain-specificconcepts•Features
  8. 8. Benefits• Economies of scope● Less time and effort to produce variety of products• Software quality improvement● Reuse of proven components• Scalability● Can be applied to parts of a system or to wholesystems• Optimisation at domain level● Maximal combinability● Minimal redundancy● Maximum reuse
  9. 9. Steps• Domain scoping• Feature & concept modelling• Common architecture design andimplementation technology identification• Domain-specific notations• Specify configuration knowledge (metadata)• Implement generic components• Apply configuration knowledge using generatorsThere is no specific order to these steps !
  10. 10. Configuration Knowledge vsMetadata• Configuration knowledge is the term preferredby Czarnecki & Eisenecker• Configuration knowledge can be considered theholistic encapsulation of all knowledge related tobuilding all variants• Metadata is probably a more codified form ofconfiguration knowledge.• Some people find the term metadata easier tograsp and less confusing than configurationknowledge• The rest of this presentation uses the termmetadata
  11. 11. Introducing GP into theProcess• Start small !● Use a pilot project or a small subset of an existingsystem● Experiential learning is important – learn to learn.• Take an iterative and incremental approach to thedifferent GP steps.• Dont worry too much about modelling up-front.
  12. 12. Building a Team• Domain experts● Require the ability to codify configuration knowledgeinto a reusable form.• Language experts● Converting configuration knowledge in toprogramming language representation.• Guiding light● One person whom understands GP well, ensures thatthe development process stays on track and notdescend into maintenance hell.
  13. 13. Strategies for C++• Templates are the C++ way to genericprogramming• Develop elementary components as genericcomponents● Fully testable outside of the intended productconfiguration• Configure these components using generatedtraits / policy classes• Aim for zero cyclomatic-complexity in thegenerated classes
  14. 14. Template Metaprogramming• MPL is a key technology to build genericcomponents● Best example is Boost C++ MPL• MPL has been suggested as a domain-specificlanguage● Metadata difficult to review to someone not familiarwith MPL• MPL should rather be used as implementationstrategy
  15. 15. Example #1: Configuration systemName: NetworkPortDescription: Unrestricted port on which aservice can be startedType: uint16Minimum Value: 1024Maximum Value: 65535
  16. 16. Example #1: Configuration systemtemplate <typename CfgAttr>typename CfgAttr::value_typeget_config();std::cout << “The network port well use is “ <<get_config<NetworkPort>();
  17. 17. Example #1: A traits classstruct NetworkPort{typedef uint16_t value_type;static const value_type const_min = 1024;static const value_type const_max = 65535;// ... rest to follow};
  18. 18. Example #1: Alternative traitsBecause other non-integral types cannot beinitialised inline, it might be more practical to usethe following alternative.struct NetworkPort{typedef uint16_t value_type;static value_type min_value() {return 1024;}static value_type max_value() {return 65535;}// ... rest to follow};
  19. 19. Example #1: Basic generic functionstd::string get_cfg_string( const char* name );template <typename CfgAttr>typename CfgAttr::value_typeget_config(){// Calls a basic configuration interface functionstd::string tmp=get_cfg_string( CfgAttr::name() );// Converts to appropriate type, throws exception// on conversion failurereturn boost::lexical_cast<typenameCfgAttr::value_type>(tmp);}
  20. 20. Introducing run-time safety• In order to protect the system against externalinvalid data we need to add boundary checks.● Use min_values(), max_value() from traits● Add a default_value() to handle missing data• Additional features could include:● Throwing an exception, instead of defaulting, whendata is missing.
  21. 21. Example #1: Extending the functiontemplate <typename CfgAttr>typename CfgAttr::value_typeget_config(){std::string tmp=get_cfg_string( CfgAttr::name() );if(tmp.empty())return CfgAttr::default_value();else{typedef typename CfgAttr::value_type vtype;vtype ret= boost::lexical_cast<vtype>(tmp);return CfgAttr::bounded(ret);}}
  22. 22. Example #1: Updated traitsstruct NetworkPort{typedef uint16_t value_type;static value_type min_value() {return 1024;}static value_type max_value() {return 65535;}static value_type default_value {return 4321;}static value_type& bounded(value_type& v_){return v_=std::max(min_value(),std::min(v_,max_value()));}};
  23. 23. Capturing Configuration Knowledge• Various methods have been used for codifyingmetadata● Text files● Graphical Tools● CASE Tools• XML is a very convenient form for new projects● Semi-human readable● Text – Unrestricted source-control● Easy to transform to other formats• Includes non-code artefacts● Custom editor can be created in Python or Java
  24. 24. Example #1: Configuration system<ConfigSystem><Attr name=”NetworkPort” adt=”uint16”><Description>Unrestricted port on which aservice can be started</Description><Min>1024</Min><Max>65535</Max><Default>4321</Default></Attr></ConfigSystem>
  25. 25. Prefer ADTs• Use abstract data types (ADTs)• Use a XML lookup table to go from ADT to C++type• Underlying C++ representation can be changedwithout changing any of the metadata
  26. 26. Example #1: Simple Generator<xsl:template match="Attr">struct <xsl:value-of select="@name"/>{typedef <xsl:apply-template select="." mode="adt"/>value_type;static const char* name() {return &quot;<xsl:value-of select="@name"/>&quot;;}static value_type min_value() {return <xsl:value-ofselect="Min/text()"/>;}static value_type max_value() {return <xsl:value-ofselect="Max/text()"/>;}static value_type default_value() {return<xsl:value-of select="Default/text()"/>;}};</xsl:template>
  27. 27. ADT Lookup Table<Types><Type adt=”uint16” posix-type=”uint16_t” win32-type=”WORD” embedded-type=”unsigned short”quoted=”no”/><Type adt=”string” posixtype=”std::string” win32-type=”std::string” embedded-type=”MyFixedString”quoted=”yes”/><!--adt: ADT namewin32-type: What type to use on a Win32 systemposix-type: Use this type on a POSIX systemembedded-type: Type for embedded systems.quoted: Whether to quote the type in a traits class--></Types>
  28. 28. Example #2• Logging is an aspect of most systems thatcrosscuts the architecture.• There might be many requirements in yoursystem, on how logging and reporting is used.● Loggable entities● Levels of logging● User display issues● Localisation• From a C++ point-of-view one important featureis how logging is generated at logging points• Using a GP approach it is possible to introducecompile-time validation
  29. 29. Example #2: Legacy Logging#define MINOR_FAILURE 1#define MAJOR_PROBLEM 2#define GENERAL_PANIC 3void log_it( int id, const char* text );// and then some smartie comes alonglog_it(MINOR_PROBLEM|GENERAL_PANIC,”Voila!! An unsupported error”);
  30. 30. Example #2: Logging Metadata<Logging><Report id=”1” name=”MINOR_FAILURE”><Text>The projectors bulb needs replacing</Text></Report><Report id=”2” name=”MAJOR_PROBLEM”><Text>Were out of Belgium beer</Text></Report><Report id=”3” name=”GENERAL_PANIC”><Text>Elvis has left the building</Text></Report></Logging>
  31. 31. Example #2: Logging Functiontemplate <typename Report>void log_it( Report const&, const char* text );log_it( 3,”My code” ); // compile errorlog_it( MAJOR_PROBLEM, “Out of German beer too” );log_it(MINOR_FAILURE|MAJOR_PROBLEM,“No way” ); // Compile error
  32. 32. Example #2: Logging ID Class// Define typeclass Reportable{public:Reportable( unsigned id_ );unsigned id() const;};// then do either, initialising MINOR_FAILURE in .cppextern const Reportable MINOR_FAILURE;// ornamespace { const Reportable MINOR_FAILURE =Reportable(1); }
  33. 33. Preventing Code-bloat• Only instantiate what is needed● For constant objects this is very easy using the MPL-value idiom• Due to ways some linkers work, concrete code might beincluded in a final link even if the code is not used,therefore only generate what is needed● Control the config elements available to a specificsystem from metadata● Only generate the appropriate traits classes• Cleanly separate common concrete class into a mixinclass
  34. 34. The MPL-value idiomtemplate <int V>class A{public:static const A<V> value;private:A();};template <int V>static const A<V> A<V>::value;
  35. 35. Logging Reworkedtemplate <unsigned id_>class Reportable{public:unsigned id() const {return id_;}static const Reportable<id_> value;};const Reportable<id_> Reportable<id_>::value;typedef Reportable<1> MINOR_FAILURE;log_it( MINOR_PROBLEM::value,”Only instantiated whenused”);
  36. 36. Adding logging actions• A user might want to specify that some reports can havecertain associated actions.• For the logging example we might have● GO_BUY● MAKE_ANNOUNCEMENT● CALL_SECURITY.• As this is configuration knowledge we can addthis to the metadata and then generateappropriate metacode.
  37. 37. Example #2: Logging Metadata<Logging><Report id=”1” name=”MINOR_FAILURE”><Action>GO_BUY</Action></Report><Report id=”2” name=”MAJOR_PROBLEM”><Action>GO_BUY</Action><Action>MAKE_ANNOUNCEMENT</Action></Report><Report id=”3” name=”GENERAL_PANIC”><Action>CALL_SECURITY</Action><Action>MAKE_ANNOUNCEMENT</Action></Report></Logging>
  38. 38. Using MPL as gluetemplate <unsigned id_,typename actions_list>class Reportable{public:unsigned id() const {return id_;}static const Reportable<id_> value;typedef actions_list valid_actions;};// Generated codetypedef Reportable<2,boost::mpl::vector<GO_BUY,MAKE_ANNOUNCEMENT>> MAJOR_PROBLEM;
  39. 39. Using SFINAE as validatortemplate <typename Report,typename Action>void log_it( const char* text,boost::enable_if<boost::mpl::contains<typename Report::valid_actions, Action>::type::value>*_= 0);// Fails to compilelog_it<GENERAL_PROBLEM>( GO_BUY,”Bought Elvis beer”);// OK,log_it<MAJOR_PROBLEM>( GO_BUY,”Imported some Hoegaarden”);
  40. 40. Multiple Systems• Examples until now have shown the GP stepsfor a configuration system and a logging system.• The next step is to apply these to three systems:● System 1 uses XML files for configuration and sendslogs to syslog.● System2 uses INI files, and sends logs to NT Evlog● System 3 keeps configuration in a binary format (read-only) and sends logs via SNMP.
  41. 41. Example Product Metadata<Products><System id=”1”><Config type=”xml”/><Logging type=”syslog”/><Functionality> ... <Functionality></System><System id=”2”><Config type=”ini”/><Logging type=”ntevlog”/><Functionality> ... <Functionality></System></Products>
  42. 42. Building Multiple Systems• Four generators can be applied to this productmetadata.● Two of them we have already seen● These will generate configurations and logging aspects• Another generator looks at logging andconfigurations and adds the appropriatesubsystems.• A fourth generator looks at the functionality andloads up all of the functional classes for thesystem● A creative exercise for the reader …
  43. 43. Testing• Can tests be generated?● There have been various argument around this topic.• Validate metadata independently• Test data can be generated from metadata• DO NOT generate unit tests to validate that thegenerated code is correct!● How can you verify that the generated tests is correct?
  44. 44. Maintenance• Long-tem maintenance requires upfrontinvestment in building quality genericcomponents• Metadata might have to refactored into smallerXML files or a tools should be developed to editthe metadata.• Refactoring components into more genericcomponents over time means that you will beable to configure and customise products evenmore.
  45. 45. Integration• Many modern systems are multi-language /multi-platform• These techniques extend easily into otherprogramming languages / developmentenvironments• The same configuration knowledge remains thedriver• Localisation data can be generated in variousformats• Parts of technical documents can also begenerated.
  46. 46. Further Reading•••
  47. 47. In this new millennium, engineers can build high-quality proven generic components with high reusepotential and adaptively configure them.This leads to decreased time required to use acomponent in another domain and an increase inthe variety of products that can be assembled.Did anyone mention competitive edge?