Your SlideShare is downloading. ×
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Gregor modules
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Gregor modules

232

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
232
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Modules Doug Gregor, AppleWednesday, November 7, 12
  • 2. Roadmap • The fundamental brokenness of headers • A module system for the C family • Building better toolsWednesday, November 7, 12
  • 3. On the Fundamental Brokenness of HeadersWednesday, November 7, 12
  • 4. C Preprocessing Model #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 5. C Preprocessing Model #include <stdio.h> // stdio.h int main() { typedef struct { printf(“Hello, world!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 6. C Preprocessing Model #include <stdio.h> // stdio.h int main() { typedef struct { printf(“Hello, world!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 7. C Preprocessing Model // from stdio.h typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 8. Problems with the Model // from stdio.h typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); • Fragility int remove(const char*); // on and on... • Performance int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 9. Header Fragility #define FILE “MyFile.txt” #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 10. Header Fragility #define FILE “MyFile.txt” // stdio.h #include <stdio.h> typedef struct { int main() { ... printf(“Hello, world!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 11. Header Fragility #define FILE “MyFile.txt” // stdio.h #include <stdio.h> typedef struct { int main() { ... printf(“Hello, world!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 12. Header Fragility // from stdio.h typedef struct { ... } “MyFile.txt”; int printf(const char*, ...); int fprintf(“MyFile.txt” *, const char*, ...); int remove(const char*); // on and on... int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 13. Conventional WorkaroundsWednesday, November 7, 12
  • 14. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROSWednesday, November 7, 12
  • 15. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_HWednesday, November 7, 12
  • 16. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_H • template<class _Tp> const _Tp& min(const _Tp &__a, const _Tp &__b);Wednesday, November 7, 12
  • 17. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_H • template<class _Tp> const _Tp& min(const _Tp &__a, const _Tp &__b); • #include <windows.h> #undef min // because #define NOMINMAX #undef max // doesn’t always workWednesday, November 7, 12
  • 18. How Big is a Source File?Wednesday, November 7, 12
  • 19. How Big is a Source File? #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 20. How Big is a Source File? #include <stdio.h> int main() { printf(“Hello, world!n”); } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 21. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 22. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 23. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 24. Inherently Non-Scalable • M headers with N source files • M x N build cost • C++ templates exacerbate the problem • Precompiled headers are a terrible solutionWednesday, November 7, 12
  • 25. A Module System for the C FamilyWednesday, November 7, 12
  • 26. What Is a Module? • A module is a package describing a library • Interface of the library (API) • Implementation of the libraryWednesday, November 7, 12
  • 27. Module Imports import std; int main() { printf(“Hello, World!n”); } • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 28. Module Imports // std module (includes stdio) import std; typedef struct { int main() { ... printf(“Hello, World!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 29. Module Imports // std module (includes stdio) import std; typedef struct { int main() { ... printf(“Hello, World!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 30. Import Resilience • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 31. Import Resilience #define FILE “MyFile.txt” import std; int main() { printf(“Hello, World!n”); } • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 32. Import Resilience // std module (includes stdio) #define FILE “MyFile.txt” import std; typedef struct { ... int main() { } FILE; printf(“Hello, World!n”); } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 33. Import Resilience // std module (includes stdio) #define FILE “MyFile.txt” import std; typedef struct { ... int main() { } FILE; printf(“Hello, World!n”); } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 34. Selective Import // std module // stdio submodule typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 35. Selective Import // std module import std.stdio; // stdio submodule int main() { typedef struct { printf(“Hello, World!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 36. Selective Import // std module import std.stdio; // stdio submodule int main() { typedef struct { printf(“Hello, World!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 37. What Does import Import? • Functions, variables, types, templates, macros, etc. • Only public API -- everything else can be hidden. • No special namespace mechanism.Wednesday, November 7, 12
  • 38. Writing a Module Futuristic VersionWednesday, November 7, 12
  • 39. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; int printf(const char*, ...) { // ... } int fprintf(FILE *, const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 40. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { // ... } int fprintf(FILE *, const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 41. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { } // ... • Public access int fprintf(FILE *, describes API const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 42. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { } // ... • Public access int fprintf(FILE *, describes API const char*, ...) { } // ... • No headers! int remove(const char*) { // ... }Wednesday, November 7, 12
  • 43. Problems With This Future • Transitioning existing header-based libraries • Interoperability with compilers that don’t implement modules • Requires tools that understand modulesWednesday, November 7, 12
  • 44. Writing a Module Transitional VersionWednesday, November 7, 12
  • 45. Embracing Headers • Build modules directly from the headers • Headers remain “the truth” • Good for interoperability • Doesn’t change the programmer modelWednesday, November 7, 12
  • 46. Module Maps // /usr/include/module.map module std { module stdio { header “stdio.h” } module stdlib { header “stdlib.h” } module math { header “math.h” } } • module defines a named (sub)module • header includes the contents of the named header in the current (sub)moduleWednesday, November 7, 12
  • 47. Umbrella Headers // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } } • An umbrella header includes all of the headers in its directoryWednesday, November 7, 12
  • 48. Umbrella Headers // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } } • An umbrella header includes all of the headers in its directory • Wildcard submodules (module *) create a submodule for each included header • AST/Decl.h -> ClangAST.Decl • AST/Expr.h -> ClangAST.ExprWednesday, November 7, 12
  • 49. Module Map FeaturesWednesday, November 7, 12
  • 50. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * }Wednesday, November 7, 12
  • 51. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * } module _Builtin { module avx { • Submodule requirements } requires avx header “avxintrin.h” }Wednesday, November 7, 12
  • 52. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * } module _Builtin { module avx { • Submodule requirements } requires avx header “avxintrin.h” } • Excluded headers module std { exclude header “assert.h” }Wednesday, November 7, 12
  • 53. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); }Wednesday, November 7, 12
  • 54. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named moduleWednesday, November 7, 12
  • 55. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module fileWednesday, November 7, 12
  • 56. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module file 3. Load the module file at the ‘import’ declarationWednesday, November 7, 12
  • 57. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module file 3. Load the module file at the ‘import’ declaration 4. Cache module file for later re-useWednesday, November 7, 12
  • 58. Adopting Modules: Libraries • Eliminate non-modular behavior: • Multiply-defined structs, functions, macros, must be consolidated • Headers should import what they depend on • Write module maps covering the libraryWednesday, November 7, 12
  • 59. Adopting Modules: Users #include <stdio.h> int main() { printf(“Hello, World!n”); }Wednesday, November 7, 12
  • 60. Adopting Modules: Users #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • “Simply” rewrite each #include as an importWednesday, November 7, 12
  • 61. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } }Wednesday, November 7, 12
  • 62. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d headerWednesday, November 7, 12
  • 63. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d header • Optional Fix-Its, tooling to finalize the rewriteWednesday, November 7, 12
  • 64. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d header • Optional Fix-Its, tooling to finalize the rewrite • Enabling modules is transparent to the userWednesday, November 7, 12
  • 65. Building Better Tools (For the Future)Wednesday, November 7, 12
  • 66. Compilation Performance • Algorithmic improvement for parsing time • Module headers parsed once, cached • MxN M+N • Benefits for all source-based toolsWednesday, November 7, 12
  • 67. Automatic Linking // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } link “-lclangAST” } • Drastically simplifies use of a libraryWednesday, November 7, 12
  • 68. Automatic Import int main() { std::vector<int> v; } • Forgotten #include terrible diagnosticWednesday, November 7, 12
  • 69. Automatic Import int main() { std::vector<int> v; } • Forgotten #include terrible diagnostic • vector.cpp:2:6: error: ‘vector‘ template is not available std::vector<int> v; ^ note: import ‘std.vector‘ to use ‘std::vector’Wednesday, November 7, 12
  • 70. Debugging FlowWednesday, November 7, 12
  • 71. Debugging Flow <iostream> foo.cppWednesday, November 7, 12
  • 72. Debugging Flow <iostream> foo.cpp Clang ASTsWednesday, November 7, 12
  • 73. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 74. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB DWARF object codeWednesday, November 7, 12
  • 75. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object codeWednesday, November 7, 12
  • 76. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code • Round-trip through DWARF is lossy • Only ‘used’ types, functions available • Inline functions, template definitions lostWednesday, November 7, 12
  • 77. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object codeWednesday, November 7, 12
  • 78. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code bar.o iostream DWARF <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 79. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code bar.o iostream DWARF <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 80. Debugging with Modules foo.o <iostream> foo.cpp Clang ASTs foo DWARF object code LLDB Clang ASTs bar.o <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 81. Debugging with Modules std.iostream module foo.o <iostream> foo.cpp Clang ASTs foo DWARF object code LLDB Clang ASTs bar.o <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 82. Debugging with Modules • Improved build performance • Compiler emits less DWARF • Linker de-duplicates less DWARFWednesday, November 7, 12
  • 83. Debugging with Modules • Improved build performance • Compiler emits less DWARF • Linker de-duplicates less DWARF • Improved debugging experience • Perfect AST fidelity in debugger • Debugger doesn’t need to search DWARFWednesday, November 7, 12
  • 84. Modules SummaryWednesday, November 7, 12
  • 85. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experienceWednesday, November 7, 12
  • 86. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experience • Design enables smooth transition pathWednesday, November 7, 12
  • 87. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experience • Design enables smooth transition path • Clang implementation underwayWednesday, November 7, 12

×