Your SlideShare is downloading. ×
Gregor modules
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Gregor modules

219
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
219
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Modules Doug Gregor, AppleWednesday, November 7, 12
  • 2. Roadmap • The fundamental brokenness of headers • A module system for the C family • Building better toolsWednesday, November 7, 12
  • 3. On the Fundamental Brokenness of HeadersWednesday, November 7, 12
  • 4. C Preprocessing Model #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 5. C Preprocessing Model #include <stdio.h> // stdio.h int main() { typedef struct { printf(“Hello, world!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 6. C Preprocessing Model #include <stdio.h> // stdio.h int main() { typedef struct { printf(“Hello, world!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 7. C Preprocessing Model // from stdio.h typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 8. Problems with the Model // from stdio.h typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); • Fragility int remove(const char*); // on and on... • Performance int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 9. Header Fragility #define FILE “MyFile.txt” #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 10. Header Fragility #define FILE “MyFile.txt” // stdio.h #include <stdio.h> typedef struct { int main() { ... printf(“Hello, world!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 11. Header Fragility #define FILE “MyFile.txt” // stdio.h #include <stdio.h> typedef struct { int main() { ... printf(“Hello, world!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on...Wednesday, November 7, 12
  • 12. Header Fragility // from stdio.h typedef struct { ... } “MyFile.txt”; int printf(const char*, ...); int fprintf(“MyFile.txt” *, const char*, ...); int remove(const char*); // on and on... int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 13. Conventional WorkaroundsWednesday, November 7, 12
  • 14. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROSWednesday, November 7, 12
  • 15. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_HWednesday, November 7, 12
  • 16. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_H • template<class _Tp> const _Tp& min(const _Tp &__a, const _Tp &__b);Wednesday, November 7, 12
  • 17. Conventional Workarounds • LLVM_WHY_PREFIX_UPPER_MACROS • LLVM_CLANG_INCLUDE_GUARD_H • template<class _Tp> const _Tp& min(const _Tp &__a, const _Tp &__b); • #include <windows.h> #undef min // because #define NOMINMAX #undef max // doesn’t always workWednesday, November 7, 12
  • 18. How Big is a Source File?Wednesday, November 7, 12
  • 19. How Big is a Source File? #include <stdio.h> int main() { printf(“Hello, world!n”); }Wednesday, November 7, 12
  • 20. How Big is a Source File? #include <stdio.h> int main() { printf(“Hello, world!n”); } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 21. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 22. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 23. How Big is a Source File? #include <stdio.h> #include <iostream> int main() { int main() { printf(“Hello, world!n”); std::cout << “Hello, world!” } << std::endl; } C Hello C++ Hello SemaOverload Source 64 81 469,939 Headers 11,072 1,161,033 3,824,521Wednesday, November 7, 12
  • 24. Inherently Non-Scalable • M headers with N source files • M x N build cost • C++ templates exacerbate the problem • Precompiled headers are a terrible solutionWednesday, November 7, 12
  • 25. A Module System for the C FamilyWednesday, November 7, 12
  • 26. What Is a Module? • A module is a package describing a library • Interface of the library (API) • Implementation of the libraryWednesday, November 7, 12
  • 27. Module Imports import std; int main() { printf(“Hello, World!n”); } • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 28. Module Imports // std module (includes stdio) import std; typedef struct { int main() { ... printf(“Hello, World!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 29. Module Imports // std module (includes stdio) import std; typedef struct { int main() { ... printf(“Hello, World!n”); } FILE; } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ makes the API of the named module availableWednesday, November 7, 12
  • 30. Import Resilience • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 31. Import Resilience #define FILE “MyFile.txt” import std; int main() { printf(“Hello, World!n”); } • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 32. Import Resilience // std module (includes stdio) #define FILE “MyFile.txt” import std; typedef struct { ... int main() { } FILE; printf(“Hello, World!n”); } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 33. Import Resilience // std module (includes stdio) #define FILE “MyFile.txt” import std; typedef struct { ... int main() { } FILE; printf(“Hello, World!n”); } int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... • ‘import’ ignores preprocessor state within the source fileWednesday, November 7, 12
  • 34. Selective Import // std module // stdio submodule typedef struct { ... } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 35. Selective Import // std module import std.stdio; // stdio submodule int main() { typedef struct { printf(“Hello, World!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 36. Selective Import // std module import std.stdio; // stdio submodule int main() { typedef struct { printf(“Hello, World!n”); ... } } FILE; int printf(const char*, ...); int fprintf(FILE *, const char*, ...); int remove(const char*); // on and on... // stdlib submodule void abort(void); int rand(void); // on and on...Wednesday, November 7, 12
  • 37. What Does import Import? • Functions, variables, types, templates, macros, etc. • Only public API -- everything else can be hidden. • No special namespace mechanism.Wednesday, November 7, 12
  • 38. Writing a Module Futuristic VersionWednesday, November 7, 12
  • 39. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; int printf(const char*, ...) { // ... } int fprintf(FILE *, const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 40. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { // ... } int fprintf(FILE *, const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 41. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { } // ... • Public access int fprintf(FILE *, describes API const char*, ...) { // ... } int remove(const char*) { // ... }Wednesday, November 7, 12
  • 42. Writing a Module // stdio.c export std.stdio: public: typedef struct { ... } FILE; • Specify module name in source file int printf(const char*, ...) { } // ... • Public access int fprintf(FILE *, describes API const char*, ...) { } // ... • No headers! int remove(const char*) { // ... }Wednesday, November 7, 12
  • 43. Problems With This Future • Transitioning existing header-based libraries • Interoperability with compilers that don’t implement modules • Requires tools that understand modulesWednesday, November 7, 12
  • 44. Writing a Module Transitional VersionWednesday, November 7, 12
  • 45. Embracing Headers • Build modules directly from the headers • Headers remain “the truth” • Good for interoperability • Doesn’t change the programmer modelWednesday, November 7, 12
  • 46. Module Maps // /usr/include/module.map module std { module stdio { header “stdio.h” } module stdlib { header “stdlib.h” } module math { header “math.h” } } • module defines a named (sub)module • header includes the contents of the named header in the current (sub)moduleWednesday, November 7, 12
  • 47. Umbrella Headers // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } } • An umbrella header includes all of the headers in its directoryWednesday, November 7, 12
  • 48. Umbrella Headers // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } } • An umbrella header includes all of the headers in its directory • Wildcard submodules (module *) create a submodule for each included header • AST/Decl.h -> ClangAST.Decl • AST/Expr.h -> ClangAST.ExprWednesday, November 7, 12
  • 49. Module Map FeaturesWednesday, November 7, 12
  • 50. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * }Wednesday, November 7, 12
  • 51. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * } module _Builtin { module avx { • Submodule requirements } requires avx header “avxintrin.h” }Wednesday, November 7, 12
  • 52. Module Map Features module LLVMADT { • Umbrella directories } umbrella “llvm/ADT” module * { export * } module _Builtin { module avx { • Submodule requirements } requires avx header “avxintrin.h” } • Excluded headers module std { exclude header “assert.h” }Wednesday, November 7, 12
  • 53. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); }Wednesday, November 7, 12
  • 54. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named moduleWednesday, November 7, 12
  • 55. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module fileWednesday, November 7, 12
  • 56. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module file 3. Load the module file at the ‘import’ declarationWednesday, November 7, 12
  • 57. Compilation Model import std.stdio; int main() { printf(“Hello, World!n”); } 1. Find a module map for the named module 2. Spawn a separate instance of the compiler: • Parse the headers in the module map • Write the module file 3. Load the module file at the ‘import’ declaration 4. Cache module file for later re-useWednesday, November 7, 12
  • 58. Adopting Modules: Libraries • Eliminate non-modular behavior: • Multiply-defined structs, functions, macros, must be consolidated • Headers should import what they depend on • Write module maps covering the libraryWednesday, November 7, 12
  • 59. Adopting Modules: Users #include <stdio.h> int main() { printf(“Hello, World!n”); }Wednesday, November 7, 12
  • 60. Adopting Modules: Users #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • “Simply” rewrite each #include as an importWednesday, November 7, 12
  • 61. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } }Wednesday, November 7, 12
  • 62. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d headerWednesday, November 7, 12
  • 63. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d header • Optional Fix-Its, tooling to finalize the rewriteWednesday, November 7, 12
  • 64. Translating #include to import #include <stdio.h> import std.stdio; int main() { int main() { printf(“Hello, World!n”); printf(“Hello, World!n”); } } • Use module maps to determine (sub)module corresponding to an #include’d header • Optional Fix-Its, tooling to finalize the rewrite • Enabling modules is transparent to the userWednesday, November 7, 12
  • 65. Building Better Tools (For the Future)Wednesday, November 7, 12
  • 66. Compilation Performance • Algorithmic improvement for parsing time • Module headers parsed once, cached • MxN M+N • Benefits for all source-based toolsWednesday, November 7, 12
  • 67. Automatic Linking // clang/include/clang/module.map module ClangAST { umbrella header “AST/AST.h” module * { } link “-lclangAST” } • Drastically simplifies use of a libraryWednesday, November 7, 12
  • 68. Automatic Import int main() { std::vector<int> v; } • Forgotten #include terrible diagnosticWednesday, November 7, 12
  • 69. Automatic Import int main() { std::vector<int> v; } • Forgotten #include terrible diagnostic • vector.cpp:2:6: error: ‘vector‘ template is not available std::vector<int> v; ^ note: import ‘std.vector‘ to use ‘std::vector’Wednesday, November 7, 12
  • 70. Debugging FlowWednesday, November 7, 12
  • 71. Debugging Flow <iostream> foo.cppWednesday, November 7, 12
  • 72. Debugging Flow <iostream> foo.cpp Clang ASTsWednesday, November 7, 12
  • 73. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 74. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB DWARF object codeWednesday, November 7, 12
  • 75. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object codeWednesday, November 7, 12
  • 76. Debugging Flow foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code • Round-trip through DWARF is lossy • Only ‘used’ types, functions available • Inline functions, template definitions lostWednesday, November 7, 12
  • 77. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object codeWednesday, November 7, 12
  • 78. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code bar.o iostream DWARF <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 79. Redundant Debug Info foo.o iostream DWARF <iostream> foo.cpp Clang ASTs foo LLDB Clang ASTs DWARF object code bar.o iostream DWARF <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 80. Debugging with Modules foo.o <iostream> foo.cpp Clang ASTs foo DWARF object code LLDB Clang ASTs bar.o <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 81. Debugging with Modules std.iostream module foo.o <iostream> foo.cpp Clang ASTs foo DWARF object code LLDB Clang ASTs bar.o <iostream> bar.cpp Clang ASTs foo DWARF object codeWednesday, November 7, 12
  • 82. Debugging with Modules • Improved build performance • Compiler emits less DWARF • Linker de-duplicates less DWARFWednesday, November 7, 12
  • 83. Debugging with Modules • Improved build performance • Compiler emits less DWARF • Linker de-duplicates less DWARF • Improved debugging experience • Perfect AST fidelity in debugger • Debugger doesn’t need to search DWARFWednesday, November 7, 12
  • 84. Modules SummaryWednesday, November 7, 12
  • 85. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experienceWednesday, November 7, 12
  • 86. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experience • Design enables smooth transition pathWednesday, November 7, 12
  • 87. Modules Summary • Modules are a huge potential win for C(++) • Compile/build time improvements • Fix various preprocessor problems • Far better tool experience • Design enables smooth transition path • Clang implementation underwayWednesday, November 7, 12