C++ is known for things such as performance, expressiveness, the lack of a standard build system and package management, complexity and long compile times. The inability to iterate quickly is one of the biggest killers of productivity. This talk is aimed at anyone interested in improving the last of these points - it will provide insights into why compilation (and linking) take so long for C++ and will then provide an exhaustive list of techniques and tools to mitigate the problem, such as: - tooling and infrastructure - hardware, build systems, caching, distributed builds, diagnostics of bottlenecks, code hygiene - techniques - unity builds, precompiled headers, linking (static vs shared libraries) - source code modification - the PIMPL idiom, better template use, annotations - modules - what they are, when they are coming to C++ and what becomes obsolete because of them
The presentation about the fundamentals of DevOps workflow and CI/CD practices I presented at Centroida (https://centroida.ai/) as a back-end development intern.
Delivered as plenary at USENIX LISA 2013. video here: https://www.youtube.com/watch?v=nZfNehCzGdw and https://www.usenix.org/conference/lisa13/technical-sessions/plenary/gregg . "How did we ever analyze performance before Flame Graphs?" This new visualization invented by Brendan can help you quickly understand application and kernel performance, especially CPU usage, where stacks (call graphs) can be sampled and then visualized as an interactive flame graph. Flame Graphs are now used for a growing variety of targets: for applications and kernels on Linux, SmartOS, Mac OS X, and Windows; for languages including C, C++, node.js, ruby, and Lua; and in WebKit Web Inspector. This talk will explain them and provide use cases and new visualizations for other event types, including I/O, memory usage, and latency.
BPF of Berkeley Packet Filter mechanism was first introduced in linux in 1997 in version 2.1.75. It has seen a number of extensions of the years. Recently in versions 3.15 - 3.19 it received a major overhaul which drastically expanded it's applicability. This talk will cover how the instruction set looks today and why. It's architecture, capabilities, interface, just-in-time compilers. We will also talk about how it's being used in different areas of the kernel like tracing and networking and future plans.
Talk for SCaLE13x. Video: https://www.youtube.com/watch?v=_Ik8oiQvWgo . Profiling can show what your Linux kernel and appliacations are doing in detail, across all software stack layers. This talk shows how we are using Linux perf_events (aka "perf") and flame graphs at Netflix to understand CPU usage in detail, to optimize our cloud usage, solve performance issues, and identify regressions. This will be more than just an intro: profiling difficult targets, including Java and Node.js, will be covered, which includes ways to resolve JITed symbols and broken stacks. Included are the easy examples, the hard, and the cutting edge.
The presentation about the fundamentals of DevOps workflow and CI/CD practices I presented at Centroida (https://centroida.ai/) as a back-end development intern.
Delivered as plenary at USENIX LISA 2013. video here: https://www.youtube.com/watch?v=nZfNehCzGdw and https://www.usenix.org/conference/lisa13/technical-sessions/plenary/gregg . "How did we ever analyze performance before Flame Graphs?" This new visualization invented by Brendan can help you quickly understand application and kernel performance, especially CPU usage, where stacks (call graphs) can be sampled and then visualized as an interactive flame graph. Flame Graphs are now used for a growing variety of targets: for applications and kernels on Linux, SmartOS, Mac OS X, and Windows; for languages including C, C++, node.js, ruby, and Lua; and in WebKit Web Inspector. This talk will explain them and provide use cases and new visualizations for other event types, including I/O, memory usage, and latency.
BPF of Berkeley Packet Filter mechanism was first introduced in linux in 1997 in version 2.1.75. It has seen a number of extensions of the years. Recently in versions 3.15 - 3.19 it received a major overhaul which drastically expanded it's applicability. This talk will cover how the instruction set looks today and why. It's architecture, capabilities, interface, just-in-time compilers. We will also talk about how it's being used in different areas of the kernel like tracing and networking and future plans.
Talk for SCaLE13x. Video: https://www.youtube.com/watch?v=_Ik8oiQvWgo . Profiling can show what your Linux kernel and appliacations are doing in detail, across all software stack layers. This talk shows how we are using Linux perf_events (aka "perf") and flame graphs at Netflix to understand CPU usage in detail, to optimize our cloud usage, solve performance issues, and identify regressions. This will be more than just an intro: profiling difficult targets, including Java and Node.js, will be covered, which includes ways to resolve JITed symbols and broken stacks. Included are the easy examples, the hard, and the cutting edge.
Numerous technologies exist for profiling and tracing live Linux systems - from the traditional and straight forward gProf and strace to the more elaborate SystemTap, oProfile and the Linux Trace Toolkit. Very recently some new technologies, perf events and ftrace, have appeared that can already largely take the place of these traditional tools and have gained mainline acceptance in the Linux community - meaning that they will become more and more relevant in the future and are already being used to shed light on real world performance issues.
This presentation provides an overview of a number of the more noteworthy instrumentation tools available for Linux and the technologies that they build upon. Some examples of using perf events to analyse a running system to help track down real world performance problems are demonstrated.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
Recently I was asked to explain what dev-ops is at a large enterprise software vendor undergoing transformation.
In these slides, I present the concepts, tools and mindset that drive DevOPS.
eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
Velocity 2017 Performance analysis superpowers with Linux eBPFBrendan Gregg
Talk by for Velocity 2017 by Brendan Gregg: Performance analysis superpowers with Linux eBPF.
"Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will investigate this new technology, which sooner or later will be available to everyone who uses Linux. The talk will dive deep on these new tracing, observability, and debugging capabilities. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
Performance Wins with BPF: Getting StartedBrendan Gregg
Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Let's turn the table. Suppose your goal is to deliberately create buggy programs in C and C++ with serious security vulnerabilities that can be "easily" exploited. Then you need to know about things like stack smashing, shellcode, arc injection, return-oriented programming. You also need to know about annoying protection mechanisms such as address space layout randomization, stack canaries, data execution prevention, and more. These slides will teach you the basics of how to deliberately write insecure programs in C and C++.
A PDF version of the slides can be downloaded from my homepage: http://olvemaudal.com/talks
Here is a video recording of me presenting these slides at NDC 2014: http://vimeo.com/channels/ndc2014/97505677
Enjoy!
Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you have been using for years, are in fact, misleading, dangerous, or broken.
The speaker, Brendan Gregg, has given many talks on tools that work, including giving the Linux PerformanceTools talk originally at SCALE. This is an anti-version of that talk, to focus on broken tools and metrics instead of the working ones. Metrics can be misleading, and counters can be counter-intuitive! This talk will include advice for verifying new performance tools, understanding how they work, and using them successfully.
Video: https://www.youtube.com/watch?v=FJW8nGV4jxY and https://www.youtube.com/watch?v=zrr2nUln9Kk . Tutorial slides for O'Reilly Velocity SC 2015, by Brendan Gregg.
There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explains methodologies for using these tools, and provides a tour of four tool types: observability, benchmarking, tuning, and static tuning. Many tools will be discussed, including top, iostat, tcpdump, sar, perf_events, ftrace, SystemTap, sysdig, and others, as well observability frameworks in the Linux kernel: PMCs, tracepoints, kprobes, and uprobes.
This tutorial is updated and extended on an earlier talk that summarizes the Linux performance tool landscape. The value of this tutorial is not just learning that these tools exist and what they do, but hearing when and how they are used by a performance engineer to solve real world problems — important context that is typically not included in the standard documentation.
Marco Cavallini - Yocto Project, an automatic generator of embedded Linux dis...linuxlab_conf
The Yocto Project is an open source collaboration project that provides models, tools and methods to create custom Linux-based systems for embedded products that are independent from the adopted hardware architecture. The project was created in 2010 as a collaboration among several hardware manufacturers, open-source operating system providers and electronics companies to bring some order into the chaos of Linux Embedded development. Over the years, Yocto Project has established itself as the de-facto standard for the generation of embedded Linux systems, surpassing alternative products thanks to its characteristics.
The free tools that Yocto provides are powerful and easily generated (including emulation environments, debuggers, an application generator toolkit, etc.). The complete abstraction from the hardware of the development environment allows to optimize the investments made during the prototyping phase. The Yocto Project encourages the adoption of this technology by the open source community allowing users to focus on the characteristics and development of their product.
Let's trace Linux Lernel with KGDB @ COSCUP 2021Jian-Hong Pan
https://coscup.org/2021/en/session/39M73K
https://www.youtube.com/watch?v=L_Gyvdl_d_k
Engineers have plenty of debug tools for user space programs development, code tracing, debugging and analyzing. Except “printk”, do we have any other debug tools for Linux kernel development? The “KGDB” mentioned in Linux kernel document provides another possibility.
Will share how to experiment with the KGDB in a virtual machine. And, use GDB + OpenOCD + JTAG + Raspberry Pi in the real environment as the demo in this talk.
開發 user space 軟體時,工程師們有方便的 debug 工具進行查找、分析、除錯。但在 Linux kernel 的開發,除了 printk 外,還可以有哪些工具可以使用呢?從 Linux kernel document 可以看到 KGDB 相關的資訊,提供了在 kernel 除錯時的另一個可能性。
本次將分享,從建立最簡單環境的虛擬機機開始,到實際使用 GDB + OpenOCD + JTAG + Raspberry Pi 當作展示範例。
Kernel Recipes 2017: Using Linux perf at NetflixBrendan Gregg
Talk for Kernel Recipes 2017 by Brendan Gregg. "Linux perf is a crucial performance analysis tool at Netflix, and is used by a self-service GUI for generating CPU flame graphs and other reports. This sounds like an easy task, however, getting perf to work properly in VM guests running Java, Node.js, containers, and other software, has been at times a challenge. This talk summarizes Linux perf, how we use it at Netflix, the various gotchas we have encountered, and a summary of advanced features."
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all startedAnne Nicolas
Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls!
Steven Rostedt
Numerous technologies exist for profiling and tracing live Linux systems - from the traditional and straight forward gProf and strace to the more elaborate SystemTap, oProfile and the Linux Trace Toolkit. Very recently some new technologies, perf events and ftrace, have appeared that can already largely take the place of these traditional tools and have gained mainline acceptance in the Linux community - meaning that they will become more and more relevant in the future and are already being used to shed light on real world performance issues.
This presentation provides an overview of a number of the more noteworthy instrumentation tools available for Linux and the technologies that they build upon. Some examples of using perf events to analyse a running system to help track down real world performance problems are demonstrated.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
Recently I was asked to explain what dev-ops is at a large enterprise software vendor undergoing transformation.
In these slides, I present the concepts, tools and mindset that drive DevOPS.
eBPF is an exciting new technology that is poised to transform Linux performance engineering. eBPF enables users to dynamically and programatically trace any kernel or user space code path, safely and efficiently. However, understanding eBPF is not so simple. The goal of this talk is to give audiences a fundamental understanding of eBPF, how it interconnects existing Linux tracing technologies, and provides a powerful aplatform to solve any Linux performance problem.
Velocity 2017 Performance analysis superpowers with Linux eBPFBrendan Gregg
Talk by for Velocity 2017 by Brendan Gregg: Performance analysis superpowers with Linux eBPF.
"Advanced performance observability and debugging have arrived built into the Linux 4.x series, thanks to enhancements to Berkeley Packet Filter (BPF, or eBPF) and the repurposing of its sandboxed virtual machine to provide programmatic capabilities to system tracing. Netflix has been investigating its use for new observability tools, monitoring, security uses, and more. This talk will investigate this new technology, which sooner or later will be available to everyone who uses Linux. The talk will dive deep on these new tracing, observability, and debugging capabilities. Whether you’re doing analysis over an ssh session, or via a monitoring GUI, BPF can be used to provide an efficient, custom, and deep level of detail into system and application performance.
This talk will also demonstrate the new open source tools that have been developed, which make use of kernel- and user-level dynamic tracing (kprobes and uprobes), and kernel- and user-level static tracing (tracepoints). These tools provide new insights for file system and storage performance, CPU scheduler performance, TCP performance, and a whole lot more. This is a major turning point for Linux systems engineering, as custom advanced performance instrumentation can be used safely in production environments, powering a new generation of tools and visualizations."
Performance Wins with BPF: Getting StartedBrendan Gregg
Keynote by Brendan Gregg for the eBPF summit, 2020. How to get started finding performance wins using the BPF (eBPF) technology. This short talk covers the quickest and easiest way to find performance wins using BPF observability tools on Linux.
Let's turn the table. Suppose your goal is to deliberately create buggy programs in C and C++ with serious security vulnerabilities that can be "easily" exploited. Then you need to know about things like stack smashing, shellcode, arc injection, return-oriented programming. You also need to know about annoying protection mechanisms such as address space layout randomization, stack canaries, data execution prevention, and more. These slides will teach you the basics of how to deliberately write insecure programs in C and C++.
A PDF version of the slides can be downloaded from my homepage: http://olvemaudal.com/talks
Here is a video recording of me presenting these slides at NDC 2014: http://vimeo.com/channels/ndc2014/97505677
Enjoy!
Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you have been using for years, are in fact, misleading, dangerous, or broken.
The speaker, Brendan Gregg, has given many talks on tools that work, including giving the Linux PerformanceTools talk originally at SCALE. This is an anti-version of that talk, to focus on broken tools and metrics instead of the working ones. Metrics can be misleading, and counters can be counter-intuitive! This talk will include advice for verifying new performance tools, understanding how they work, and using them successfully.
Video: https://www.youtube.com/watch?v=FJW8nGV4jxY and https://www.youtube.com/watch?v=zrr2nUln9Kk . Tutorial slides for O'Reilly Velocity SC 2015, by Brendan Gregg.
There are many performance tools nowadays for Linux, but how do they all fit together, and when do we use them? This tutorial explains methodologies for using these tools, and provides a tour of four tool types: observability, benchmarking, tuning, and static tuning. Many tools will be discussed, including top, iostat, tcpdump, sar, perf_events, ftrace, SystemTap, sysdig, and others, as well observability frameworks in the Linux kernel: PMCs, tracepoints, kprobes, and uprobes.
This tutorial is updated and extended on an earlier talk that summarizes the Linux performance tool landscape. The value of this tutorial is not just learning that these tools exist and what they do, but hearing when and how they are used by a performance engineer to solve real world problems — important context that is typically not included in the standard documentation.
Marco Cavallini - Yocto Project, an automatic generator of embedded Linux dis...linuxlab_conf
The Yocto Project is an open source collaboration project that provides models, tools and methods to create custom Linux-based systems for embedded products that are independent from the adopted hardware architecture. The project was created in 2010 as a collaboration among several hardware manufacturers, open-source operating system providers and electronics companies to bring some order into the chaos of Linux Embedded development. Over the years, Yocto Project has established itself as the de-facto standard for the generation of embedded Linux systems, surpassing alternative products thanks to its characteristics.
The free tools that Yocto provides are powerful and easily generated (including emulation environments, debuggers, an application generator toolkit, etc.). The complete abstraction from the hardware of the development environment allows to optimize the investments made during the prototyping phase. The Yocto Project encourages the adoption of this technology by the open source community allowing users to focus on the characteristics and development of their product.
Let's trace Linux Lernel with KGDB @ COSCUP 2021Jian-Hong Pan
https://coscup.org/2021/en/session/39M73K
https://www.youtube.com/watch?v=L_Gyvdl_d_k
Engineers have plenty of debug tools for user space programs development, code tracing, debugging and analyzing. Except “printk”, do we have any other debug tools for Linux kernel development? The “KGDB” mentioned in Linux kernel document provides another possibility.
Will share how to experiment with the KGDB in a virtual machine. And, use GDB + OpenOCD + JTAG + Raspberry Pi in the real environment as the demo in this talk.
開發 user space 軟體時,工程師們有方便的 debug 工具進行查找、分析、除錯。但在 Linux kernel 的開發,除了 printk 外,還可以有哪些工具可以使用呢?從 Linux kernel document 可以看到 KGDB 相關的資訊,提供了在 kernel 除錯時的另一個可能性。
本次將分享,從建立最簡單環境的虛擬機機開始,到實際使用 GDB + OpenOCD + JTAG + Raspberry Pi 當作展示範例。
Kernel Recipes 2017: Using Linux perf at NetflixBrendan Gregg
Talk for Kernel Recipes 2017 by Brendan Gregg. "Linux perf is a crucial performance analysis tool at Netflix, and is used by a self-service GUI for generating CPU flame graphs and other reports. This sounds like an easy task, however, getting perf to work properly in VM guests running Java, Node.js, containers, and other software, has been at times a challenge. This talk summarizes Linux perf, how we use it at Netflix, the various gotchas we have encountered, and a summary of advanced features."
Kernel Recipes 2019 - ftrace: Where modifying a running kernel all startedAnne Nicolas
Ftrace’s most powerful feature is the function tracer (and function graph tracer which is built from it). But to have this enabled on production systems, it had to have its overhead be negligible when disabled. As the function tracer uses gcc’s profiling mechanism, which adds a call to “mcount” (or more recently fentry, don’t worry if you don’t know what this is, it will all be explained) at the start of almost all functions, it had to do something about the overhead that causes. The solution was to turn those calls into “nops” (an instruction that the CPU simply ignores). But this was no easy feat. It took a lot to come up with a solution (and also turning a few network cards into bricks). This talk will explain the history of how ftrace came about implementing the function tracer, and brought with it the possibility of static branches and soon static calls!
Steven Rostedt
C++ was developed by Bjarne Stroustrup, as an extension to the C language. cp...bhargavi804095
C++ is a cross-platform language that can be used to create high-performance applications.
C++ was developed by Bjarne Stroustrup, as an extension to the C language.
C++ gives programmers a high level of control over system resources and memory.
TensorFlow meetup: Keras - Pytorch - TensorFlow.jsStijn Decubber
Slides from the TensorFlow meetup hosted on October 9th at the ML6 offices in Ghent. Join our Meetup group for updates and future sessions: https://www.meetup.com/TensorFlow-Belgium/
C++ CoreHard Autumn 2018. Создание пакетов для открытых библиотек через conan...corehard_by
Использование сторонних библиотек в языке C++ никогда не было простым - необходимо было правильно собрать их, имея дело с различными системами сборки, но с появлением пакетного менеджера conan.io процесс стал намного проще, так что теперь осталось только сделать пакеты для нужным библиотек, и в этом поможет команда bincrafter-ов.
C++ CoreHard Autumn 2018. Actors vs CSP vs Tasks vs ... - Евгений Охотниковcorehard_by
На предыдущих конференциях C++ CoreHard автор доклада рассказывал про Модель Акторов и опыт ее использования в C++. Но Модель Акторов -- это далеко не единственный способ борьбы со сложностью при работе с многопоточностью. Давайте попробуем поговорить о том, что еще можно применить и как это может выглядеть в C++.
C++ CoreHard Autumn 2018. Знай свое "железо": иерархия памяти - Александр Титовcorehard_by
Если вам важна скорость работы ваших программ, то вы обязаны понимать, как работает ваше "железо". Современный процессор -- это сложное устройство, многие механизмы которого могут неочевидным образом влиять на скорость исполнения вашего кода. В докладе дается обзорное представление основных структур современного процессора и подробно рассматривается работа иерархии памяти. Будут освещены следующие темы: организация кэш-памяти, принцип локальности, предподкачка данных, нежелательное общее владение данными, а также программные техники для эффективной работы с памятью.
C++ CoreHard Autumn 2018. Информационная безопасность и разработка ПО - Евген...corehard_by
Информационная безопасность все больше из отдельной сферы плавно перетекает в разработку ПО. А значит «обычным» программистам приходится понимать те требования и терминологию, которые специалисты по безопасности уже давно знают и используют. CWE, CERT, MISRA, SAST– для «обычных» программистов это непонятные аббревиатуры. Поэтому в обзорном докладе мы попробуем рассказать простым языком об этих понятиях так, чтобы все разработчики начали уверенно ориентироваться в этой теме.
C++ CoreHard Autumn 2018. Заглядываем под капот «Поясов по C++» - Илья Шишковcorehard_by
Вот уже более двух лет мы создаём онлайн-специализацию по С++ на платформе Coursera. Её цель — обучить языку C++ с нуля до уровня, достаточного для решения практических задач, с которыми приходилось сталкиваться авторам в своей практике. В своём докладе я расскажу, как мы создаём наши онлайн-курсы, и уделю особое внимание техническим проблемам, которые нам пришлось решить в процессе создания автоматической системы проверки программ студентов.
C++ CoreHard Autumn 2018. Ускорение сборки C++ проектов, способы и последстви...corehard_by
В докладе обсуждаются способы улучшения времени сборки C++ проектов, опыт полученный в ходе ускорения сборки клиента и тулов World Of Tanks. Также описывается эффект, который они оказывают на организацию кодобазы (как позитивный, так и негативный) и затраты, которые необходимы для поддержки этих решений, т.к. не все они бесплатны. Методики, описываемые в докладе: ускорение линковки (Incremental Linking, Fastlink), ускорение компиляции(Include what you use, использование precompiled headers).
C++ CoreHard Autumn 2018. Метаклассы: воплощаем мечты в реальность - Сергей С...corehard_by
Доклад посвящён вопросам реализации пропозала Герба Саттера PR0707 (метаклассы в С++) за пределами компилятор - в виде отдельной утилиты. Будет продемонстрированы варианты использования метаклассов в реальной жизни, затронуты вопросы их реализации на базе Clang Frontend, а также возможные перспективы развития технологии и методики.
C++ CoreHard Autumn 2018. Что не умеет оптимизировать компилятор - Александр ...corehard_by
Все мы знаем, что компиляторы в настоящее время достаточно умные. И нам как программистам зачастую не нужно думать о каких-то незначительных оптимизациях - мы полагаемся на оптимизации компилятора. Что ж, настало время выяснить, действительно ли настолько компиляторы умны и узнать, в каких местах программист всё же (может быть) умнее.
C++ CoreHard Autumn 2018. Кодогенерация C++ кроссплатформенно. Продолжение - ...corehard_by
В докладе будет рассмотрена генерация кода при компиляции различных языковых конструкций, как простых, так и сложных, на различных платформах, как общераспространённых x86/x64, так и тех, которым уделяется меньше внимания: ARM, AVR. Также будут встречаться примеры для совсем экзотических процессоров вроде PowerPC и даже MicroBlaze. Основной упор будет делаться не на обработку данных, а именно на сопоставление различных конструкций кода с инструкциями целевых платформ.
C++ CoreHard Autumn 2018. Concurrency and Parallelism in C++17 and C++20/23 -...corehard_by
What do threads, atomic variables, mutexes, and conditional variables have in common? They are the basic building blocks of any concurrent application in C++, which are even for the experienced C++ programmers a big challenge. This massively changed with C++17 and change even more with C++20/23. What did we get with C++17, what can we hope for with C++20/23? With C++17, most of the standard template library algorithms are available in sequential, parallel, and vectorised variants. With the upcoming standards, we can look forward to executors, transactional memory, significantly improved futures and coroutines. To make it short. These are just the highlights from the concurrent and parallel perspective. Thus there is the hope that in the future C++ abstractions such as executors, transactional memory, futures and coroutines are used and that threads, atomic variables, mutexes and condition variables are just implementation details.
C++ CoreHard Autumn 2018. Обработка списков на C++ в функциональном стиле - В...corehard_by
Язык C++, претерпев долгую эволюцию, обрёл ряд черт, характерных для функциональной парадигмы: функции стали полноправными объектами, над которыми могут выполняться операции, а аппарат шаблонов позволяет проводить вычисления на типах на этапе компиляции. Математический фундамент этих двух главных аспектов составляют, соответственно, ламбда-исчисление и теория категорий. Расширение языка этими средствами способствовало реализации на языке C++ ряда инструментов, известных из функционального программирования. Некоторые из этих реализаций вошли в стандартную библиотеку (std::function, std::bind), другие - в сторонние библиотеки, в том числе в коллекцию библиотек Boost (functional, hana). Важную роль в арсенале функционального программирования играют операции свёртки и развёртки, которые очевиднее всего определяются для списков, но также естественным образом обобщаются на другие индуктивные и коиндуктивные структуры данных. Например, суммирование списка чисел можно представить себе как свёртку списка по операции сложения, а построение списка простых множителей заданного целого числа - как развёртку. Обобщения свёртки и развёртки известны как анаморфизмы и катаморфизмы. Также в функциональном программировании находит применение понятие гиломорфизма - композиция развёртки некоторого объекта в коллекцию с последующей свёрткой её в новый объект. В докладе продемонстрировано, что свёртки, развёртки и их композиции допускают довольно простую реализацию на языке C++.
C++ Corehard Autumn 2018. Обучаем на Python, применяем на C++ - Павел Филоновcorehard_by
Доклад посвящен часто используемому шаблону в моих проектах по анализу данных, когда обучение и настройка моделей происходят с использованием python, а вот их запуск в промышленное использование на языке C++. Предлагается рассмотреть несколько учебных примеров реализации такого подхода, от простой линейной регрессии до обработки изображений с помощью нейронных сетей.
C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukićcorehard_by
This talk will be about the design and implementation of a reactive programming model inspired by ranges that allows easy implementation of asynchronous and distributed software systems by writing code that looks like a sequence of ordinary range transformations like filter, transform, etc. This programming model will be demonstrated along with the implementation of a simple asynchronous web service where the whole system logic is defined as a chain of range transformations.
C++ CoreHard Autumn 2018. Debug C++ Without Running - Anastasia Kazakovacorehard_by
Macros, templates, compile-time evaluation and code generation, reflection and metaclasses – C++ tends to hide the final code passed to the compiler under the tons of various names and aliases. Add here the preprocessor that shadows the actual running curve of your program with dozens of alternatives mixed in a non-trivial way. While this allows to avoid boilerplate code and reduce copy-paste and other errors, such an approach demands better tooling support to make the debugging process easier. To find an error in such a code, one has to continuously read-fix-run it and compare the results with some etalon, or to debug in order to find actual substitutions. But should you really wait until your code is run or even compiled to debug it? Or how to deal with the situations when the code can’t be run on the local machine? A text editor with code completion won’t help, while a smart IDE that “gets” your code can do a better job. In this talk we’ll see interesting approaches to solving cases like macro and typedef ‘debug’, understanding types when auto/decltype hide them, dealing with different code branches depending on the preprocessor’s pass-through, and other ideas. Some suggestions are already implemented as ready-to-use features in CLion and ReSharper C++, tools for C++ from JetBrains (that means I can show it in action), others are planned for the future. The aim of this talk is to share the workflows supported by the tools that can help C++ developers create better modern C++ code.
C++ CoreHard Autumn 2018. Полезный constexpr - Антон Полухинcorehard_by
В C++11 добавили новое ключевое слово - constexpr. Выглядит оно весьма невзрачно, да и на первый взгляд кажется, что смысла в нём маловато... Для чего же оно нужно, какие у него есть тайные супер способности и какую роль оно сыграет в дальнейшем развитии языка C++ - обо всём об этом мы и поговорим.
C++ CoreHard Autumn 2018. Text Formatting For a Future Range-Based Standard L...corehard_by
Text formatting has been a favorite problem of C++ library authors for a long time. The standard C++ iostreams have been criticized for being difficult to use due to their statefulness and slow due to runtime polymorphism. Despite its age, printf is still popular because of simplicity and speed. The Boost library offers two more alternatives, Boost.Format and Boost.LexicalCast. And finally, the P0645 standard proposal sponsored by Facebook is currently finding its way through the C++ committee. All these approaches are still firmly based on standard containers and iterators. But the Standard Library is changing radically with the advent of ranges, range adaptors and functional style programming in C++. Generating optimized code with metaprogramming is becoming standard fare. In this talk, I want to convince you that the combination of ranges with a bit of metaprogramming makes for a very elegant solution to the text formatting problem. We introduce a form of ranges with internal iteration, which are generating their elements one by one rather than exposing external iterators. We can use these generator ranges to represent the values to be formatted, conceptually turning them into lazily evaluated strings. These can be used just like regular strings are used today: in function returns; as standard algorithm input; embedded into other, equally lazily evaluated strings; and so on, before they are finally expanded for display. By choosing the right interfaces, we can optimize this expansion at compile-time, making it no less pretty to write, but more efficient to expand than any text formatting approaches that rely on format strings that must be parsed at runtime. I believe that this approach is the natural extension of a range-based future standard library to text formatting.
Исключительная модель памяти. Алексей Ткаченко ➠ CoreHard Autumn 2019corehard_by
Память в компьютере - это не только гигабайты оперативной памяти в слоте, но и занятная абстракция. В докладе мы рассмотрим, как можно эту абстракцию использовать необычным образом для моделирования других абстракций - регистровых файлов периферийных устройств. Доклад будет полезен не только embedded-разработчикам, но и, возможно, заставит переосмыслить свой взгляд на память.
Как помочь и как помешать компилятору. Андрей Олейников ➠ CoreHard Autumn 2019corehard_by
Как правило, можно положиться на то, что компилятор оптимизирует результирующий бинарный файл так, чтобы она работала максимально быстро. Но компилятор не знает на каких данных и на каком железе программа будет запущена. Плюс хотелось бы, чтобы компиляция занимала приемлемое время. Из-за этого результат может оказаться субоптимальным. Предлагаю на примерах для LLVM посмотреть как можно подсказать компилятору как оптимизировать программу и сделать результат лучше или хуже.
Мы напишем простейший веб-сервис из клиента и сервера на C++. На этом C++ часть закончится, и пойдет настройка окружения и инфраструктуры. Мы обеспечим детерминируемость сборки и прогона тестов. Облегчим последующее обновление зависимых библиотек. Автоматизируем статические проверки, верификацию кода, прогон тестов. Обеспечим доступность сервиса, настроим инфраструктуру, сбалансируем нагрузку, добавим автоматическое и ручное масштабирование. И под конец мы настроим continious delivery таким образом, что код будет на продакшене через 5 минут после реквеста, при этом даже невалидные изменения и ошибки программиста не смогут повлиять на его работу.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Securing your Kubernetes cluster_ a step-by-step guide to success !
The Hitchhiker's Guide to Faster Builds. Viktor Kirilov. CoreHard Spring 2019
1. The Hitchhiker's Guide toThe Hitchhiker's Guide to
FASTER BUILDSFASTER BUILDS
by Viktor Kirilov
1
2. Me, myself and IMe, myself and I
my name is Viktor Kirilov - from Bulgaria
7+ years of professional C++ in the games / VFX industries
worked only on open source for 3 years (2016-2019)
creator of - the fastest C++ testing frameworkdoctest
!!! not wasting time !!!
Passionate about
2
3. C++ is known for...C++ is known for...
▲ performance
▲ expressiveness
▲ being multi-paradigm
▼ complexity
▼ metaprogramming could be
much more powerful and simple
▼ no standard build system
and package management
▼ L0oo0oOoNG COMPILE TIMES
3
4. Major C++ issue - compile timesMajor C++ issue - compile times
4
5. This presentationThis presentation
Introduction: WHAT and WHY
precompiled headers
unity builds
hardware
build systems
caching
distributed builds
diagnostics
static vs dynamic linkinglinkers
PIMPL
forward declarations
includes
templates
binary bloat
annotations
modules
parallel builds
toolchains
tooling
5
symbol visibility
compilers
Slides: https://slides.com/onqtam/faster_builds
memes
HOW
6. RealityReality
some projects take (tens of) hours to compile - WebKit: a big offender
both minimal and full rebuilds suffer more and more in time
all projects start small - build times increase inevitably
Opera 32 bit on Windows
in 3 years 6 months:
15 minutes to 64+
+ incorporated PCH !!!
6
7. A unique problem to C++A unique problem to C++
Actually the folks at sympathize with us :DRust
7
8. Why reduce build timesWhy reduce build times
idle time is costly - lets do the math:
assuming a 80k $ annual salary
1 hour daily for waiting (out of 8 hours: 12.5%)
10k $ per developer annually
value to company > at least x3 the salary
100 developers ==> 3 million $ annually
the unquantifiable effects of long build times
discourage refactoring and experimentation
lead to mental context switches (expensive)
any time spent reducing build times is worthwhile !!! 8
9. The steps in building C++ codeThe steps in building C++ code
preprocessor: source file => translation unit
includes
macros, conditional compilation (ifdef)
compiler: translation unit => object file
parsing, tokens, AST
optimizations
code generation
linker: object files & libraries => executable
Absolutely fantastic explanation:
https://github.com/green7ea/cpp-compilation 9
10. Why C++ compilation takes so longWhy C++ compilation takes so long
backwards compatibility with C: a killer feature ==> adoption!!!
inherits the archaic textual include approach
originally for sharing just declarations & structs between TUs
language is complex (ADL, overloads, templates, etc.)
multiple phases of translation - each depending on the previous
not context independent
new standards complicate things further
complex zero-cost abstractions end up in headers
even the preprocessor can become a bottleneck
sub-optimal tools and build system setups
bad project structure - component and header dependencies
touch a header - rebuild everything
10
11. C vs C++ and different standardsC vs C++ and different standards
compiled as time increase
C 5 sec baseline
C++98 5.6 sec +12%
C++11 5.6 sec +12%
C++17 5.8 sec +14%
compiled as time increase
C++98 8.4 sec baseline
C++11 9.8 sec +17%
C++17 10.8 sec +29%
C codebase (impl)
https://github.com/vurtun/nuklear
C++ codebase (impl + tests) (ver 1.2.8)
https://github.com/onqtam/doctest
Environment: GCC 7.2 on Windows, only compilation: "-c", in Debug: "-O0"
optimizer doesn't care about language/standard: -O2 ==> same diff in seconds
newer standards might be even heavier depending on the code
more constexpr, metaclasses, concepts - further pressure on compilers
constexpr - more complex compiler, but faster compile time algorithms
11
13. Why includes suckWhy includes suck
textual - not symbolic
parsed results cannot be cached - macros...
implementation dependencies are leaked/dragged
privately used types by value need the definition
the same code gets compiled many times
the same headers end up used in many sources
M headers with N source files ==> M x N build cost
templates
re-instantiated for the same types
require more code to be in headers
inline functions in headers (this includes templates)
more work for the compiler/linker 13
17. GCC/Clang warning: -Wunused, -Wunused-function
-ffunction/data-sections -Wl,--gc-sections,--print-gc-sections
--enable=unusedFunction
- DEADCODE/UNREACHABLE
by scitools
- static and dynamic
... also runtime coverage ( (& lcov for GUI), others)
cppcheck
Coverity
Understand
CppDepend
PVS Studio
OCLint
Parasoft C/C++ Test
Polyspace
oovcde dead code detection
others gcov
Unreachable (dead) codeUnreachable (dead) code
The C++ philosophy: "don't pay for what you don't use"
might also uncover bugs
& improve runtime!
17
18. compile in Debug instead of Release if feasible
-O0, -O1, -O2, -O3, -Os - different results!
unused sources/libraries
look for such after dead code removal
often a problem with makefiles and old cruft
MSVC warning LNK4221: This object file does not
define any previously undefined public symbols
compile each source file just once
scan the build logs for such instances
commonly used sources => static libraries
Other low hanging fruitOther low hanging fruit
18
19. strive for loose coupling
specific includes is better than #include "everything.h"
more granular rebuilds when touching headers
- the most popular (Clang-based)
(or the ) -
<< also has a test runner for doctest!
- plugin for Eclipse CDT (or the )
ProFactor IncludeManager -
MSVC: /showIncludes GCC: -H
https://include-what-you-use.org/
Doxygen + Graphviz
https://github.com/myint/cppclean
Header Hero fork article
https://github.com/tomtom-international/cpp-dependencies
ReSharper C++ Includes Analyzer
Includator Cevelop IDE
https://gitlab.com/esr/deheader
https://www.profactor.co.uk/
Finding unnecessary includesFinding unnecessary includes
find_program(iwyu_path NAMES include-what-you-use iwyu)
set_property(TARGET hello PROPERTY CXX_INCLUDE_WHAT_YOU_USE ${iwyu_path})
19
20. Declarations vs definitions - functionsDeclarations vs definitions - functions
// interface.h
int foo(); // fwd decl - definition is in a source file somewhere
inline int bar() { // needs to be inline if the body is in a header
return 42;
}
struct my_type {
int method { return 666; } // implicitly inline - can get inlined!
};
template <typename T> // a template - "usually" has to be in a header
T identity(T in) { return in; } // implicitly inline
move function definitions out of headers
builds will be faster
use link time optimizations for inlining (or unity builds!)
some optimizations for templates exist - in later slides 20
21. Declarations vs definitions - typesDeclarations vs definitions - types
// interface.h
#pragma once
struct foo; // fwd decl - don't #include "foo.h"!
#include "bar.h" // drag the definition of bar
struct my_type {
foo f_1(foo obj); // fwd decl used by value in signature - OK
foo* foo_ptr; // fwd decl used as pointer - OK
bar member; // requires the definition of bar
};
// interface.cpp
#include "interface.h"
#include "foo.h" // drag the definition of foo
foo my_type::f_1(foo obj) { return obj; } // needs the definition of f
21
22. used by value
including in class definitions (size for memory layout)
not in function declarations !!!
types are used by reference/pointer
pointer size is known by the compiler (4/8 bytes)
compiler doesn't need to know the size of types
types are used by value in function declarations
either as a return type or as an argument
this is less known !!!
function definition will still need the full type definition
call site also needs the full type definition
Declarations vs definitions - typesDeclarations vs definitions - types
definition of types is necessary only when they are:
forward declarations are enough if:
22
23. PIMPL - pointer to implementationPIMPL - pointer to implementation
AKA compilation firewall
holds a pointer to the implementation type
// widget.h
struct widget {
widget();
~widget(); // cannot be defaulted here
void foo();
private:
struct impl; // just a fwd decl
std::unique_ptr<impl> m_impl;
};
// widget.cpp
struct widget::impl {
void foo() {}
};
widget::widget() : m_impl(std::make_unique<widget::impl>()) {}
widget::~widget() = default; // here it is possible
void widget::foo() { m_impl->foo(); }
23
24. PIMPL - pros & consPIMPL - pros & cons
breaks the interface/implementation dependency
less dragged headers ==> better compile times!
implementation data members no longer affect the size
helps with preserving ABI when changing the implementation
lots of APIs use it (example: Autodesk Maya)
requires allocation, also space overhead for pointer
can use allocators
extra indirection of calls - link time optimizations can help
more code
PROS:
CONS:
on const propagation, moves, copies and other details:
http://en.cppreference.com/w/cpp/language/pimpl 24
26. Abstract interfaces & factoriesAbstract interfaces & factories
breaks the interface/implementation dependency
helps with preserving ABI (careful with the virtual interface)
requires allocation, also space overhead for pointer
extra indirection of calls (virtual)
more code
users work with (smart) pointers instead of values
link time optimizations - harder than for PIMPL
PROS:
CONS:
26
27. Precompiled headersPrecompiled headers
put headers that are used in many of the sources
put headers that don't change often
do regular audits to reflect the latest project needs
often disregarded as a hack by many - HUGE mistake IMHO
easy setup and benefits can be huge
// precompiled.h
#pragma once
// system, runtime, STL
#include <cstdio>
#include <vector>
#include <map>
// third-party
#include <dynamix/dynamix.hpp>
#include <doctest.h>
// rarely changing project-specific
#include "utils/singleton.h"
#include "utils/transform.h"
27
28. Precompiled headers - usagePrecompiled headers - usage
supported by mainstream compilers
MSVC: "/Yc" & "/Yu"
GCC/Clang: "-x c++-header"
only one PCH per translation unit :(
force-include for convenience
/FI (MSVC) or -include (GCC/Clang)
build system support
(native)
(native)
(native)
CMake - with modules
Meson
Buck
FASTBuild
cotire
cmake-precompiled-header 28
29. AKA: amalgamated, jumbo, SCU
include all original source files in one (or a few) unity source file(s)
Unity buildsUnity builds
// unity file
#include "core.cpp"
#include "widget.cpp"
#include "gui.cpp"
headers get parsed only once if included from multiple sources
less compiler invocations
less instantiation of the same templates
HIGHLY underrated
NOT because of reduced disk I/O - filesystems cache aggressively
less linker work
used at Ubisoft (14+ years), Unreal, WebKit
The main benefit is compile times:
29
30. up to 10+ times faster (depends on modularity)
best gains: short sources + lots of includes
faster (and not slower) LTO (WPO/LTCG)
ODR (One Definition Rule) violations get caught
this one is huge - still no reliable tools
.
.
enforces code hygiene
include guards in headers
no statics and anon namespaces in headers
less copy/paste of types & inline functions
Unity builds - prosUnity builds - pros
// a.cpp
struct Foo {
// implicit inline
int method() { return 42; }
};
// b.cpp
struct Foo {
// implicit inline
int method() { return 666; }
};
30
31. might slow down some workflows
minimal rebuilds
might interfere with parallel compilation
these ^^ can be solved with tuning/configuring
some C++ stops compiling as a unity
clashes of symbols in anonymous namespaces
overload ambiguities
using namespaces in sources
preprocessor
miscompilation - rare (but possible)
successful compilation with a wrong result
preprocessor, overloads (also non-explicit 1 arg constructors)
good tests can help!
Unity builds - consUnity builds - cons
31
32. (as a CMake module, there are others)
(build system)
experimental support for Unity builds
can produce multiple unity source files
can exclude files (if problematic, or for incremental builds)
(build system)
(build system)
(Visual Studio extension)
custom solution
cotire
FASTBuild
Visual Studio
Meson
waf
RudeBuild
Unity builds - how to maintainUnity builds - how to maintain
these:
others:
32
33. __declspec(noinline) / __attribute__((noinline))
and careful with __forceinline / __attribute__((always_inline))
leads to compile time savings
might even help with runtime (instruction cache bloat)
implicit culprit: destructors
especially for heavily repeated constructs
asserts in testing frameworks
example: - "The fastest feature-rich C++11 single-
header testing framework"
doctest
CppCon 2017: Viktor Kirilov "Mix Tests and Production Code
With Doctest - Implementation and usage"
Inlining annotationsInlining annotations
explicitly disable inlining:
33
34. linkers sometimes can fold identical functions - but more work
only if proven you aren't taking their addresses (required by the standard)
this way: less work for the compiler/linker
now it is syntactically easier to move function bodies out of the header
use link time optimizations for inlining (or unity builds!)
Templates - argument-independent codeTemplates - argument-independent code
template <typename T>
class my_vector {
int m_size;
// ...
public:
int size() const { return m_size; }
// ...
};
// my_vector<T>::size() is instantiated
// for types such as int, char, etc...
// even though it doesn't depend on T
class my_vector_base { // type independent
protected: // code is moved out
int m_size;
public:
int size() const { return m_size; }
};
template <typename T> // type dependent
class my_vector : public my_vector_base {
// ...
public:
// ...
};
34
35. Templates - C++11 extern templateTemplates - C++11 extern template
// interface.h
#include <vector>
template <typename T>
T identity(T in) { return in; }
// somewhere.cpp
#include "interface.h"
// do not instantiate these templates here
extern template int identity<int>(int);
extern template float identity<float>(float);
extern template class std::vector<int>;
// will not instantiate
auto g_int = identity(5);
auto g_float = identity(15.f);
std::vector<int> a;
// will instantiate for bool
auto g_unsigned = identity(true);
no instantiation/codegen where extern-ed but still has to be parsed...
precompiled headers or unity builds !!!!
some STL implementations do it for std::basic_string<char, ...>
https://arne-mertz.de/2019/02/extern-template-reduce-compile-times/
// somewhere_else.cpp
#include "interface.h"
// instantiation for float
template float identity<float>(float);
// instantiation for std::vector<int>
template class std::vector<int>;
// instantiation for identity<int>()
auto g_int_2 = identity(1);
35
36. the class/function has to be explicitly instantiated somewhere
for all types it is used with... or linker errors!
Templates - move functions out of headersTemplates - move functions out of headers
// interface.h
// declaration
template <typename T>
T identity(T in);
template <typename T>
struct answer {
// declaration
T calculate();
};
// interface.cpp
#include "interface.h"
// definition
template <typename T>
T identity(T in) { return in; }
// definition
template <typename T>
T answer<T>::calculate() {
return T(42);
}
// explicit instantiations for int/float
template int identity<int>(int);
template float identity<float>(float);
// explicit instantiation for int
template struct answer<int>;
// user.cpp
#include "interface.h"
auto g_int = identity(666);
auto g_float = identity(0.541f);
auto g_answer = answer<int>().calculate();
36
37. The "Rule of ":
1. looking up a memoized type
2. adding a parameter to an alias call
3. adding a parameter to a type
4. calling an alias
5. instantiating a type
6. instantiating a function template
7. SFINAE
Chiel
Templates - cost of operationsTemplates - cost of operations
A fascinating talk:
code::dive 2017 – Odin Holmes – The fastest
template metaprogramming in the West
37
38. use variadic templates sparingly
recursive templates are bad business
especially when not memoized
recursive template arguments are even worse
now that we have "constexpr if" we can use a lot less SFINAE
if constexpr might be preferrable even to tag dispatching
Templates - other notesTemplates - other notes
38
40. reducing compile times of -generated code: type tags to
circumvent SFINAE and use overload resolution
diagnostics
-ftime-report - breakdown of time spent in compiler phases
templight
- Alternative title: "Boost your TMP-Fu"
metaprogramming through composition
builds onto his previous talk
brigand, kvasir, metal, mp11 -
CppCon 2017: Eddie Elizondo “Optimizing compilation times with
Templates”
Thrift
C++Now 2018: Odin Holmes “Boost.TMP: Your DSL for
Metaprogramming”
http://metaben.ch/
Templates - other good talksTemplates - other good talks
40
41. -ftime-report - breakdown of different compiler phases
-ftime-trace (Clang) - - by Aras Pranckevičius
/Bt - frontend & backend (optimizer/codegen)
/time - for the linker
/d2cgsummary - Anomalistic Compile Times for functions
first advertised by Aras Pranckevičius -
/d1reportTime - includes, classes, functions - times -
/d1templateStats
thanks to these flags
Chrome tracing flamegraph
"Best-unknown-MSVC-flag"
blog post
surprising 1 line fix => 20% faster builds
Updating VS 2017 15.6 to 15.8 => 2x slower for template instantiations
Diagnostics - compilationDiagnostics - compilation
some functions trigger pathological cases (or bugs) in compilers/optimizers
Clang/GCC:
MSVC:
41
42. Bloaty McBloatface: a size profiler for ELF/Mach-O binaries
Sizer: Win32/64 executable size report utility
SymbolSort: measuring C++ code bloat in Windows binaries
twiggy: call graph analyzer (wasm only, no ELF/Mach-O/PE/COFF yet)
nm - Unix tool to inspect the symbol table of binaries
nm --print-size --size-sort --demangle --radix=d YOUR_BINARY
dumpbin: Microsoft COFF Binary File Dumper
common culprit: templates (because length of names matters!!!)
demangling: undname.exe / c++filt /
https://github.com/google/bloaty
https://github.com/aras-p/sizer
https://github.com/adrianstone55/SymbolSort
https://github.com/rustwasm/twiggy
https://demangler.com
Diagnostics - binary bloatDiagnostics - binary bloat
42
44. Build systemsBuild systems
"g++ src1.cpp src2.cpp ..." is not a build system
dependency tracking - for minimal and parallel rebuilds
help managing scale
includes, flags, link libs
help with refactoring/restructuring
help with tool integration
not all are created equal
HUGE topic (trade-offs, approaches) - but lets focus just on speed
most old build systems have high overhead (MSBuild, make)
suboptimal dependency and change detection/scanning
44
45. Parallel buildsParallel builds
Ninja, FASTBuild, IncrediBuild: best in this regard
make: -jN (and through CMake: "cmake --build . -- -jN")
MSBuild (build system behind Visual Studio):
/MP[processMax] - in CMake: "target_compile_options(target /MP)"
inability to parallelize compilation of dependent dlls
fixes this when integrated with Visual Studio
thread oversubscription (no overarching scheduler):
8 projects in parallel (8 cpp each + /MP) ==> 64 threads!
Ninja instead of MSBuild as the VS backend!
since recently supports
- Bruce Dawson
careful with unity builds and "long poles"
modern LTO/LTCG technology is parallelizable too
IncrediBuild
running Custom Build Tools in parallel
Make VC++ Compiles Fast Through Parallel Compilation
45
49. Sort-of winners (in terms of speed)Sort-of winners (in terms of speed)
FASTBuild
unity builds
precompiled headers
caching
distributed builds
best parallelization
generates project files
multiplatform
multi-compiler
tiny
might get a CMake backend
others...
haven't looked into these yet
buck << by Facebook
bazel << by Google
build2 << modules! :D
meson << hot contender
...
Ninja - less features but crazy
fast - intented to be generated
by a meta-build system
GN (GYP replacement)
can generate ONLY ninja
IndrediBuild
tup
49
50. big discussion overall - but here we care about build times
much much faster - handles only the exported interface
Static vs dynamic linkingStatic vs dynamic linking
prefer dynamic (.dll, .so, .dylib) over static (.lib, .a)
50
51. Unix: all symbols exported by default
opposite to Windows - which got the default right!
UNIX: compile with -fvisibility=hidden
improves link time
annotate the required parts of the API for exporting
circumvents the GOT (Global Offset Table) for calls => profit
load times also improved (extreme templates case: 45 times)
reduces the size of your DSO by 5-20%
for details:
clang-cl on windows: => 30% faster chrome build
alternative: -fvisibility-inlines-hidden
all inlined member functions are hidden
no source code modifications needed
careful with local statics in inline functions
can also explicitly annotate with default visibility
https://gcc.gnu.org/wiki/Visibility
/Zc:dllexportInlines-
Dynamic linking - symbol visibilityDynamic linking - symbol visibility
51
52. Multiplatform symbol export annotationsMultiplatform symbol export annotations
#if defined _WIN32 || defined __CYGWIN__
#define SYMBOL_EXPORT __declspec(dllexport)
#define SYMBOL_IMPORT __declspec(dllimport)
#define SYMBOL_HIDDEN
#else
#define SYMBOL_EXPORT __attribute__((visibility("default")))
#define SYMBOL_IMPORT
#define SYMBOL_HIDDEN __attribute__((visibility("hidden")))
#endif
#ifdef DLL_EXPORTS // on this depends if we are exporting or importing
#define MY_API SYMBOL_EXPORT
#else
#define MY_API SYMBOL_IMPORT
#endif
MY_API void foo();
class MY_API MyClass {
int c;
SYMBOL_HIDDEN void privateMethod(); // only for use within this DSO
public:
MyClass(int _c) : c(_c) { }
static void foo(int a);
};
52
53. LinkersLinkers
gold (unix)
comes with binutils starting from version 2.19
-fuse-ld=gold
not threaded by default
configure with "--enable-threads"
use: "--threads", "--thread-count COUNT", "--preread-archive-symbols"
lld even faster than gold - - multithreaded by default
Unix: Split DWARF - - building clang/llvm codebase
10%+ faster full builds and 40%+ faster incremental builds
CMake -
https://lld.llvm.org/
article
LINK_WHAT_YOU_USE
53
54. Linkers - LTO and other flagsLinkers - LTO and other flags
clang
ThinLTO:
MSVC - lots of improvements, also regarding LTCG
lots of improvements in the recent years
/debug:fastlink – generate partial PDB (new format)
/INCREMENTAL for debug
VS 2019 -
LTCG
/LTCG:incremental
also parallel - /cgthreads:8 instead of the default 4
CppCon 2017: Teresa Johnson “ThinLTO: Scalable and Incremental Link-
Time Optimization”
GCC 8: link time and interprocedural optimization
Visual Studio 2017 Throughput Improvements and Advice
Speeding up the Incremental Developer Build Scenario
from x1.4 to x1.75 faster link times for Chrome with VS 2019
huge speedups (~x2)
54
55. Compiler cachesCompiler caches
"compiler wrapper" -
maps a hash of:
preprocessed source (TU)
compilation flags
& compiler version
==> to an object file
First run (cold cache) is a bit slower
than without the wrapper
Consecutive >> FULL << rebuilds
are much faster (warm cache) 55
56. Compiler cachesCompiler caches
For Unix alike: (GCC, Clang), (GCC)
For Windows: (MSVC), (MSVC)
- straight out of the build system!
(GCC, Clang, MSVC)
ccache with cloud storage over AWS S3/GCS/Redis
(MSVC)
also distributed & with cloud storage
- a Clang fork - not a wrapper! -
in memory caching server for requested compilations
biggest gains on heavy templates -
recently - future not clear
x2~x5 times faster -
ccache cachecc1
clcache cclash
http://fastbuild.org/
https://github.com/mozilla/sccache
https://stashed.io/
zapcc mail thread
article
open-sourced
WebKit can compile more benchmarks 56
57. Compiler cachesCompiler caches
caches work best for full rebuilds - not minimal
Other links:
Faster C++ builds
Speeding up the Build of C and C++ Projects
ccache and CMake:
Using ccache with CMake
Speed up C++ compilation, part 2: compiler cache
find_program(CCACHE_PROGRAM ccache)
if(CCACHE_PROGRAM)
set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
endif()
project(SomeProject)
57
58. Distributed buildsDistributed builds
take advantage of unused
processing power of computers
prepend in front of compiler
just like caches: distcc gcc ...
caching + distributed builds = ❤
, and
watch out: network bottlenecks
include directories - sort em!
bandwidth
here here here
OMG again that build system!!! ==>
some article:
https://github.com/distcc/distcc
https://github.com/icecc/icecream
https://www.incredibuild.com/
http://fastbuild.org/
Speed up C++ compilation, part 3: distributed compilation
export CCACHE_PREFIX=DISTCC
58
60. HardwareHardware
more cores
make sure you have enough RAM && parallel builds setup
RAM disks (file system drive in your RAM instead of ssd/hdd)
better than SSDs
don't expect huge gains though...
mount $TEMP as a RAM disk (%TEMP% for windows)
build output to RAM disk (most important - because of the link step)
also put the compiler there!
touches on network I/O issues and many other topics
CppCon 2014: Matt Hargett "Common-sense acceleration of your MLOC build"
60
61. What if the fundament is badWhat if the fundament is bad
61
62. Physical designPhysical design
physical structure/design
often neglected
every project starts small... some end up big and it's too late
reasoning about components and modularity - IMPORTANT!!!
physically (and logically): files, dependencies, includes
benefits: fast builds, easy to reason, easy to refactor
Article:
John Lakos (Bloomberg)
book:
multi-part talk: "Advanced Levelization Techniques"
Controversial article:
Physical Structure and C++ - A First Look
"Large-scale C++ Software Design"
C++Now 2018: “C++ Modules & Large-Scale Development”
Physical Design of The Machinery 62
63. Dependency analysisDependency analysis
cmake --graphviz=[file]
visualize with
some (most) build systems have such a feature ( )
Most tools from the "Finding unnecessary includes" slide
by scitools
- Graphviz
- Graphviz
maybe even
IDE tools
options
Graphviz
bazel
Understand
CppDepend
cpp-dependencies
Doxygen
SourceTrail
Visual Studio Code Maps 63
66. Modules - historyModules - history
2004.11.05: first proposal: - Modules in C++
by Daveed Vandevoorde - working on EDG - the only compiler supporting
exported templates (and aiding in their deprecation)
dropped from C++11 after a few revisions ==>
2011: work on - by Douglas Gregor
2014.05.27: standardization continues with - A Module System for C++
by Gabriel Dos Reis, Mark Hall and Gor Nishanov
after a few revisions - didn't enter C++17 - left as a TS
latest work: will enter C++20
2017.06.16:
2018.01.29: - Working Draft, Extensions to C++ for Modules
2018.03.06: ,
2018.05.02:
2018.10.08: , ,
n1736
headed for a TR
first implementation in Clang
n4047
Business Requirements for Modules
n4720
Another take on Modules p0924r0
Modules, Macros, and Build Systems
Merging Modules p1218r0 p1242r0 66
67. Modules - motivationModules - motivation
compile times...!!!
preprocessor includes led to:
leaked symbols & implementation details
name clashes
making tooling hard
workarounds (include guards)
include order matters
more DRY (don't repeat yourself)
forward declarations
code duplication in headers/sources
67
69. Modules - how they workModules - how they work
BMI (Binary Module Interface) files
consumed by importers
produced by compiling a MIU (Module Interface Unit)
(maybe) contains serialized AST (abstract syntax tree)
with the exported parts of the parsed C++
unlike precompiled headers:
composability of multiple modules
explicit code annotations to define visible interfaces
69
70. Modules - obsolete techniquesModules - obsolete techniques
PIMPL!!! (mostly)
unless you need to provide ABI stability of your APIs
forward declarations (mostly)
precompiled headers
unity builds
70
71. Modules - the current stateModules - the current state
big companies are invested in them (think BIG)
there are 3 implementations already ( , , )
coming in C++20
when standardized and implemented in compilers:
do not expect astronomical gains in speed (for now)
build system support is hard
notably supports modules!
slow adoption in existing software
updating toolchains
source code changes
third party dependencies
if it ain't "broke" don't fix it
MSVC Clang GCC
build2
71
72. Next stepsNext steps
consider runtime C++ compilation & hotswap
make source control checkouts faster
tests
? please!!! it's great, trust me :)
ctest has -j N
Continuous integration
IDE integration of tools for better productivity
change language :| Nim is the anwser *cough*
http://bit.ly/runtime-compilation-alternatives
doctest
mtime_cache
72