Ever wondered how to use modern OpenGL in a way that radically reduces driver overhead? Then this talk is for you.
John McDonald and Cass Everitt gave this talk at Steam Dev Days in Seattle on Jan 16, 2014.
The goal of this session is to demonstrate techniques that improve GPU scalability when rendering complex scenes. This is achieved through a modular design that separates the scene graph representation from the rendering backend. We will explain how the modules in this pipeline are designed and give insights to implementation details, which leverage GPU''s compute capabilities for scene graph processing. Our modules cover topics such as shader generation for improved parameter management, synchronizing updates between scenegraph and rendering backend, as well as efficient data structures inside the renderer.
Video here: http://on-demand.gputechconf.com/gtc/2013/video/S3032-Advanced-Scenegraph-Rendering-Pipeline.mp4
Optimizing the Graphics Pipeline with Compute, GDC 2016Graham Wihlidal
With further advancement in the current console cycle, new tricks are being learned to squeeze the maximum performance out of the hardware. This talk will present how the compute power of the console and PC GPUs can be used to improve the triangle throughput beyond the limits of the fixed function hardware. The discussed method shows a way to perform efficient "just-in-time" optimization of geometry, and opens the way for per-primitive filtering kernels and procedural geometry processing.
Takeaway:
Attendees will learn how to preprocess geometry on-the-fly per frame to improve rendering performance and efficiency.
Intended Audience:
This presentation is targeting seasoned graphics developers. Experience with DirectX 12 and GCN is recommended, but not required.
OpenGL NVIDIA Command-List: Approaching Zero Driver OverheadTristan Lorach
This presentation introduces a new NVIDIA extension called Command-list.
The purpose of this presentation is to explain the basic concepts on how to use it and show what are the benefits.
The sample I used for the talk is here: https://github.com/nvpro-samples/gl_commandlist_bk3d_models
The driver for trying should be PreRelease 347.09
http://www.nvidia.com/download/driverResults.aspx/80913/en-us
OpenGL 4.4 provides new features for accelerating scenes with many objects, which are typically found in professional visualization markets. This talk will provide details on the usage of the features and their effect on real-life models. Furthermore we will showcase how more work for rendering a scene can be off-loaded to the GPU, such as efficient occlusion culling or matrix calculations.
Video presentation here: http://on-demand.gputechconf.com/gtc/2014/video/S4379-opengl-44-scene-rendering-techniques.mp4
Secrets of CryENGINE 3 Graphics TechnologyTiago Sousa
In this talk, the authors will describe an overview of a different method for deferred lighting approach used in CryENGINE 3, along with an in-depth description of the many techniques used. Original file and videos at http://crytek.com/cryengine/presentations
Game engines have long been in the forefront of taking advantage of the ever increasing parallel compute power of both CPUs and GPUs. This talk is about how the parallel compute is utilized in practice on multiple platforms today in the Frostbite game engine and how we think the parallel programming models, hardware and software in the industry should look like in the next 5 years to help us make the best games possible
The goal of this session is to demonstrate techniques that improve GPU scalability when rendering complex scenes. This is achieved through a modular design that separates the scene graph representation from the rendering backend. We will explain how the modules in this pipeline are designed and give insights to implementation details, which leverage GPU''s compute capabilities for scene graph processing. Our modules cover topics such as shader generation for improved parameter management, synchronizing updates between scenegraph and rendering backend, as well as efficient data structures inside the renderer.
Video here: http://on-demand.gputechconf.com/gtc/2013/video/S3032-Advanced-Scenegraph-Rendering-Pipeline.mp4
Optimizing the Graphics Pipeline with Compute, GDC 2016Graham Wihlidal
With further advancement in the current console cycle, new tricks are being learned to squeeze the maximum performance out of the hardware. This talk will present how the compute power of the console and PC GPUs can be used to improve the triangle throughput beyond the limits of the fixed function hardware. The discussed method shows a way to perform efficient "just-in-time" optimization of geometry, and opens the way for per-primitive filtering kernels and procedural geometry processing.
Takeaway:
Attendees will learn how to preprocess geometry on-the-fly per frame to improve rendering performance and efficiency.
Intended Audience:
This presentation is targeting seasoned graphics developers. Experience with DirectX 12 and GCN is recommended, but not required.
OpenGL NVIDIA Command-List: Approaching Zero Driver OverheadTristan Lorach
This presentation introduces a new NVIDIA extension called Command-list.
The purpose of this presentation is to explain the basic concepts on how to use it and show what are the benefits.
The sample I used for the talk is here: https://github.com/nvpro-samples/gl_commandlist_bk3d_models
The driver for trying should be PreRelease 347.09
http://www.nvidia.com/download/driverResults.aspx/80913/en-us
OpenGL 4.4 provides new features for accelerating scenes with many objects, which are typically found in professional visualization markets. This talk will provide details on the usage of the features and their effect on real-life models. Furthermore we will showcase how more work for rendering a scene can be off-loaded to the GPU, such as efficient occlusion culling or matrix calculations.
Video presentation here: http://on-demand.gputechconf.com/gtc/2014/video/S4379-opengl-44-scene-rendering-techniques.mp4
Secrets of CryENGINE 3 Graphics TechnologyTiago Sousa
In this talk, the authors will describe an overview of a different method for deferred lighting approach used in CryENGINE 3, along with an in-depth description of the many techniques used. Original file and videos at http://crytek.com/cryengine/presentations
Game engines have long been in the forefront of taking advantage of the ever increasing parallel compute power of both CPUs and GPUs. This talk is about how the parallel compute is utilized in practice on multiple platforms today in the Frostbite game engine and how we think the parallel programming models, hardware and software in the industry should look like in the next 5 years to help us make the best games possible
Bill explains some of the ways that the Vertex Shader can be used to improve performance by taking a fast path through the Vertex Shader rather than generating vertices with other parts of the pipeline in this AMD technology presentation from the 2014 Game Developers Conference in San Francisco March 17-21. Check out more technical presentations at http://developer.amd.com/resources/documentation-articles/conference-presentations/
Graphics Gems from CryENGINE 3 (Siggraph 2013)Tiago Sousa
This lecture covers rendering topics related to Crytek’s latest engine iteration, the technology which powers titles such as Ryse, Warface, and Crysis 3. Among covered topics, Sousa presented SMAA 1TX: an update featuring a robust and simple temporal antialising component; performant and physically-plausible camera related post-processing techniques such as motion blur and depth of field were also covered.
A technical deep dive into the DX11 rendering in Battlefield 3, the first title to use the new Frostbite 2 Engine. Topics covered include DX11 optimization techniques, efficient deferred shading, high-quality rendering and resource streaming for creating large and highly-detailed dynamic environments on modern PCs.
A description of the next-gen rendering technique called Triangle Visibility Buffer. It offers up to 10x - 20x geometry compared to Deferred rendering and much higher resolution. Generally it aligns better with memory access patterns in modern GPUs compared to Deferred Lighting like Clustered Deferred Lighting etc.
Siggraph2016 - The Devil is in the Details: idTech 666Tiago Sousa
A behind-the-scenes look into the latest renderer technology powering the critically acclaimed DOOM. The lecture will cover how technology was designed for balancing a good visual quality and performance ratio. Numerous topics will be covered, among them details about the lighting solution, techniques for decoupling costs frequency and GCN specific approaches.
Talk by Yuriy O’Donnell at GDC 2017.
This talk describes how Frostbite handles rendering architecture challenges that come with having to support a wide variety of games on a single engine. Yuriy describes their new rendering abstraction design, which is based on a graph of all render passes and resources. This approach allows implementation of rendering features in a decoupled and modular way, while still maintaining efficiency.
A graph of all rendering operations for the entire frame is a useful abstraction. The industry can move away from “immediate mode” DX11 style APIs to a higher level system that allows simpler code and efficient GPU utilization. Attendees will learn how it worked out for Frostbite.
Course presentation at SIGGRAPH 2014 by Charles de Rousiers and Sébastian Lagarde at Electronic Arts about transitioning the Frostbite game engine to physically-based rendering.
Make sure to check out the 118 page course notes on: http://www.frostbite.com/2014/11/moving-frostbite-to-pbr/
During the last few months, we have revisited the concept of image quality in Frostbite. The core of our approach was to be as close as possible to a cinematic look. We used the concept of reference to evaluate the accuracy of produced images. Physically based rendering (PBR) was the natural way to achieve this. This talk covers all the different steps needed to switch a production engine to PBR, including the small details often bypass in the literature.
The state of the art of real-time PBR techniques allowed us to achieve good overall results but not without production issues. We present some techniques for improving convolution time for image based reflection, proper ambient occlusion handling, and coherent lighting units which are mandatory for level editing.
Moreover, we have managed to reduce the quality gap, highlighted by our systematic reference comparison, in particular related to rough material handling, glossy screen space reflection, and area lighting.
The technical part of PBR is crucial for achieving good results, but represents only the top of the iceberg. Frostbite has become the de facto high-end game engine within Electronic Arts and is now used by a large amount of game teams. Moving all these game teams from “old fashion” lighting to PBR has required a lot of education, which have been done in parallel of the technical development. We have provided editing and validation tools to help the transition of art production. In addition, we have built a flexible material parametrisation framework to adapt to the various authoring tools and game teams’ requirements.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Bindless Deferred Decals in The Surge 2Philip Hammer
These are the slides for my talk at Digital Dragons 2019 in Krakow.
Update: The recordings are online on youtube now:
https://www.youtube.com/watch?v=e2wPMqWETj8
Taking Killzone Shadow Fall Image Quality Into The Next GenerationGuerrilla
This talk focuses on the technical side of Killzone Shadow Fall, the platform exclusive launch title for PlayStation 4.
We present the details of several new techniques that were developed in the quest for next generation image quality, and the talk uses key locations from the game as examples. We discuss interesting aspects of the new content pipeline, next-gen lighting engine, usage of indirect lighting and various shadow rendering optimizations. We also describe the details of volumetric lighting, the real-time reflections system, and the new anti-aliasing solution, and include some details about the image-quality driven streaming system. A common, very important, theme of the talk is the temporal coherency and how it was utilized to reduce aliasing, and improve the rendering quality and image stability above the baseline 1080p resolution seen in other games.
Bill explains some of the ways that the Vertex Shader can be used to improve performance by taking a fast path through the Vertex Shader rather than generating vertices with other parts of the pipeline in this AMD technology presentation from the 2014 Game Developers Conference in San Francisco March 17-21. Check out more technical presentations at http://developer.amd.com/resources/documentation-articles/conference-presentations/
Graphics Gems from CryENGINE 3 (Siggraph 2013)Tiago Sousa
This lecture covers rendering topics related to Crytek’s latest engine iteration, the technology which powers titles such as Ryse, Warface, and Crysis 3. Among covered topics, Sousa presented SMAA 1TX: an update featuring a robust and simple temporal antialising component; performant and physically-plausible camera related post-processing techniques such as motion blur and depth of field were also covered.
A technical deep dive into the DX11 rendering in Battlefield 3, the first title to use the new Frostbite 2 Engine. Topics covered include DX11 optimization techniques, efficient deferred shading, high-quality rendering and resource streaming for creating large and highly-detailed dynamic environments on modern PCs.
A description of the next-gen rendering technique called Triangle Visibility Buffer. It offers up to 10x - 20x geometry compared to Deferred rendering and much higher resolution. Generally it aligns better with memory access patterns in modern GPUs compared to Deferred Lighting like Clustered Deferred Lighting etc.
Siggraph2016 - The Devil is in the Details: idTech 666Tiago Sousa
A behind-the-scenes look into the latest renderer technology powering the critically acclaimed DOOM. The lecture will cover how technology was designed for balancing a good visual quality and performance ratio. Numerous topics will be covered, among them details about the lighting solution, techniques for decoupling costs frequency and GCN specific approaches.
Talk by Yuriy O’Donnell at GDC 2017.
This talk describes how Frostbite handles rendering architecture challenges that come with having to support a wide variety of games on a single engine. Yuriy describes their new rendering abstraction design, which is based on a graph of all render passes and resources. This approach allows implementation of rendering features in a decoupled and modular way, while still maintaining efficiency.
A graph of all rendering operations for the entire frame is a useful abstraction. The industry can move away from “immediate mode” DX11 style APIs to a higher level system that allows simpler code and efficient GPU utilization. Attendees will learn how it worked out for Frostbite.
Course presentation at SIGGRAPH 2014 by Charles de Rousiers and Sébastian Lagarde at Electronic Arts about transitioning the Frostbite game engine to physically-based rendering.
Make sure to check out the 118 page course notes on: http://www.frostbite.com/2014/11/moving-frostbite-to-pbr/
During the last few months, we have revisited the concept of image quality in Frostbite. The core of our approach was to be as close as possible to a cinematic look. We used the concept of reference to evaluate the accuracy of produced images. Physically based rendering (PBR) was the natural way to achieve this. This talk covers all the different steps needed to switch a production engine to PBR, including the small details often bypass in the literature.
The state of the art of real-time PBR techniques allowed us to achieve good overall results but not without production issues. We present some techniques for improving convolution time for image based reflection, proper ambient occlusion handling, and coherent lighting units which are mandatory for level editing.
Moreover, we have managed to reduce the quality gap, highlighted by our systematic reference comparison, in particular related to rough material handling, glossy screen space reflection, and area lighting.
The technical part of PBR is crucial for achieving good results, but represents only the top of the iceberg. Frostbite has become the de facto high-end game engine within Electronic Arts and is now used by a large amount of game teams. Moving all these game teams from “old fashion” lighting to PBR has required a lot of education, which have been done in parallel of the technical development. We have provided editing and validation tools to help the transition of art production. In addition, we have built a flexible material parametrisation framework to adapt to the various authoring tools and game teams’ requirements.
Talk by Graham Wihlidal (Frostbite Labs) at GDC 2017.
Checkerboard rendering is a relatively new technique, popularized recently by the introduction of the PlayStation 4 Pro. Many modern game engines are adding support for it right now, and in this talk, Graham will present an in-depth look at the new implementation in Frostbite, which is used in shipping titles like 'Battlefield 1' and 'Mass Effect Andromeda'. Despite being conceptually simple, checkerboard rendering requires a deep integration into the post-processing chain, in particular temporal anti-aliasing, dynamic resolution scaling, and poses various challenges to existing effects. This presentation will cover the basics of checkerboard rendering, explain the impact on a game engine that powers a wide range of titles, and provide a detailed look at how the current implementation in Frostbite works, including topics like object id, alpha unrolling, gradient adjust, and a highly efficient depth resolve.
Bindless Deferred Decals in The Surge 2Philip Hammer
These are the slides for my talk at Digital Dragons 2019 in Krakow.
Update: The recordings are online on youtube now:
https://www.youtube.com/watch?v=e2wPMqWETj8
Taking Killzone Shadow Fall Image Quality Into The Next GenerationGuerrilla
This talk focuses on the technical side of Killzone Shadow Fall, the platform exclusive launch title for PlayStation 4.
We present the details of several new techniques that were developed in the quest for next generation image quality, and the talk uses key locations from the game as examples. We discuss interesting aspects of the new content pipeline, next-gen lighting engine, usage of indirect lighting and various shadow rendering optimizations. We also describe the details of volumetric lighting, the real-time reflections system, and the new anti-aliasing solution, and include some details about the image-quality driven streaming system. A common, very important, theme of the talk is the temporal coherency and how it was utilized to reduce aliasing, and improve the rendering quality and image stability above the baseline 1080p resolution seen in other games.
A set of mobile game optimization best practices. This presentation extensively covers PowerVR series of GPUs from Imagination Technologies and iOS, however the majority of recommendations can be applied to other GPUs and mobile operating systems.
Stockage, manipulation et analyse de données matricielles avec PostGIS RasterACSG Section Montréal
La plus importantes nouveautés de la base de données spatiale open source PostgreSQL/PostGIS 2.0 est le support pour les données raster. PostGIS Raster comprend un outil d’importation similaire à shp2pgsql basé sur GDAL et une série d’opérateurs SQL pour la manipulation et l'analyse des données matricielles. Le nouveau type RASTER est géoréférencé, multi-résolutions et multi-bandes et il supporte une valeur nulle (nodata) et un type de valeur de pixel par bande. PostGIS raster s’inspire de la simplicité de l’expérience vecteur offerte par PostGIS pour rendre toutes les opérations raster aussi simples que possible. Comme pour une couverture vecteur, une couverture raster est divisée en un ensemble d’enregistrements (une ligne = une tuile) stockés dans une seule table (contrairement à Oracle Spatial qui utilise deux types et donc deux tables ou plus). Il est possible d’importer une couverture complète et de la retuiler en une seule commande avec l’outil d’importation et de multiples résolutions de la même couverture peuvent être importées dans des tables adjacentes. Les propriétés des objets raster et de chacune des bandes peuvent être consultées et modifiées ainsi que les valeurs des pixels. Des fonctions existent pour obtenir le minimum, le maximum, la somme, la moyenne, la déviation standard, l’histogramme d’une tuile ou d’une couverture complète. Les fonctions ST_Intersection() et ST_Intersects() fonctionnent pratiquement de manière transparente entre des données raster et vecteur et une série de fonctions pour l’algèbre matricielle (ST_MapAlgebra()) permet de faire de l’analyse de type raster. Il est possible de reclasser les bandes et de les convertir en n’importe quel format d’écriture GDAL. Des fonctions pour générer des rasters et des bandes existent également pour du développement PL/pgSQL. Un driver GDAL pour convertir les couvertures raster en fichiers images est en développement et des plugins pour QGIS et svSIG existent déjà pour les visualiser.
The presentation covers three engine systems created for new Shadow Warrior: skinned decals, vegetation, and seawater rendering.
Subsequent presentation here:
https://www.slideshare.net/jaros_p/shadow-warrior-2-and-the-evolution-of-the-roadhog-engine
GDC talk on how to render Per-Face Texture Mapping (PTEX) datasets on commodity GPUs in Realtime in a much simpler fashion than the earlier proposed techniques. This method is heavily suitable for real time consumption, and can be altered to support other texture techniques.
Rendering Technologies from Crysis 3 (GDC 2013)Tiago Sousa
This talk covers changes in CryENGINE 3 technology during 2012, with DX11 related topics such as moving to deferred rendering while maintaining backward compatibility on a multiplatform engine, massive vegetation rendering, MSAA support and how to deal with its common visual artifacts, among other topics.
Hadoop interview questions for freshers and experienced people. This is the best place for all beginners and Experts who are eager to learn Hadoop Tutorial from the scratch.
Read more here http://softwarequery.com/hadoop/
Accelerating HPC Applications on NVIDIA GPUs with OpenACCinside-BigData.com
In this deck from the Stanford HPC Conference, Doug Miles from NVIDIA presents: Accelerating HPC Applications on NVIDIA GPUs with OpenACC."
"OpenACC is a directive-based parallel programming model for GPU accelerated and heterogeneous parallel HPC systems. It offers higher programmer productivity compared to use of explicit models like CUDA and OpenCL.
Application source code instrumented with OpenACC directives remains portable to any system with a standard Fortran/C/C++ compiler, and can be efficiently parallelized for various types of HPC systems – multicore CPUs, heterogeneous CPU+GPU, and manycore processors.
This talk will include an introduction to the OpenACC programming model, provide examples of its use in a number of production applications, explain how OpenACC and CUDA Unified Memory working together can dramatically simplify GPU programming, and close with a few thoughts on OpenACC future directions."
Watch the video: https://youtu.be/CaE3n89QM8o
Learn more: https://www.openacc.org/
and
http://hpcadvisorycouncil.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
SOM (Self-Organizing Map) is one of the most popular artificial neural network algorithms in the unsupervised learning category. For efficient construction of large maps searching the best-matching unit is usually the computationally heaviest operation in the SOM. The parallel nature of the algorithm and the huge computations involved makes it a good target for GPU based parallel implementation. This paper presents an overall idea of the optimization strategies used for the parallel implementation of Basic-SOM on GPU using CUDA programming paradigm.
ClickHouse and the Magic of Materialized Views, By Robert Hodges and Altinity...Altinity Ltd
Presented at the webinar, June 26, 2019
Materialized views are a killer feature of ClickHouse that can speed up queries 20X or more. Our webinar will teach you how to use this potent tool starting with how to create materialized views and load data. We'll then walk through cookbook examples to solve practical problems like deriving aggregates that outlive base data, answering last point queries, and using AggregateFunctions to handle problems like counting unique values, which is a special ClickHouse feature. There will be time for Q&A at the end. At that point you'll be a wizard of ClickHouse materialized views and able to cast spells of your own.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
2. Who are we?
Cass Everitt, NVIDIA Corporation
John McDonald, NVIDIA Corporation
3. What will we cover?
Dynamic Buffer Generation
Efficient Texture Management
Increasing Draw Call Count
4. Dynamic Buffer Generation
Problem
Our goal is to generate dynamic geometry directly in place.
It will be used one time, and will be completely regenerated next frame.
Particle systems are the most common example
Vegetation / foliage also common
6. The horror
void UpdateParticleData(uint _dstBuf) {
BindBuffer(ARRAY_BUFFER, _dstBuf);
access = MAP_UNSYNCHRONIZED | MAP_WRITE_BIT;
for particle in allParticles {
dataSize = GetParticleSize(particle);
void* dst = MapBuffer(ARRAY_BUFFER, offset, dataSize, access);
(*(Particle*)dst) = *particle;
UnmapBuffer(ARRAY_BUFFER);
This is so slow.
offset += dataSize;
}
};
// Now render with everything.
7. Driver interlude
First, a quick interlude on modern GL drivers
In the application (client) thread, the driver is very thin.
It simply packages work to hand off to the server thread.
The server thread does the real processing
It turns command sequences into push buffer fragments.
9. MAP_UNSYNCHRONIZED
Avoids an application-GPU sync point (a CPU-GPU sync point)
But causes the Client and Server threads to serialize
This forces all pending work in the server thread to complete
It’s quite expensive (almost always needs to be avoided)
12. It’s okay
Q: What’s better than mapping in an unsynchronized manner?
A: Keeping around a pointer to GPU-visible memory forever.
Introducing: ARB_buffer_storage
13. ARB_buffer_storage
Conceptually similar to ARB_texture_storage (but for buffers)
Creates an immutable pointer to storage for a buffer
The pointer is immutable, the contents are not.
So BufferData cannot be called—BufferSubData is still okay.
Allows for extra information at create time.
For our usage, we care about the PERSISTENT and COHERENT
bits.
PERSISTENT: Allow this buffer to be mapped while the GPU is using it.
COHERENT: Client writes to this buffer should be immediately visible to
the GPU.
http://www.opengl.org/registry/specs/ARB/buffer_storage.txt
14. ARB_buffer_storage cont’d
Also affects the mapping behavior (pass persistent and coherent
bits to MapBufferRange)
Persistently mapped buffers are good for:
Dynamic VB / IB data
Highly dynamic (~per draw call) uniform data
Multi_draw_indirect command buffers (more on this later)
Not a good fit for:
Static geometry buffers
Long lived uniform data (still should use BufferData or BufferSubData for
this)
15. Armed with persistently mapped buffers
// At the beginning of time
flags = MAP_WRITE_BIT | MAP_PERSISTENT_BIT | MAP_COHERENT_BIT;
BufferStorage(ARRAY_BUFFER, allParticleSize, NULL, flags);
mParticleDst = MapBufferRange(ARRAY_BUFFER, 0, allParticleSize,
flags);
mOffset = 0;
// allParticleSize should be ~3x one frame’s worth of particles
// to avoid stalling.
16. Update Loop (old and busted)
void UpdateParticleData(uint _dstBuf) {
BindBuffer(ARRAY_BUFFER, _dstBuf);
access = MAP_UNSYNCHRONIZED | MAP_WRITE_BIT;
for particle in allParticles {
dataSize = GetParticleSize(particle);
void* dst = MapBuffer(ARRAY_BUFFER, offset, dataSize, access);
(*(Particle*)dst) = *particle;
offset += dataSize;
UnmapBuffer(ARRAY_BUFFER);
}
};
// Now render with everything.
17. Update Loop (new hotness)
void UpdateParticleData() {
for particle in allParticles {
dataSize = GetParticleSize(particle);
mParticleDst[mOffset] = *particle;
mOffset += dataSize; // Wrapping not shown
}
};
// Now render with everything.
19. Performance results
160,000 point sprites
Specified in groups of 6 vertices (one particle at a time)
Synthetic (naturally)
Method
FPS
Particles / S
Map(UNSYNCHRONIZED)
1.369
219,040
BufferSubData
17.65
2,824,000
D3D11 Map(NO_OVERWRITE)
20.25
3,240,000
20. Performance results
160,000 point sprites
Specified in groups of 6 vertices (one particle at a time)
Synthetic (naturally)
Method
FPS
Particles / S
Map(UNSYNCHRONIZED)
1.369
219,040
BufferSubData
17.65
2,824,000
D3D11 Map(NO_OVERWRITE)
20.25
3,240,000
Map(COHERENT|PERSISTENT)
79.9
12,784,000
Room for improvement still, but much, much better.
21. The other shoe
You are responsible for not stomping on data in flight.
Why 3x?
1x: What the GPU is using right now.
2x: What the driver is holding, getting ready for the GPU to use.
3x: What you are writing to.
3x should ~ guarantee enough buffer room*…
Use fences to ensure that rendering is complete before you begin
to write new data.
22. Fencing
Use FenceSync to place a new fence.
When ready to scribble over that memory again, use
ClientWaitSync to ensure that memory is done.
ClientWaitSync will block the client thread until it is ready
So you should wrap this function with a performance counter
And complain to your log file (or resize the underlying buffer) if you
frequently see stalls here
For complete details on correct management of buffers with
fencing, see Efficient Buffer Management [McDonald 2012]
24. Problem
Changing textures breaks batches.
Not all texture data is needed all the time
Texture data is large (typically the largest memory bucket for games)
Bindless solves this, but can hurt GPU performance
Too many different textures can fall out of TexHdr$
Not a bindless problem per se
25. Terminology
Reserve – The act of allocating virtual memory
Commit – Tying a virtual memory allocation to a physical backing
store (Physical memory)
Texture Shape – The characteristics of a texture that affect its
memory consumption
Specifically: Height, Width, Depth, Surface Format, Mipmap Level Count
27. Texture Arrays
Introduced in GL 3.0, and D3D 10.
Arrays of textures that are the same shape and format
Typically can contain many ―layers‖ (2048+)
Filtering works as expected
As does mipmapping!
28. Sparse Bindless Texture Arrays
Organize loose textures into Texture Arrays.
Sparsely allocate Texture Arrays
Introducing ARB_sparse_texture
Consume virtual memory, but not physical memory
Use Bindless handles to deal with as many arrays as needed!
Introducing ARB_bindless_texture
uncommitted
uncommitted
uncommitted
layer
layer
layer
29. ARB_sparse_texture
Applications get fine-grained control of physical
memory for textures with large virtual allocations
Inspired by Mega Texture
Primary expected use cases:
Sparse texture data
Texture paging
Delayed-loading assets
http://www.opengl.org/registry/specs/ARB/sparse_texture.txt
30. ARB_bindless_texture
Textures specified by GPU-visible ―handle‖ (really an address)
Rather than by name and binding point
Can come from ~anywhere
Uniforms
Varying
SSBO
Other textures
Texture residency also application-controlled
Residency is ―does this live on the GPU or in sysmem?‖
https://www.opengl.org/registry/specs/ARB/bindless_texture.txt
31. Advantages
Artists work naturally
No preprocessing required (no bake-step required)
Although preprocessing is helpful if ARB_sparse_texture is unavailable
Reduce or eliminate TexHdr$ thrashing
Even as compared to traditional texturing
Programmers manage texture residency
Works well with arbitrary streaming
Faster on the CPU
Faster on the GPU
32. Disadvantages
Texture addresses are now structs (96 bits).
64 bits for bindless handle
32 bits for slice index (could reduce this to 10 bits at a perf cost)
ARB_sparse_texture implementations are a bit immature
Early adopters: please bring us your bugs.
ARB_sparse_texture requires base level be a multiple of tile size
(Smaller is okay)
Tile size is queried at runtime
Textures that are power-of-2 should almost always be safe.
33. Implementation Overview
When creating a new texture…
Check to see if any suitable texture array exists
Texture arrays can contain a large number of textures of the same shape
Ex. Many TEXTURE_2Ds grouped into a single TEXTURE_2D_ARRAY
If no suitable texture, create a new one.
34. Texture Container Creation (example)
GetIntegerv( MAX_SPARSE_ARRAY_TEXTURE_LAYERS, maxLayers );
Choose a reasonable size (e.g. array size ~100MB virtual )
If new internalFormat, choose page size
GetInternalformativ( …, internalformat, NUM_VIRTUAL_PAGE_SIZES, 1, &numIndexes);
Note: numIndexes can be 0, so have a plan
Iterate, select suitable pageSizeIndex
BindTexture( TEXTURE_2D_ARRAY, newTexArray );
TexParameteri( TEXTURE_SPARSE, TRUE );
TexParameteri( VIRTUAL_PAGE_SIZE_INDEX, pageSizeIndex );
Allocate the texture’s virtual memory using TexStorage3D
35. Specifying Texture Data
Using the located/created texture array from the previous step
Allocate a layer as the location of our data
For each mipmap level of the allocated layer:
Commit the entire mipmap level (using TexPageCommitment)
Specify actual texel data as usual for arrays
gl(Compressed|Copy|)TexSubImage3D
PBO updates are fine too
uncommitted
Allocated layer
free
free
layer
slice
slice
36. Freeing Textures
To free the texture, reverse the process:
Use TexPageCommitment to mark the entire layer (slice) as free.
Do once for each mipmap level
Add the layer to the free list for future allocation
uncommitted
free
free
layer
slice
slice
uncommitted
Freed layer
layer
37. Combining with Bindless to eliminate binds
At container create time:
Specify sampling parameters via SamplerParameter calls first
Call GetTextureSamplerHandleARB to return a GPU-visible pointer to the
texture+sampler container
Call MakeTextureHandleResident to ensure the resource lives on the GPU
At delete time, call MakeTextureHandleNonResident
With bindless, you explicitly manage the GPU’s working set
38. Using texture data in shaders
When a texture is needed with the default sampling parameters
Create a GLSL-visible TextureRef object:
struct TextureRef {
sampler2DArray container;
float slice;
};
When a texture is needed with custom sampling parameters
Create a separate sampler object for the shader with the parameters
Create a bindless handle to the pair using
GetTextureSamplerHandle, then call MakeTextureHandleResident
with the new value
And fill out a TextureRef as above for usage by GLSL
39. C++ Code
Basic implementation (some details missing)
BSD licensed (use as you will)
https://github.com/nvMcJohn/apitest/blob/pdoane_newtests/sparse_bindless_texarray.h
https://github.com/nvMcJohn/apitest/blob/pdoane_newtests/sparse_bindless_texarray.cpp
41. All the Draw Calls!
Problem
You want more draw calls of smaller objects.
D3D is slow at this.
Naïve GL is faster than D3D, but not fast enough.
42. XY Problem
Y: How can I have more draw calls?
X: You don’t really care if it’s more draw calls, right?
Really what you want is to be able to draw more small geometry
groupings. More objects.
43. Well why didn’t you just say so??
First, some background.
What makes draw calls slow?
Real world API usage
Draw Call Cost Visualization
44. Some background
What causes slow draw calls?
Validation is the biggest bucket (by far).
Pre-validation is ―difficult‖
―Every application does the same things.‖
Not really. Most applications are in completely disjoint states
Try this experiment: What is important to you?
Now ask your neighbor what’s important to him.
45. Why is prevalidation difficult?
The GPU is an exceedingly complex state machine.
(Honestly, it’s probably the most complex state machine in all of CS)
Any one of those states may have a problem that requires WAR
Usually the only problem is overall performance
But sometimes not.
There are millions of tests covering NVIDIA GPU functionality.
46. FINE.
How can app devs mitigate these costs?
Minimize state changes.
All state changes are not created equal!
Cost of a draw call:
Small fixed cost + Cost of validation of changed state
47. Feels limiting…
Artists want lots of materials, and small amounts of geometry
Even better: What if artists just didn’t have to care about this?
Ideal Programmer->Artist Interaction
―You make pretty art. I’ll make it fit.‖
48. Relative costs of State Changes
In decreasing cost…
Render Target
Program
ROP
Texture Bindings
Vertex Format
UBO Bindings
Vertex Bindings
Uniform Updates
~60K / s
~300K / s
~1.5M / s
~10M / s
Note: Not to scale
49. Real World API frequency
API usage looks roughly like this…
Increasing Frequency of Change
Render Target (scene)
Per Scene Uniform Buffer + Textures
IB / VB and Input Layout
Shader (Material)
Per-material Uniform Buffer + Textures
Per-object Uniform Buffer + Textures
Per-piece Uniform Buffer + Textures
Draw
51. Draw Calls visualized (cont’d)
Read down, then right
Black—no change
Render Target
Texture
Uniform Updates
Program
UBO Binding
Draw
ROP
Vertex Format
52. Goals
Let’s minimize validation costs without affecting artists
Things we need to be fast (per app call frequency):
Uniform Updates and binding
Texture Updates and binding
These happen most often in app, ergo driving them to ~0 should
be a win.
53. Textures
Using Sparse Bindless Texture Arrays (as previously described)
solves this.
All textures are set before any drawing begins
(No need to change textures between draw calls)
Note that from the CPU’s perspective, just using bindless is
sufficient.
That was easy.
54. Eliminating Texture Binds -- visualized
Increasing Frequency of Change
Render Target (scene)
Per Scene Uniform Buffer + Textures
IB / VB and Input Layout
Shader (Material)
Per-material Uniform Buffer + Textures
Per-object Uniform Buffer + Textures
Per-piece Uniform Buffer + Textures
Draw
Render Target
Texture
Uniform Updates
Program
UBO Binding
Draw
ROP
Vertex Format
55. Boom!
Increasing Frequency of Change
Render Target (scene)
Per Scene Uniform Buffer
IB / VB and Input Layout
Shader (Material)
Per-material Uniform Buffer
Per-object Uniform Buffer
Per-piece Uniform Buffer
Draw
Render Target
Texture
Uniform Updates
Program
UBO Binding
Draw
ROP
Vertex Format
56. Buffer updates (old and busted)
Typical Scene Graph Traversal
for obj in visibleObjectSet {
update(buffer, obj);
draw(obj);
}
57. Buffer updates (new hotness)
Typical Scene Graph Traversal
for obj in visibleObjectSet {
update(bufferFragment, obj);
}
for obj in visibleObjectSet {
draw(obj);
}
58. bufferFragma-wha?
Rather than one buffer per object, we share UBOs for many
objects.
ie, given struct ObjectUniforms { /* … */ };
// Old (probably not explicitly instantiated,
// just scattered in GLSL)
ObjectUniforms uniformData;
// New
ObjectUniforms uniformData[ObjectsPerKickoff];
Use persistent mapping for even more win here!
For large amounts of data (bones) consider SSBO.
Introducing ARB_shader_storage_buffer_object
59. SSBO?
Like ―large‖ uniform buffer objects.
Minimum required size to claim support is 16M.
Accessed like uniforms in shader
Support for better packing (std430)
Caveat: They are typically implemented in hardware as textures
(and can introduce dependent texture reads)
Just one of a laundry list of things to consider, not to discourage use.
http://www.opengl.org/registry/specs/ARB/shader_storage_buffer_object.txt
60. Eliminating Buffer Update Overhead
Increasing Frequency of Change
Render Target (scene)
Per Scene Uniform Buffer
IB / VB and Input Layout
Shader (Material)
Per-material Uniform Buffer
Per-object Uniform Buffer
Per-piece Uniform Buffer
Draw
Render Target
Texture
Uniform Updates
Program
UBO Binding
Draw
ROP
Vertex Format
61. Sweet!
Increasing Frequency of Change
Render Target (scene)
IB / VB and Input Layout
Shader (Material)
Draw ( * each object )
Hrrrrmmmmmm….
Render Target
Texture
Uniform Updates
Program
UBO Binding
Draw
ROP
Vertex Format
62. So now…
It’d be awesome if we could do all of those kickoffs at once.
Validation is already only paid once
But we could just pay the constant startup cost once.
If only…….
63. So now…
It’d be awesome if we could do all of those kickoffs at once.
Validation is already only paid once
But we could just pay the constant startup cost once.
If only…….
Introducing ARB_multi_draw_indirect
64. ARB_multi_draw_indirect
Allows you to specify parameters to draw commands from a
buffer.
This means you can generate those parameters wide (on the CPU)
Or even on the GPU, via compute program.
http://www.opengl.org/registry/specs/ARB/multi_draw_indirect.txt
68. Knowing which shader data is mine
Use ARB_shader_draw_parameters, a necessary companion to
ARB_multi_draw_indirect
Adds a builtin to the VS: DrawID (InstanceID already available)
This tells you which command of a MultiDraw command is being
executed.
When not using MultiDraw, the builtin is specified to be 0.
Caveat: Right now, you have to pass this down to other shader
stages as an interpolant.
Hoping to have that rectified via ARB or EXT extension ―real soon now.‖
http://www.opengl.org/registry/specs/ARB/shader_draw_parameters.txt
69. Applying everything
CPU Perf is massively better
5-30x increase in number of distinct objects / s
Interaction with driver is decreased ~75%
Note: GPU perf can be affected negatively (although not too
badly)
As always: Profile, profile, profile.
Lamest point sprites imaginableTriangles, but specified as 2-D coordinates. Spaced for obviousness
Could obviously improve efficiency by only calling regular Map once per frame. This performance is worse than BufferSubData, but better Map(UNSYNCHRONIZED otherwise).
Could obviously improve efficiency by only calling regular Map once per frame. This performance is worse than BufferSubData, but better Map(UNSYNCHRONIZED otherwise).
With bindless, sampler* can be passed around like any other 64-bit integer. Note that if MakeTextureHandleResident succeeds, the texture is definitely on the GPUBut that MakeTextureHandleNonResident only tells the driver that it may choose to free the GPU memory consumed by the texture.
M
Speed and power come from fixed-functionality.But that means that the GPU is rigid in behaviors.