This document discusses Go memory management and garbage collection. It explains that Go uses garbage collection to free unused memory blocks and scavenging to return idle memory to the operating system. It provides details on allocation primitives like new() and make(), garbage collection configuration using GOGC, and memory statistics available via runtime.ReadMemStats(). Code examples are provided to demonstrate buffer pooling to reduce garbage collection overhead.
Go debugging and troubleshooting tips - from real life lessons at SignalFxSignalFx
Exploring tips and advice on writing production Go systems that are easy to debug and troubleshoot. Jack Lindamood from SignalFx presents patterns that facilitate this process.
Jack addresses tools built into Go you can take advantage of, build process techniques they've learned over time, and open source tools and libraries you can use that help troubleshoot your production code when things go wrong.
Read more here: http://blog.signalfx.com/a-pattern-for-optimizing-go
Most computers today have multiple cores and processes. That means, if you really want to fully explore the capabilities of them, you must explore concurrent programming resources. Go was developed with this in mind, offering goroutines and channels.
Although concurrent programming may seem simple with Go, there are a couple of details to keep in mind, to avoid problems. In this talk, I’m going to show a couple of practical examples with goroutines and channels, and some common pitfalls
Go debugging and troubleshooting tips - from real life lessons at SignalFxSignalFx
Exploring tips and advice on writing production Go systems that are easy to debug and troubleshoot. Jack Lindamood from SignalFx presents patterns that facilitate this process.
Jack addresses tools built into Go you can take advantage of, build process techniques they've learned over time, and open source tools and libraries you can use that help troubleshoot your production code when things go wrong.
Read more here: http://blog.signalfx.com/a-pattern-for-optimizing-go
Most computers today have multiple cores and processes. That means, if you really want to fully explore the capabilities of them, you must explore concurrent programming resources. Go was developed with this in mind, offering goroutines and channels.
Although concurrent programming may seem simple with Go, there are a couple of details to keep in mind, to avoid problems. In this talk, I’m going to show a couple of practical examples with goroutines and channels, and some common pitfalls
Lua: the world's most infuriating languagejgrahamc
Slides from a talk I gave at the Lua London Meetup on October 17: "When first confronted with Lua an experienced programmer (like me!) finds themselves infuriated by the languages little differences and hopes that can dismiss it as not worth learning :-) Later they find themselves infuriated to learn that they can't dismiss it: Lua is just too fast, too useful and too flexible. This talk will look at my experience of learning Lua and using it to send a high-altitude balloon into the stratosphere and build CloudFlare's new low latency WAF."
All you need to know about the JavaScript event loopSaša Tatar
Learn the difference between JavaScript Engine, JavaScript Runtime, what is JavaScript event loop and why we should care.
At the end the presentation goes through a couple of examples and implementations of throttle and debounce utility functions.
Go is a language developed by Google with multi-core in mind. Differ from other languages, concurrency is a first-class primitive in Go. This talk covers some useful patterns for dealing with concurrency.
Go provides:
+ Concurrent execution
+ Synchronization and messaging
+ Multi-way concurrent control
Go does NOT provide direct thread manipulation
- Go multithreading is fully managed by Go runtime.
- In Go you use goroutines in place of threads
Learn about Goroutines, channels and standard library
Capture the Flag (CTF) are information security challenges. They are fun, but they also provide a opportunity to practise for real-world security challenges.
In this talk we present the concept of CTF. We focus on some tools used by our team, which can also be used to solve real-world problems.
A presentation about combining the powers of the Go language and Asterisk in order to provide fast, reliable and hugely scalable voice applications. A brief introduction about why Go presents a big opportunity for the asterisk community, a primer into developing FastAGI applications, demonstration of the AGI package [1] and a walk through the idioms and challenges of developing in Go. The talk is focused mainly on our experience of porting existing code into Go with the aim to scale our services to larger numbers accompanied with benchmarks and an introduction to some tools [2] we developed to help us test, debug and benchmark AGI applications.
~10min dive to Python Asynchronous IO
HTML version (recommended): https://dl.dropboxusercontent.com/u/1565687/speak/Python3%20AsyncIO%20Horizon/index.html
Droidcon Berlin 2021 - With coroutines being the de facto way of exposing async work and streams of changes for Kotlin on Android, developers are obviously attempting to use the same approaches when moving their code to Multiplatform.
But due to the way the memory model differs between JVM and Kotlin Native, it can be a painful experience.
In this talk, we will take a deep dive into the Coroutine API for Kotlin Multiplatform. You will learn how to expose your API with Coroutines while working with the Kotlin Native memory model instead of against it, and avoid the dragons along the way.
Sebastian Pożoga
Z komputerami miałem styczność, odkąd pamiętam. Od początku moje zainteresowania podzielone były między elektronikę a programowanie. Poświęciłem się drugiemu. Jednak rozwój mikrokomputerów pozwolił mi przypomnieć sobie o starej pasji. Jestem organizatorem HardGroup, Poznańskiej społeczności osób hobbystycznie i zawodowo się elektroniką i majsterkowaniem. Organizujemy darmowe spotkania i warsztaty, na które serdecznie zapraszam. Po pracy rozwijam własne projekt. Odwiedzam liczne imprezy edukacyjne i konferencje. Prowadzę bloga. Jestem dumny, że pomagam rozwijać pasję innym.
Temat prelekcji: GOLang & GoatCore
GOLang to nowy język Google. Już prawie od 10-lecia rozwijany i promowany przez firmę. Podczas prezentacji opowiem, jakie ma mocne strony. Ile nauczyłem się, rozwijając projekt Open Source w GO i dlaczego mi się tak podoba. Podczas prezentacji omówię, dlaczego powstał GoatCore, jak działa i kiedy nam pomaga. Omówię różne elementy języka. Poruszę temat współbieżności, optymalizacji pamięci, dependency injection, optymalizacji. Prezentacja jest przeznaczona dla doświadczonych programistów, którzy w stopniu co najmniej dobrym znają inny język programowania, narzędzia i wzorce projektowe. Omówię specyfikę języka i charakterystyczne cechy. Jednak skupiać się będę na konkretnych problemach i rozwiązaniach.
GoatCore źródło: github.com/goatcms/goatcore
Lua: the world's most infuriating languagejgrahamc
Slides from a talk I gave at the Lua London Meetup on October 17: "When first confronted with Lua an experienced programmer (like me!) finds themselves infuriated by the languages little differences and hopes that can dismiss it as not worth learning :-) Later they find themselves infuriated to learn that they can't dismiss it: Lua is just too fast, too useful and too flexible. This talk will look at my experience of learning Lua and using it to send a high-altitude balloon into the stratosphere and build CloudFlare's new low latency WAF."
All you need to know about the JavaScript event loopSaša Tatar
Learn the difference between JavaScript Engine, JavaScript Runtime, what is JavaScript event loop and why we should care.
At the end the presentation goes through a couple of examples and implementations of throttle and debounce utility functions.
Go is a language developed by Google with multi-core in mind. Differ from other languages, concurrency is a first-class primitive in Go. This talk covers some useful patterns for dealing with concurrency.
Go provides:
+ Concurrent execution
+ Synchronization and messaging
+ Multi-way concurrent control
Go does NOT provide direct thread manipulation
- Go multithreading is fully managed by Go runtime.
- In Go you use goroutines in place of threads
Learn about Goroutines, channels and standard library
Capture the Flag (CTF) are information security challenges. They are fun, but they also provide a opportunity to practise for real-world security challenges.
In this talk we present the concept of CTF. We focus on some tools used by our team, which can also be used to solve real-world problems.
A presentation about combining the powers of the Go language and Asterisk in order to provide fast, reliable and hugely scalable voice applications. A brief introduction about why Go presents a big opportunity for the asterisk community, a primer into developing FastAGI applications, demonstration of the AGI package [1] and a walk through the idioms and challenges of developing in Go. The talk is focused mainly on our experience of porting existing code into Go with the aim to scale our services to larger numbers accompanied with benchmarks and an introduction to some tools [2] we developed to help us test, debug and benchmark AGI applications.
~10min dive to Python Asynchronous IO
HTML version (recommended): https://dl.dropboxusercontent.com/u/1565687/speak/Python3%20AsyncIO%20Horizon/index.html
Droidcon Berlin 2021 - With coroutines being the de facto way of exposing async work and streams of changes for Kotlin on Android, developers are obviously attempting to use the same approaches when moving their code to Multiplatform.
But due to the way the memory model differs between JVM and Kotlin Native, it can be a painful experience.
In this talk, we will take a deep dive into the Coroutine API for Kotlin Multiplatform. You will learn how to expose your API with Coroutines while working with the Kotlin Native memory model instead of against it, and avoid the dragons along the way.
Sebastian Pożoga
Z komputerami miałem styczność, odkąd pamiętam. Od początku moje zainteresowania podzielone były między elektronikę a programowanie. Poświęciłem się drugiemu. Jednak rozwój mikrokomputerów pozwolił mi przypomnieć sobie o starej pasji. Jestem organizatorem HardGroup, Poznańskiej społeczności osób hobbystycznie i zawodowo się elektroniką i majsterkowaniem. Organizujemy darmowe spotkania i warsztaty, na które serdecznie zapraszam. Po pracy rozwijam własne projekt. Odwiedzam liczne imprezy edukacyjne i konferencje. Prowadzę bloga. Jestem dumny, że pomagam rozwijać pasję innym.
Temat prelekcji: GOLang & GoatCore
GOLang to nowy język Google. Już prawie od 10-lecia rozwijany i promowany przez firmę. Podczas prezentacji opowiem, jakie ma mocne strony. Ile nauczyłem się, rozwijając projekt Open Source w GO i dlaczego mi się tak podoba. Podczas prezentacji omówię, dlaczego powstał GoatCore, jak działa i kiedy nam pomaga. Omówię różne elementy języka. Poruszę temat współbieżności, optymalizacji pamięci, dependency injection, optymalizacji. Prezentacja jest przeznaczona dla doświadczonych programistów, którzy w stopniu co najmniej dobrym znają inny język programowania, narzędzia i wzorce projektowe. Omówię specyfikę języka i charakterystyczne cechy. Jednak skupiać się będę na konkretnych problemach i rozwiązaniach.
GoatCore źródło: github.com/goatcms/goatcore
The JVM memory model describes how threads in the Java eco-system interact through memory. While the memory model impact on developing for the JVM may not be obvious, it is the cause for certain number of "anomalies" that are, well, by design.
In this presentation we will explore the aspects of the memory model, including things like reordering of instructions, volatile members, monitors, atomics and JIT.
Many developers are forced to develop on small screens: Travelling, Coworking spaces, changing project locations, you name it. But what can you do to improve your workflow if you are bound to a small screen? This presentation tries to give some answers and suggestions.
Golang Performance : microbenchmarks, profilers, and a war storyAerospike
Slides for Brian Bulkowski's talk about Golang performance:
microbenchmarks, profilers, and a war story about optimizing the Aerospike Database Go client.
http://www.meetup.com/Go-lang-Developers-NYC/events/216650022/
JDD2015: On-heap cache vs Off-heap cache - Radek GrębskiPROIDEA
ON-HEAP CACHE VS OFF-HEAP CACHE
W prezentacji przybliżę temat pamięci off-heap, czyli obszaru pamięci nie zarządzanej przez JVM. Przedstawię wady i zalety korzystania z tego rozwiązania, sposoby użycia klas ByteBuffer (direct/non-direct, big_endian/little_endian), Unsafe.allocateMemory oraz powiem czym jest memory mapped file.
Następnie omówię 3 rodzaje cache'e:
- Hazelcast (on heap cache)
- ChronicleMap (off-heap cache/store)
- Redis (napisany w C)
oraz przedstawię porównanie wydajności (zapis, odczyt, pauzy GC) z naciskiem na off-heapowe rozwiązanie.
Pokażę również fragmenty kodu ChronicleMap.
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...Víctor Bolinches
Charla Universidad de Valencia - http://www.uv.es/
Cátedra de Capgemini 2016/17 - http://www.uv.es/capgeminiuv/sobre_nosotros.html
Asignatura : Programación avanzada y Lenguajes de programación
Título : Paradigma FP y OOP usando técnicas avanzadas de Programación
Compromising Linux Virtual Machines with Debugging MechanismsRussell Sanford
This presentation covers utilizing VMwares (GDB) debugging protocol to invasive inject commands into a Linux-x64 target. Automatic detection of kernel API is performed to locate _vmalloc & call_usermodehelper* functions across all 3x and 4x kernels.
It's time to deprecate JavaScript. It's security model and the language itself are appalling.
As data moves into the cloud the JavaScript threat is increasing and I believe the only way to fix this is to start all over again. The 14 year old language and security model aren't up to today's threats.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
2. www.cloudflare.com!
Allocation Primitives
• new(T)!
• Allocates memory for item with type T!
• Zeroes it
!
• make(T)!
• Allocates memory for item with type T!
• Initializes it
• Needed for channels, maps and slices
• Memory comes from internal heap
ret = runtime·mallocgc(typ->size, flag, 1, 1);!
zeroed
3. www.cloudflare.com!
Two memory freeing processes
• Garbage collection
• Determines which blocks of memory are no longer used
• Marks areas of heap so they can be reused by your program
• Scavenging
• Determines when parts of the heap are idle for a long time
• Returns memory to the operation system
4. www.cloudflare.com!
Garbage collection
• Controlled by the GOGC environment variable
• And by debug.SetGCPercentage()!
• Default is same as GOGC=100!
• Can set GOGC=off or debug.SetGCPercentage(-1) (no
garbage collection at all)
// Initialized from $GOGC. GOGC=off means no gc.!
//!
// Next gc is after we've allocated an extra amount of!
// memory proportional to the amount already in use.!
// If gcpercent=100 and we're using 4M, we'll gc again!
// when we get to 8M. This keeps the gc cost in linear!
// proportion to the allocation cost. Adjusting gcpercent!
// just changes the linear constant (and also the amount of!
// extra memory used).!
5. www.cloudflare.com!
Scavenging
• Runs once per minute
• Can also force return of all unused memory by calling
debug.FreeOSMemory()!
// If we go two minutes without a garbage collection, !
// force one to run.!
forcegc = 2*60*1e9;!
!
// If a span goes unused for 5 minutes after a garbage!
// collection, we hand it back to the operating system.!
limit = 5*60*1e9;!
6. www.cloudflare.com!
Memory Statistics
• Read with runtime.ReadMemStats(&m)
!
• The MemStats struct has tons of members
• Useful ones for looking at heap
• HeapInuse - # bytes in the heap allocated to things
• HeapIdle - # bytes in heap waiting to be used
• HeapSys - # bytes obtained from OS
• HeapReleased - # bytes released to OS
7. www.cloudflare.com!
Test garbage making program
func makeBuffer() []byte { !
return make([]byte, rand.Intn(5000000)+5000000) !
}!
!
func main() { !
pool := make([][]byte, 20)!
!
makes := 0 !
for { !
b := makeBuffer()
makes += 1!
!
i := rand.Intn(len(pool))!
pool[i] = b!
!
time.Sleep(time.Second)!
}!
}!
10. www.cloudflare.com!
Use a buffered channel
func main() {!
pool := make([][]byte, 20)!
idle:= make(chan []byte, 5)!
!
makes := 0!
for {!
var b []byte!
select {!
case b = <-idle:!
default:!
makes += 1!
b = makeBuffer()!
}!
!
!
i := rand.Intn(len(pool))!
if pool[i] != nil {!
select {!
case idle<- pool[i]:!
pool[i] = nil!
default:!
}!
}!
!
pool[i] = b!
!
time.Sleep(time.Second)!
}!
}
11. www.cloudflare.com!
select for non-blocking receive
idle:= make(chan []byte, 5)!
!
select {!
case b = <-idle:
!
default:!
makes += 1!
b = makeBuffer()!
}!
Try to get from the
idle queue
Idle queue
empty? Make a
new buffer
A buffered
channel makes a
simple queue
12. www.cloudflare.com!
select for non-blocking send
idle:= make(chan []byte, 5)!
!
select {!
case buffer <- pool[i]:!
pool[i] = nil!
!
default:!
}!
A buffered
channel makes a
simple queue
Try to return buffer
to the idle queue
Idle queue full?
GC will have to
deal with the
buffer
18. www.cloudflare.com!
More realistic example
• Alter code to
• Always try to give back a random buffer from the pool
• 50% of the time get a new one
• Should create more garbage
22. www.cloudflare.com!
Also
• This works for things other than []byte
• Can be done with arbitrary types
• Just need some way to reset
• There’s a proposal to add something like this to the Go
package library
• sync.Cache
• Follow TODO
