The document discusses Guild, a new concurrency abstraction for Ruby that allows parallel execution across Guilds while avoiding data races. It provides demonstrations of Guild performance on a 40 vCPU system. Guilds allow sharing immutable and specially defined mutable objects between Guilds, while normal mutable objects cannot be shared to avoid data races. Inter-Guild communication uses an actor model with sending/receiving semantics. Non-shareable objects can be sent between Guilds either by copying or moving ownership.
Introduction to Kotlin Language and its application to Android platformEastBanc Tachnologies
Author: Oleg Godovykh, eastbanctech.com
Kotlin is a new programming language built by Jetbrains and is a new member of JVM family. As opposed to typical reason to introduce some new language, Kotlin main goal isn't to create new paradigm or fill a new niche, but to make routine tasks much easier and safer. Kotlin gains popularity across Android developer community, and in this presentation it is shown how Kotlin usage can dramatically simplify typical mobile app development.
Go for Object Oriented Programmers or Object Oriented Programming without Obj...Steven Francia
Object Oriented (OO) programming has dominated software engineering for the last two decades. The paradigm built on powerful concepts such as Encapsulation, Inheritance, and Polymoprhism has been internalized by the majority of software engineers. Although Go is not OO in the strict sense, we can continue to leverage the skills we’ve honed as OO engineers to come up with simple and solid designs.
Gopher Steve Francia, Author of
[Hugo](http://hugo.spf13.com), [Cobra](http://github.com/spf13/cobra), and many
other popular Go packages makes these difficult concepts accessible for everyone.
If you’re a OO programmer, especially one with a background with dynamic languages and are curious about go then this talk is for you. We will cover everything you need to know to leverage your existing skills and quickly start coding in go including:
How to use our Object Oriented programming fundamentals in go
Static and pseudo dynamic typing in go
Building fluent interfaces in go
Using go interfaces and duck typing to simplify architecture
Common mistakes made by those coming to go from other OO languages (Ruby, Python, Javascript, etc.),
Principles of good design in go.
JRuby 9000 introduced a new intermediate representation that allows us to use classic compiler strategies to optimize Ruby. This talk describes what we're doing with this new IR and why current JVM capabilities are not sufficient.
A teaser talk for Scala newbies, introducing five basic elements that (in my opinion) make the transition from Java to Scala a no-brainer.
Given at the 7th JJTV (Israeli Java/JVM user group) tool night, July 2nd, 2013.
The full video from CppCon is available here: https://www.youtube.com/watch?v=PVYdHDm0q6Y
C++ provides a much richer set of abstractions than C. Classes, templates, overloading, and other core C++ features can be leveraged for more readable syntax, better compile time typechecking, more open ended genericity and improved modularity. On the flip side, C89 still boasts some advantages over C++, especially when viewed through a pragmatic lens. C ABIs on many platforms have been stable for decades, practically every language supports binding to C code through foreign function interfaces, and including nearly any C89 header has a negligible effect on compile time on modern computers.
The hourglass pattern provides the best of both worlds. By placing a headers-only C++ interface on top of a minimal C89 interface, it makes ABI issues a non-concern, and enables just a single binary to be shipped for a given platform. It makes providing bindings from other languages easier, and prevents ABI issues like incompatibilities between debug and release variants of runtimes.
This talk provides an overview of the pattern and teaches practical techniques for its implementation using C++14. Real shipping projects inspired the best practices described in the slides.
Code corresponding to this presentation can be found here: https://github.com/CppCon/CppCon2014/tree/master/Presentations/Hourglass%20Interfaces%20for%20C%2B%2B%20APIs%20-%20Stefanus%20Du%20Toit%20-%20CppCon%202014/code
A presentation at Twitter's official developer conference, Chirp, about why we use the Scala programming language and how we build services in it. Provides a tour of a number of libraries and tools, both developed at Twitter and otherwise.
Introduction to Kotlin Language and its application to Android platformEastBanc Tachnologies
Author: Oleg Godovykh, eastbanctech.com
Kotlin is a new programming language built by Jetbrains and is a new member of JVM family. As opposed to typical reason to introduce some new language, Kotlin main goal isn't to create new paradigm or fill a new niche, but to make routine tasks much easier and safer. Kotlin gains popularity across Android developer community, and in this presentation it is shown how Kotlin usage can dramatically simplify typical mobile app development.
Go for Object Oriented Programmers or Object Oriented Programming without Obj...Steven Francia
Object Oriented (OO) programming has dominated software engineering for the last two decades. The paradigm built on powerful concepts such as Encapsulation, Inheritance, and Polymoprhism has been internalized by the majority of software engineers. Although Go is not OO in the strict sense, we can continue to leverage the skills we’ve honed as OO engineers to come up with simple and solid designs.
Gopher Steve Francia, Author of
[Hugo](http://hugo.spf13.com), [Cobra](http://github.com/spf13/cobra), and many
other popular Go packages makes these difficult concepts accessible for everyone.
If you’re a OO programmer, especially one with a background with dynamic languages and are curious about go then this talk is for you. We will cover everything you need to know to leverage your existing skills and quickly start coding in go including:
How to use our Object Oriented programming fundamentals in go
Static and pseudo dynamic typing in go
Building fluent interfaces in go
Using go interfaces and duck typing to simplify architecture
Common mistakes made by those coming to go from other OO languages (Ruby, Python, Javascript, etc.),
Principles of good design in go.
JRuby 9000 introduced a new intermediate representation that allows us to use classic compiler strategies to optimize Ruby. This talk describes what we're doing with this new IR and why current JVM capabilities are not sufficient.
A teaser talk for Scala newbies, introducing five basic elements that (in my opinion) make the transition from Java to Scala a no-brainer.
Given at the 7th JJTV (Israeli Java/JVM user group) tool night, July 2nd, 2013.
The full video from CppCon is available here: https://www.youtube.com/watch?v=PVYdHDm0q6Y
C++ provides a much richer set of abstractions than C. Classes, templates, overloading, and other core C++ features can be leveraged for more readable syntax, better compile time typechecking, more open ended genericity and improved modularity. On the flip side, C89 still boasts some advantages over C++, especially when viewed through a pragmatic lens. C ABIs on many platforms have been stable for decades, practically every language supports binding to C code through foreign function interfaces, and including nearly any C89 header has a negligible effect on compile time on modern computers.
The hourglass pattern provides the best of both worlds. By placing a headers-only C++ interface on top of a minimal C89 interface, it makes ABI issues a non-concern, and enables just a single binary to be shipped for a given platform. It makes providing bindings from other languages easier, and prevents ABI issues like incompatibilities between debug and release variants of runtimes.
This talk provides an overview of the pattern and teaches practical techniques for its implementation using C++14. Real shipping projects inspired the best practices described in the slides.
Code corresponding to this presentation can be found here: https://github.com/CppCon/CppCon2014/tree/master/Presentations/Hourglass%20Interfaces%20for%20C%2B%2B%20APIs%20-%20Stefanus%20Du%20Toit%20-%20CppCon%202014/code
A presentation at Twitter's official developer conference, Chirp, about why we use the Scala programming language and how we build services in it. Provides a tour of a number of libraries and tools, both developed at Twitter and otherwise.
Apache Storm 0.9 basic training - VerisignMichael Noll
Apache Storm 0.9 basic training (130 slides) covering:
1. Introducing Storm: history, Storm adoption in the industry, why Storm
2. Storm core concepts: topology, data model, spouts and bolts, groupings, parallelism
3. Operating Storm: architecture, hardware specs, deploying, monitoring
4. Developing Storm apps: Hello World, creating a bolt, creating a topology, running a topology, integrating Storm and Kafka, testing, data serialization in Storm, example apps, performance and scalability tuning
5. Playing with Storm using Wirbelsturm
Audience: developers, operations, architects
Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/
Verisign is a global leader in domain names and internet security.
Tools mentioned:
- Wirbelsturm (https://github.com/miguno/wirbelsturm)
- kafka-storm-starter (https://github.com/miguno/kafka-storm-starter)
Blog post at:
http://www.michael-noll.com/blog/2014/09/15/apache-storm-training-deck-and-tutorial/
Many thanks to the Twitter Engineering team (the creators of Storm) and the Apache Storm open source community!
Concurrency and Multithreading Demistified - Reversim Summit 2014Haim Yadid
Life as a software engineer is so exciting! Computing power continue to rise exponentially, software demands continue to rise exponentially as well, so far so good. The bad news are that in the last decade the computing power of single threaded application remains almost flat.
If you decide to continue ignoring concurrency and multi-threading the gap between the problems you are able to solve and your hardware capabilities will continue to rise. In this session we will discuss different approaches for taming the concurrency beast, such as shared mutability,shared immutability and isolated mutability actors, STM, etc we will discuss the shortcomings and the dangers of each approach and we will compare different programming languages and how they choose to tackle/ignore concurrency.
"WTF is Twisted? (or; owl amongst the ponies)" is a talk that introduces the Twisted asynchronous programming framework, how it works, and what uses it.
Apache Spark™ + IBM Watson + Twitter DataPalooza SF 2015Mike Broberg
Use Apache Spark Streaming in with IBM Watson on Bluemix to perform sentiment analysis and track how a conversation is trending on Twitter.
By David Taieb: https://twitter.com/DTAIEB55
Video: https://youtu.be/KLc_wazud3s
Tutorial: https://developer.ibm.com/clouddataservices/sentiment-analysis-of-twitter-hashtags/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
4. TL;DR
•We can run parallel programs with Guilds
•Current implementation is too preliminary
and so buggy (easy to halt)
→ “Thread programming” is too hard
for human beings.
7. Recent achievements for Ruby 2.6
•Speedup `Proc#call` … x1.4 improvements
[Bug #10212].
•Speedup `block.call` where `block` is
passed block parameter. [Feature #14330]
(x2.62).
8. Advertisement
Cookpad booth events
【Day 2】
15:20~15:50 Global Office Hours
【Day 3】
12:00~13:00 Q&A with @wyhaines
15:20~15:50 Ruby interpreter developme
live by @ko1 & @mame
Cookpad X RubyKaigi 2018: Day 2 Party
⏰ June 1st, 19:30 - 21:30 (opens 19:00)
💰 Free (Registration required)
Show up to this booth at 18:40 if you want to head with us!
10. Motivation
Productivity (most important for Ruby)
• Thread programming is too difficult
• Making correct/safe concurrent programs easily
Performance by Parallel execution
• Making parallel programs
• Threads can make concurrent programs, but can’t run them
in parallel on MRI (CRuby)
• People want to utilize Multi/many CPU cores
11. RubyKaigi2016 Proposal
Guild: new concurrency abstraction for Ruby 3
• Idea: DO NOT SHARE mutable objects between Guilds
→ No data races, no race conditions
Replace Threads to Guilds
12. Guilds, Threads and Fibers
• Guild has at least one thread (and a thread has at
least one fiber)
• Threads in different Guilds can run in parallel
Guild
Thread
Fiber
Guild
Thread
Fiber
Guild
Thread
Fiber
Fiber
Thread
Fiber
13. Design “Shareable” and “non-sharable”
• Non-shareable objects
• (normal) Mutable objects (String, Array, …)
• They are members of only one Guild
• Using only 1 Guild, it compatible with Ruby 2
Guild 1 Guild 2
obj
obj
obj
obj
obj
Can’t access
(read/write)
NG!!
15. Demonstration (on 40 vCPUs)
•CPU 40 virtual CPUs (2 x 10 x 2)
•x 2 CPUs
•Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
• x10 cores
• x2 hyper threading
•Ubuntu 16.10
•Already EOL
16. Demonstration (on 40 vCPUs)
•Workload
•Calculate fib(23) x 100_000 times
• Serial version: 100_000.times{ fib(23) }
• Guild version:
main
Guild
FibHub
Guild
fib
workerfib
worker
fib
worker
…load balancing
We can change
# of workers
17. FIBHUB = make_worker_hub do |n|
[n, fib(n)]
end
# library
def make_worker_hub n_workers = WN, &worker_proc
pp WN: n_workers if $VERBOSE
Guild.new(n_workers, worker_proc) do |nw, wp|
guilds = nw.times.map do
Guild.new do
while data = Guild.receive
result = wp.call(data)
Guild.parent << [:ans, Guild.current, result]
end
end
end
requests = []
while true
cmd, sender_guild, data = *Guild.receive
case cmd
when :req
if g = guilds.pop
g << data
else
requests << data
end
when :ans
Guild.parent << data
if req = requests.pop
sender_guild << req
else
guilds << sender_guild
end
end
end
end
end
# Make worker guilds # Receive an answers from workers
# Send a remaining task
# to the worker if exists
# Send a task
# if an idle worker is available
# Receive a request from master
# Send an answers to master
# You don’t need to write such common code
# but we provide some kind of a framework
https://gist.github.com/ko1/e5327126a77e078a0ffdf005013592ea
18. fib(23) with # of Guilds on 40 vCPUs
0
10
20
0 20 40 60 80 100
# of Guilds
Speedup ratio (compare with serial execution)
19. Demonstration (on 40 vCPUs)
•Workload
•Calculate fib(n) x 100_000 times (0 ≦n≦30)
• Serial version: 100_000.times{ fib(23) }
• Guild version: 40 Guilds
main
Guild
FibHub
Guild
fib
workerfib
worker
fib
worker
…load balancing
20. Execution time (sec) of fib(n) x 100_000
with 40 Guilds on 40 vCPUs
0
5000
10000
15000
0 5 10 15 20 25 30 35
sec
n for fib(n)
real (sec) real-serial (sec)
12,896.23 sec
=~ 3.5 hours
791.2541 sec
=~ 13 minutes
Order of fib(n) is “O(2^n)”
21. fib(n) with 40 Guilds on 40 vCPUs
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
0 5 10 15 20 25 30 35
Speedupratiocompare
withserialexecution
n for fib(n)
Slower Faster
fib(9): about 100 recursive calls
22. Demonstration (on 40 virtual CPU)
•Workload
•Calculate wordcount for files and find a file
which contains maximum number of words.
• on “ruby/test/**/*” files (1,108 files)
def word_count file
r = File.read(file).b.upcase.split(/¥W/).uniq.size
end
23. Demonstration (on 40 virtual CPU)
•Workload
•Calculate wordcount for files and find a file
which contains maximum number of words.
• on “ruby/test/**/*” files (1,108 files)
main
Guild
WCHub
Guild
wc
workerwc
worker
wc
worker
…load balancing
We can change
# of workers
traverse directories,
correct wc results,
and calculate maximum
24. Demonstration (on 40 virtual CPU)
1.736429 1.652815
2.473468
6.136364
0
5
10
Serial 1 Guold 2 Guild 40 Guilds
Executiontime
(sec)
It is SLOW with multiple Guilds
because GC/object allocation require naïve global locking
(current implementation limitation) and huge contentions.
26. Guilds, Threads and Fibers
•Guild has at least one thread (and a thread
has at least one fiber)
Guild
Thread
Fiber
Guild
Thread
Fiber
Guild
Thread
Fiber
Fiber
Thread
Fiber
27. Threads in different guilds
can run in PARALLEL
•Threads in different guilds can run in parallel
•Threads in a same guild can not run in parallel
because of GVL (or GGL: Giant Guild Lock)
G1:T1
G1:T2
G2:T3
Acquire GGL
Acquire GGL
28. Making Guilds
g1 = Guild.new do
expr1
end
g2 = Guild.new do
expr2
end
# Two new Guilds and Threads are created
# expr1 and expr2 can run in parallel
30. Design “Shareable” and “non-sharable”
•You can enjoy usual mutating
programming without any thread-safe
concerns.
•Because we can’t share mutable objects
between Guilds. They are “non-sharable”.
•In other words, you can’t make thread-
unsafe (data-racy) programs with Guilds.
31. Design “Shareable” and “non-sharable”
•You need to use “Sharable” objects to share
objects between Guilds
•On concurrent programs, most of objects are
not shared (thread-local)
•Tons of local objects and a few sharing objects
•You only need to care about a few objects
non-
sharable
non-
sharable
non-
sharable
non-
sharable sharable
sharable
non-
sharable
non-
sharable
non-
sharable
non-
sharable
non-
sharable
non-
sharable
non-
sharable
non-
sharablenon-
sharable
non-
sharable
non-
non-
sharable
32. Design “Shareable” and “non-sharable”
•Non-shareable objects
•(normal) Mutable objects (String, Array, …)
•They are member of only one Guild
•Using only 1 Guild, it compatible with Ruby 2
Guild 1 Guild 2
obj
obj
obj
obj
obj
Can’t access
(read/write)
NG!!
34. Shareable objects
(1) Immutable objects
• Immutable objects can be shared with any guilds
• Because no mutable operations for them
• “Immutable” != “Frozen”
• a1 = [1, 2, 3].freeze: a1 is Immutable
• a2 = [1, Object.new, 3].freeze: a2 is not Immutable
• Maybe we will introduce deep freeze feature
• Numeric objects, symbols, true, false, nil are immutable
(from Ruby 2.0, 2.1, 2.2)
• Frozen string objects are immutable (if they don’t have
instance variables)
35. Shareable objects
(2) Class/Module objects
•All objects (including any sharable objects) point
to own classes
• Good: Sharing class/module objects makes program
easier
• Bad: They can point to other mutable objects with
Constants, @@class_variable and @instance_variables
class C
Const = [1, 2, 3] # Const points a mutable
array
end
# We will introduce special protocol for them
36. Shareable objects
(3) Special mutable objects
• Introduce shared/concurrent data structure
• Shared hash, array, …
• Software transactional memory (from Clojure, …), …
• Guild objects and so on
• They require special process to force
synchronization explicitly
• They can’t mutate without synchronizations.
• Easy to make correct concurrent programs
• Compared with normal Array, Hash, … they should
require special synchronization protocol to access
37. Shareable objects
(4) Isolated Proc
•Normal Proc can point to mutable objects with
outer local variable (free-variables)
• a = []; Proc.new{p a}.call
•Introduce Isolated Proc (made by
Proc#isolate) which is prohibited to access
outer variables
a = []; Proc.new{p a}.isolate.call
#=> RuntimeError (can’t access a)
•(there are more details but skip)
38. Shareable objects
(4) Isolated Proc
# Initial block for Guild is isolated
proc
g1 = Guild.new do
expr1 # Make isolated block and invoke
end
g2 = Guild.new do
p g1 #=> RuntimeError (can’t access
“g1”)
# because block is isolated
end
39. Inter-Guild communication API
•Actor model, send/receive semantics
•Destination addresses are represented by
Guild itself like Erlang/Elixir processes
•Sending shareable objects means sending
only references to the objects (lightweight)
•Two methods to send non-shareable objects
• (1) COPY
• (2) MOVE
40. Sending objects between Guilds
g1 = Guild.new do # create Isolated Proc
n = Guild.receive
r = fib(n)
Guild.parent << r
end
g1 << 30
p Guild.receive #=> 1346269
44. Sending non-shareable objects
(2) Send by Move
Guild1 Guild2
channel
o2
o3
o1
MOVE
g2.move(o1) o1 = Guild.receive
O2:Data
O3:Data
-
-
-
From Guild1 perspective,
sent objects are invalidated
45. Sending non-shareable objects
(2) Send by Move
•If we don’t access sent objects after
sending them (and there are many such
cases), we can send them faster
•Examples
•Huge string data
•I/O operation (send request I/O to workers)
46. Summary of object sharing/non-sharing
• Shareable objects
• Several types of shareable objects
• We can share them between Guilds
• They requires special synchronization protocol to mutate them
• Non-sharable objects
• Normal mutable objects (like String, Array, …)
• Only one Guild can access such objects == membership
• We can send them by COPY or MOVE
• Mutable objects are NOT shared accidentally as Thread
programming → Correct concurrent Prog.
49. Introduce synchronizations
•Before Guild
•Multiple threads cannot run simultaneously
•After Guild
•Run (native) threads in parallel
•Need to introduce so many synchronizations
•Introduce VM-wide locks for VM-wide resources
•It is the multi-thread programming!!
50. Garbage collection
•Stop all Guilds (threads) at GC process
G1:T1
G2:T2
G3:T3
GC process
Pause
request
Restart
request
51. Future work and optimizations
•Features
•Prohibit sharing non-sharable objects
•Introduce synchronizations to protect VM-
wide resources (process-global)
•Introduce “sharable” object protocols
•Performance
•Reduce synchronizations
•Per Guild Garbage collection
•Introduce new “C API” to reduce TLS access
52. Future optimizations
• Koichi Sasada, et.al. : An
Implementation of Parallel
Threads for YARV: Yet Another
RubyVM (2007)
• They introduced several
optimization techniques to reduce
synchronizations
54. Why “Guild”?
•Prefix should be different from “P”
(Process), “T” (Therad) and “F” (Fiber).
•Ownership can be explained with the word
“Membership”.
•All (normal) objects belong to one Guild.
•Easy to explain “Move” semantics
55. Any problem?
•“Move” operation is not so popular operation
(most of case “copy” is enough)
•No other languages use this terminology
•Naming is important
•Just now “Guild” is a code name of this
project
58. Pros./Cons. Matrix
Process Guild Thread Auto-Fiber Fiber
Available Yes No Yes No Yes
Switch on
time
Yes Yes Yes No No
Switch on I/O Auto Auto Auto Auto No
Next target Auto Auto Auto Auto Specify
Parallel run Yes Yes No (on MRI) No No
Shared data N/A (mostly) N/A Everything Everything Everything
Comm. Hard Maybe Easy Easy Easy Easy
Programming
difficulty
Hard Easy Difficult Easy Easy
Debugging
difficulty
Easy? Maybe Easy Hard Maybe hard Easy
Editor's Notes
Today’s my talk is about the current prototype of Guild, a new concurrency abstraction for Ruby 3.
2 years ago, at RubyKaigi 2016, I talked this presentation, named “a proposal of new concurrency model for Ruby 3”.
At this presentation, I’ve introduced new concurrent abstractuion, named “Guild”.
This presentation only showed