C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukić

Async. programming with ranges
Corehard Autumn 2018
dr Ivan Čukić
ivan@cukic.co core-dev @ KDE
http://cukic.co www.kde.org

Iterators Ranges Push Pipes Going postal
About me
KDE development
Talks and teaching
Author of ”Functional Programming in C++” (Manning)
Functional programming enthusiast, but not a purist
2

Disclaimer
Make your code readable. Pretend the next person
who looks at your code is a psychopath and they
know where you live.
Philip Wadler
3

Iterators
At the core of generic programming in STL.
5

Iterators
std::vector<std::string> identifiers;
std::copy_if(
std::cbegin(tokens),
std::cend(tokens),
std::back_inserter(identifiers),
is_valid_identifier);
6

Iterators
std::vector<std::string> identifiers;
std::copy_if(
std::istream_iterator<std::string>(std::cin),
std::istream_iterator<std::string>(),
std::back_inserter(identifiers),
is_valid_identifier);
7

Example
8

Example
template <typename It, typename Pred>
std::pair<It, It> move_selected(
It begin, It end, It destination,
Pred is_selected)
{
return {
std::stable_partition(begin, destination,
std::not_fn(is_selected)),
std::stable_partition(destination, end,
is_selected)
};
}
9

Problems with iterators
Example: Sum of squares
This should be separable into two subtasks:
squaring the values
summation
11

std::vector<int> xs { ⋯ };
std::vector<int> squared(xs.size());
std::transform(std::cbegin(xs), std::cend(xs),
std::begin(squared),
[] (int x) { return x * x; });
return std::accumulate(std::cbegin(squared),
std::cend(squared),
0);
12

return std::transform_reduce(par,
std::cbegin(xs), std::cend(xs),
std::cbegin(xs),
0);
13

Proxy iterators
template <typename Trafo, typename It>
class transform_proxy_iterator {
public:
auto operator*() const
{
return transformation(
*real_iterator);
}
private:
Trafo transformation;
It real_iterator;
};
14

Ranges
[ iterator, sentinel )
Iterator:
*i – access the value
++i – move to the next element
Sentinel:
i == s – has iterator reached the end?
15

Ranges
accumulate(
transform(
xs,
[] (int x) { ⋯ }),
0);
16

Ranges
accumulate(
transform(
filter(
xs,
[] (int x) { ⋯ }),
[] (int x) { ⋯ }),
0);
17

Ranges
accumulate(
xs | filter([] (int x) { ⋯ })
| transform([] (int x) { ⋯ }),
0);
18

Word frequency
1986: Donald Knuth was asked to implement a program for
the ”Programming pearls” column in the Communications
of ACM journal.
The task: Read a file of text, determine the n most
frequently used words, and print out a sorted list of those
words along with their frequencies.
19

Word frequency
1986: Donald Knuth was asked to implement a program for
the ”Programming pearls” column in the Communications
of ACM journal.
The task: Read a file of text, determine the n most
frequently used words, and print out a sorted list of those
words along with their frequencies.
His solution written in Pascal was 10 pages long.
19

Word frequency
Response by Doug McIlroy was a 6-line shell script that did
the same:
tr −cs A−Za−z ' n ' |
tr A−Z a−z |
sort |
uniq −c |
sort −rn |
sed ${1}q
20

Functional thinking – data transformation
21

With ranges
const auto words =
// Range of words
istream_range<std::string>(std::cin)
// Cleaning the input
| view::transform(string_to_lower)
| view::transform(string_only_alnum)
| view::remove_if(&std::string::empty)
// Sort the result
| to_vector | action::sort;
22

With ranges
const auto results =
words
// Group repeated words
| view::group_by(std::equal_to<>())
// Count how many repetitions we have
| view::transform([] (const auto &group) {
return std::make_pair(
distance(begin(group), end(group),
*begin(group));
})
// Sort by frequency
| to_vector | action::sort;
23

With ranges
for (auto value:
results | view::reverse // descending
| view::take(n) // we need first n results
) {
std::cout << value.first << ” ”
<< value.second << std::endl;
}
24

Ranges
Iterator:
*i – access the value
Sentinel:
i == s – has iterator reached the end
26

Ranges
Iterator:
*i – access the value BLOCKING!
Sentinel:
i == s – has iterator reached the end
26

Push iterators
Input Iterator Output
27

Push iterators
Each push iterator can:
Accept values
Emit values
No need for the accepted and emitted values to be 1-to-1.
28

Push iterators
Sources – push iterators that only emit values
Sinks – push iterators that only accept values
Modifiers – push iterators that both accept and emit
values
29

Continuation
template <typename Cont>
class continuator_base {
public:
void init() { ⋯ }
template <typename T>
void emit(T&& value) const
{
std::invoke(m_continuation, FWD(value));
}
void notify_ended() const { ⋯ }
protected:
Cont m_continuation;
};
30

Invoke
std::invoke(function, arg1, arg2, ...)
For most cases (functions, function objects, lambdas)
equivalent to:
function(arg1, arg2, ...)
But it can also invoke class member functions:
arg1.function(arg2, ...)
31

Source
class values_node: public continuator_base<Cont>, non_copyable {
⋯
void init()
{
base::init();
for (auto&& value: m_values) {
base::emit(value);
}
m_values.clear();
base::notify_ended();
}
⋯
};
32

Source
33

Creating a source
As far as the rest of the system is concerned, this node
creates the messages.
34

Creating a sink
35

Creating a transformation
36

Creating a transformation
class transform_node: public continuator_base<Cont>, non_copyable {
public:
⋯
void operator() (T&& value) const
{
base::emit(std::invoke(m_transformation, FWD(value)));
}
private:
Traf m_transformation;
};
37

Filtering
class filter_node: public continuator_base<Cont>, non_copyable {
public:
⋯
void operator() (T&& value) const
{
if (std::invoke(m_predicate, value) {
base::emit(FWD(value)));
}
}
private:
Predicate m_predicate;
};
38

Connecting the components
auto pipeline = source | transform(⋯)
| filter(⋯)
| join
| transform(⋯)
⋯
40

AST
When the pipeline AST is completed and evaluated, then we
get proper pipeline nodes.
41

Associativity
auto my_transformation = filter(⋯)
| join;
auto pipeline = source | transform(⋯)
| my_transformation
| transform(⋯)
⋯
42

Different pipes
Composing two transformations
transform(⋯) | filter(⋯)
Connecting a source to the continuation
source | transform(⋯)
Ending the pipeline
filter(⋯) | sink
Closing the pipeline
source | sink
43

Different pipes
struct source_node_tag {};
struct sink_node_tag {};
struct transformation_node_tag {};
class values {
public:
using node_category = source_node_tag;
};
44

Different pipes
struct source_node_tag {};
struct sink_node_tag {};
struct transformation_node_tag {};
class filter {
public:
using node_category = transformation_node_tag;
};
45

Concepts
template <typename Left, typename Right>
auto operator| (Left&& left, Right&& right)
{
return ⋯;
}
Problem: Defines operator| on all types
46

Concepts
template < typename Node
, typename Category = detected_t<node_category, Node>
>
f_concept is_node()
{
if constexpr (!is_detected_v<node_category, Node>) {
return false;
} else if constexpr (std::is_same_v<void, Category>) {
return false;
} else {
return true;
}
}
Note: Proper concepts will come in C++20. This uses the
detection idiom to simulate concepts.
47

Concepts
template < typename Left
, typename Right
, f_require(
is_node<Left> && is_node<Right>
)
>
auto operator| (Left&& left, Right&& right)
{
return ⋯;
}
48

Concepts
if constexpr (!is_source<Left> && !is_sink<Right>) {
⋯ create a composite transformation
} else if constexpr (is_source<Left> && !is_sink<Right>) {
⋯
} else if constexpr (!is_source<Left> && is_sink<Right>) {
⋯
} else {
⋯
}
49

Concepts
⋯
⋯ transformed source is also a source
⋯
} else {
⋯
}
49

Concepts
⋯
⋯
⋯ transformation with a sink is a sink
} else {
⋯
}
49

Concepts
⋯
⋯
⋯
} else {
⋯ the pipeline is complete, evaluate it
}
49

Pipes
Easy implementation of reactive systems
Transparent handling of synchronous vs asynchronous
transformations
High-enough abstraction for more powerful idioms
51

Functional thinking – data transformation
53

Distributed pipelines
auto pipeline =
system_cmd(”ping”s, ”localhost”s)
| transform(string_to_upper)
// Parse the ping output
| transform([] (std::string&& value) {
const auto pos = value.find_last_of('=');
return std::make_pair(std::move(value), pos);
})
// Extract the ping time from the output
| transform([] (std::pair<std::string, size_t>&& pair) {
auto [ value, pos ] = pair;
return pos == std::string::npos
? std::move(value)
: std::string(value.cbegin() + pos + 1, value.cend());
})
// Remove slow pings
| filter([] (const std::string& value) {
return value < ”0.145”s;
})
// Print out the ping info
| sink{cout};
54

Distributed pipelines
auto pipeline =
system_cmd(”ping”s, ”localhost”s)
| transform(string_to_upper)
| voy_bridge(frontend_to_backend_1)
| transform([] (std::string&& value) {
const auto pos = value.find_last_of('=');
return std::make_pair(std::move(value), pos);
})
| transform([] (std::pair<std::string, size_t>&& pair) {
auto [ value, pos ] = pair;
return pos == std::string::npos
? std::move(value)
: std::string(value.cbegin() + pos + 1, value.cend());
})
| voy_bridge(backend_1_to_backend_2)
| filter([] (const std::string& value) {
return value < ”0.145”s;
})
| voy_bridge(backend_2_to_frontend)
| sink{cout};
55

Live demo
56

Answers? Questions! Questions? Answers!
Kudos (in chronological order):
Friends at KDE
Saša Malkov and Zoltán Porkoláb
спасибо Сергею и двум Антонам
MEAP – Manning Early Access Program
Functional Programming in C++
cukic.co/to/fp-in-cpp
57

C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukić

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukić

Similar to C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukić (20)

More from corehard_by

More from corehard_by (20)

Recently uploaded

Recently uploaded (20)

C++ CoreHard Autumn 2018. Asynchronous programming with ranges - Ivan Čukić