Streamly: Concurrent Data Flow Programming

© 2019 Composewell Technologies
Streamly:
Concurrent Data Flow
Programming
Harendra Kumar
15 Nov 2019
Composewell Technologies

Ergonomics of Shell
Safety of Haskell
Speed of C
Magical Concurrency
About Streamly
https://github.com/composewell/streamly

About Me
• C programming
• OS kernel, ﬁle systems
• Haskell since 2015
harendra@composewell.com
@hk_hooda

About Composewell
• Develops Streamly
• Provides commercial support
• Designs solutions using Streamly
• Provides training on Streamly
http://www.composewell.com
hello@composewell.com

“Simplicity is a great virtue but it
requires hard work to achieve it
and education to appreciate it. And
to make matters worse: complexity
sells better.”
— Edsger Wybe Dijkstra

Streams

What is a stream?
• A sequence of same type of items
• Combinators to process the sequence

Pure Effectful
List
Stream
(Effectful List)

Imperative Functional
Loop
Stream
(Modular Loops)

Do I need streams?
• Imperative version: Do I need loops?

Streamly
=
Efﬁcient, Composable and
Concurrent Loops

Who should use Streamly?
• General purpose framework
• Declarative concurrency
• High performance, nearing or beating C
• Some examples:
• Server backends
• Reactive programming
• Real time data analysis
• Streaming data applications

Streamly At a Glance

Engineering Focused Library
(Goals)
Ergonomics Python and Shell
Performance C
Composability Haskell
Safety Haskell
Simplicity Use only basic Haskell

Composability
+
Performance

Streamly Version
• Examples in this presentation use
streamly-0.7.0
• Some examples may use APIs from
Streamly.Internal.* modules
• Source code for examples can be found at:
https://github.com/composewell/streamly-examples

Fundamental Operations
(Conceptual)
Operation Shape
Generate a -> Stream m b
Transform Stream m a -> Stream m b
Eliminate Stream m a -> m b

Fundamental Data Types
(Actual)
Type Constructor
Unfold m a b forall s. Unfold (s -> m (Step s b)) (a -> m s)
state generate inject
Stream m a forall s. Stream (s -> m (Step s a)) s
state generate state
Fold m a b forall s. Fold (s -> a -> m s) (m s) (s -> m b)
state accumulate initial extract

Core Modules
Type Module Abbrev.
Unfold Streamly.Data.Unfold UF
Stream Streamly.Prelude S
Fold Streamly.Data.Fold FL

unfold and fold Functions
• S.unfold :: Unfold m a b -> a -> Stream m b
• S.fold :: Fold m b c -> Stream m b -> m c

unfold & fold
fold
S.unfold UF.fromList [1..10] & S.fold FL.sum
unfold

unfold & fold = Action
fold
a -> m c
unfolda m cStream m b

unfold & fold = Loop!
a -> m c
a m cStream m b

Stream Types
Type Coercion Modiﬁer
SerialT m a serially
AsyncT m a asyncly
AheadT m a aheadly

Summing an Int Stream
S.unfold UF.fromList [1..10] -- SerialT Identity Int
& S.fold FL.sum -- Identity Int

Summing an Int Stream
(Better Ergonomics)
S.fromList [1..10] -- SerialT Identity Int
& S.sum -- Identity Int
fromList = S.unfold UF.fromList [1..10]
sum = S.fold FL.sum

File IO Examples

IO Modules
Module Abbrev.
Streamly.FileSystem.Handle FH
Streamly.FileSystem.File File
Streamly.Network.Socket SK
Streamly.Network.Inet.TCP TCP
Streamly.Data.Unicode.Stream U

cat
File.toChunks “inFile” -- SerialT IO (Array Word8)
& FH.putChunks -- IO ()

File Copy (cp)
File.toChunks “inFile” -- SerialT IO (Array Word8)
& File.fromChunks “outFile” -- IO ()

Count bytes in a File
(wc -c)
File.toBytes “inFile” -- SerialT IO Word8
& S.fold FL.length -- IO Int

Count Lines in a File
(wc -l)
& S.fold nlines -- IO Int
countl :: Int -> Word8 -> Int
countl n ch = if (ch == 10) then n + 1 else
n
nlines :: Monad m => Fold m Word8 Int
nlines = FL.mkPure countl 0 id

Count Words in a File
(wc -w)
& S.fold nwords -- IO Int
countw :: (Int, Bool) -> Word8 -> (Int, Bool)
countw (n, wasSpace) ch =
if (isSpace $ chr $ fromIntegral ch)
then (n, True)
else (if wasSpace then n + 1 else n, False)
nwords :: Monad m => Fold m Word8 Int
nwords = FL.mkPure countw (0, True) fst

Splitting a Stream
(Composing Folds)

Count Bytes, Lines & Words
(wc -clw)
& S.fold ((,,) <$> nlines <*> nwords <*> FL.length)
-- IO (Int, Int, Int)

Sending output to multiple
ﬁles (tee)
& S.fold (FL.tee (File.write “outFile1”)
(File.write “outFile2"))
-- IO ((),())

Splitting a ﬁle
(split)
type SIO = StateT (Maybe (Handle, Int)) IO
splitFile :: FH.Handle -> IO ()
splitFile inHandle =
& S.liftInner -- SerialT SIO Word8
& S.chunksOf2 64 newHandle FH.write2 -- SerialT SIO Word8
& S.evalStateT Nothing -- SerialT IO ()
& S.drain -- IO ()
Uses experimental APIs

Transformation
Pipeline

Word Classiﬁer
& S.decodeLatin1 -- SerialT IO Char
& S.map toLower -- SerialT IO Char
& S.words FL.toList -- SerialT IO String
& S.filter (all isAlpha) -- SerialT IO String
& toHashMap -- IO (Map String (IORef Int))
See https://github.com/composewell/streamly/blob/master/examples/WordClassiﬁer.hs
Adapted from an example by Patrick Thomson
alter Nothing = fmap Just $ newIORef (1 :: Int)
alter (Just ref) = do
modifyIORef' ref (+ 1)
return (Just ref)
toHashMap = S.foldlM' (flip (Map.alterF alter)) Map.empty

Word Size Histogram
bucket :: Int -> (Int, Int)
bucket n = let i = n `mod` 10
in if i > 9 then (9,n) else (i,n)
& S.words FL.length -- SerialT IO Int
& S.map bucket -- SerialT IO (Int, Int)
& S.fold (FL.classify FL.length) -- IO (Map Int Int)
classify directs (k,v) stream to a Map applying the length fold to the stream of values in each bucket

Debugging a Pipeline
(trace/tap)
& S.words FL.length -- SerialT IO Int
& S.map bucket -- SerialT IO (Int, Int)
& S.trace print -- SerialT IO (Int, Int)
& S.fold (FL.classify FL.length) -- IO (Map Int Int)
classify directs (k,v) stream to a Map applying the length fold to the stream of values in each bucket

Combining Streams
(Composing Unfolds)

Appending N Streams
(cat dir/* > outfile)
Dir.toFiles dirname -- SerialT IO String
& S.concatUnfold File.read -- SerialT IO Word8
& File.fromBytes “outFile” -- IO()

Outer Product
(Nested Loops)
mult :: (Int, Int) -> Int
mult (x, y) = x * y
from :: Monad m => Unfold m Int Int
from = UF.enumerateFromToIntegral 1000
cross :: Monad m => Unfold m (Int, Int) Int
cross =
UF.outerProduct from from
& UF.map mult
UF.fold cross FL.sum (1,1)

Better Replacement for ListT
and LogicT
loops :: SerialT IO ()
loops = do
x <- S.fromList [1,2]
y <- S.fromList [3,4]
S.yieldM $ putStrLn $ show (x, y)
(1,3)
(1,4)
(2,3)
(2,4)

Declarative
Concurrency

Lookup words
get :: String -> IO String
get s = liftIO (httpNoBody (parseRequest_ s)) >> return s
fetch :: String -> IO (String, String)
fetch w =
(,) <$> pure w <*> get (“https://www.google.com/search?q=“ ++ w)
wordList :: [String]
wordList = [“cat”, “dog”, “mouse”]
meanings :: [IO (String, String)]
meanings = map fetch wordList

Serially
S.fromListM meanings -- SerialT IO (String, String)
& S.map show -- SerialT IO String
& FH.putStrings -- IO ()

Asynchronously
S.fromListM meanings -- AsyncT IO (String, String)
& asyncly -- SerialT IO (String, String)
& FH.putStrings — IO ()

Speculatively
(Look Ahead)
S.fromListM meanings -- AheadT IO (String, String)
& aheadly -- SerialT IO (String, String)
& FH.putStrings -- IO ()

Word Lookup Server
S.unfold TCP.acceptOnPort 8090 -- SerialT IO Socket
& S.serially -- AsyncT IO ()
& S.mapM serve -- AsyncT IO ()
& S.asyncly -- SerialT IO ()
& S.drain -- IO ()
lookupWords :: Socket -> IO ()
lookupWords sk =
S.unfold SK.read sk -- SerialT IO Word8
& U.decodeLatin1 -- SerialT IO Char
& U.words FL.toList -- SerialT IO String
& S.serially -- AheadT IO String
& S.mapM fetch -- AheadT IO (String, String)
& S.aheadly -- SerialT IO (String, String)
& S.intercalateSuffix "n" UF.identity -- SerialT IO String
& S.fold (SK.writeStrings sk) — IO ()
serve :: Socket -> IO ()
serve sk = finally (lookupWords sk) (close sk)

Rate Control Req/Sec
lookupWords :: Socket -> IO ()
lookupWords sk =
& serially -- AheadT IO String
& S.mapM lookup -- AheadT IO (String, String)
& S.maxRate 10 -- AheadT IO (String, String)
& S.aheadly -- SerialT IO (String, String)
& S.map Show -- SerialT IO String
& S.fold Sk.write sk -- IO ()

Rate Control Conns/Sec
& S.serially -- AsyncT IO ()
& S.mapM serve -- AsyncT IO ()
& S.maxRate 10 -- AsyncT IO ()
& S.asyncly -- SerialT IO ()
& S.drain -- IO ()

Merging Live Word Streams
& S.concatMapWith S.parallel recv -- SerialT IO String
& U.unwords UF.fromList -- SerialT IO Char
& U.encodeLatin1 -- SerialT IO Word8
& File.fromBytes “outFile” -- IO ()
readWords :: Socket -> SerialT IO String
readWords sk =
recv :: Socket -> SerialT IO String
recv sk = S.finally (liftIO $ close sk) (readWords sk)

Recursive Directory Listing
Concurrently
listDir :: Either String String -> AheadT IO String
listDir (Left dir) =
Dir.toEither dir -- SerialT IO (Either String String)
& S.map (prefixDir dir) -- SerialT IO (Either String String)
& S.consM (return dir)
. S.concatMapWith ahead listDir -- SerialT IO String
listDir (Right file) = S.yield file -- SerialT IO String
S.mapM_ print $ aheadly $ listDir (Left ".")

Demand Scaled
Concurrency
• No threads if no one is consuming the stream
• Concurrency increases as consuming rate
increases
• maxThreads and maxBuffer can control the
limits

Concurrent Folds
(Consume Concurrently)

Write Concurrently to multiple Destinations
FH.getBytes -- SerialT IO Word8
& S.tapAsync (TCP.fromBytes (192,168,1,10) 8091) -- SerialT IO Word8
& S.tapAsync (TCP.fromBytes (192,168,1,11) 8091) -- SerialT IO Word8
& File.fromBytes “outFile” -- IO ()

Concurrent ListT
(Nested Loops)

Non-determinism
(Looping)
loops = $ do
x <- each [1,2]
y <- each [3,4]
liftIO $ putStrLn $ show (x, y)
main = S.drain $ serially $ loops
main = S.drain $ asyncly $ loops
main = S.drain $ aheadly $ loops

Streaming + Concurrency
=
Reactive Programming

Reactive Programming
• Reactive programs (games, GUI) can be elegantly
expressed by declarative concurrency.
• See the Acid Rain game example in the package
• See the Circling Square example from Yampa, in
the package
https://github.com/composewell/streamly/blob/master/examples/AcidRain.hs
https://github.com/composewell/streamly/blob/master/examples/CirclingSquare.hs

Performance

Micro Benchmarks (GHC 8.8.1)
• A stream of 1 million elements is generated
• unfoldrM is used to generate the stream
• Two types of operations on the stream are
measured:
• single operation applied once
• a mix of operations applied multiple times
• Compiled using GHC 8.8.1
• All benchmarks are single threaded
• Ran on MacBook Pro with Intel Core i7 processor
• Because there is a lot of variance, the comparison
is in multiples rather than as percentage diff.

Comparison with Haskell lists
(GHC-8.8.1) (time)

Comparison with lists (GHC-8.8.1)
(Micro Benchmarks)
• List is slower than streamly in most operations, the
worse is 150 times slow.
• Streamly is slower than lists for concatMap and
append operations.
• There is no signiﬁcant difference in memory
consumption.

Comparison with streaming libraries
(time)
streaming-0.2.3.0
conduit-1.3.1.1
pipes-4.3.12

(memory)
streaming-0.2.3.0
conduit-1.3.1.1
pipes-4.3.12

• All libraries are signiﬁcantly slower (ranging from
1.2x to 1100x) than streamly for all operations.
• Streaming and streamly both consistently utilize
the same amount of memory across all ops.
• Conduit and pipes spike up to 32x memory in
certain operations.

Comparison With C

Counting Words in C
(The State)
struct statfs fsb;
uintmax_t linect, wordct, charct;
int fd, len;
short gotsp;
uint8_t *p;
uint8_t *buf;
linect = wordct = charct = 0;
if ((fd = open(argv[1], O_RDONLY, 0)) < 0) {
perror("open");
exit(EXIT_FAILURE);
}
if (fstatfs(fd, &fsb)) {
perror("fstatfs");
exit(EXIT_FAILURE);
}
buf = malloc(fsb.f_bsize);
if (!buf) {
perror("malloc");
exit(EXIT_FAILURE);
}
gotsp = 1;

Counting Words in C
(The Logic)
while ((len = read(fd, buf, fsb.f_bsize)) != 0) {
if (len == -1) {
perror("read");
exit(EXIT_FAILURE);
}
p = buf;
…
}
while (len > 0) {
uint8_t ch = *p;
charct++;
len -= 1;
p += 1;
if (ch == 'n')
++linect;
if (isspace(ch))
gotsp = 1;
else if (gotsp) {
gotsp = 0;
++wordct;
}
}
}

Counting Words in Haskell
data WordCount = WordCount !Int !Bool
data Counts = Counts !Int !Int !WordCount
initialCounts = Counts 0 0 (WordCount 0 True)
countl :: Int -> Word8 -> Int
countl n ch = if (ch == 10) then n + 1 else n
countw :: WordCount -> Word8 -> WordCount
countw (WordCount n wasSpace) ch =
if (isSpace $ chr $ fromIntegral ch)
then WordCount n True
else WordCount (if wasSpace then n + 1 else n) False
{-# INLINE updateCounts #-}
updateCounts :: Counts -> Word8 -> Counts
updateCounts (Counts c l w) ch = Counts (c + 1) (countl l ch) (countw w ch)
wc :: Handle -> IO Counts
wc h =
S.unfold FH.read h -- SerialT IO Char
& S.foldl' updateCounts initialCounts -- IO Counts

Word Counting: C vs Haskell
(550 MB text ﬁle)
C Haskell
2.42 Second 2.17 Second

Can Haskell be as fast as C?
• Each Haskell combinator represents a small
piece of the loop
• The programmer composes the loop using
these combinators.
• GHC fuses these pieces together (stream
fusion) to create a monolithic loop like C.
• Finally, the structure of the optimized code
churned out by GHC is like the C code, with the
both the loops as you see in the C program.
• This is global program optimization, an efﬁcient
big picture is created using smaller pieces.
• GCC can only perform low level optimization

How can GHC perform
global optimizations?
• Strong types and purity makes equational
reasoning possible.
• This allows GHC to reliably perform
transformations over the code and fuse parts of
the code to generate efﬁcient code.
• Global program optimization is not possible in
C.

What are the downsides?
• Stream fusion depends on inlining and SPEC
constructor optimizations.
• Often careful INLINE annotations are needed.
• Higher order functions require INLINE in the
“right” phase.
• GHC may need more work to perform it fully
reliably
• Due to global optimization compilation is slower
and may get slower as the size of the program
increases.

Streamly GHC Plugin For
Fusion
• Internal join bindings beyond a size threshold
are not inlined, blocking fusion.
• We mark stream constructors with a pragma to
identify them in the core.
• Join points with such constructors are inlined
irrespective of the size.
• This allows more reliable fusion.

The Project

Current State of The Project
• ~25K LOC, ~16K Doc, ~95 ﬁles, 18 contributors
• High quality, tested, production capable
• Some parts of the API may change in near future
• type names may change
• module structure may change

Work In Progress
• Stream parsers
• Concurrent folds
• Splitting and merging transformations

Roadmap
• Shared concurrent state
• Persistent queues
• Vector instructions
• Distributed processing
• Lot more stuff

Thank You!
harendra@composewell.com
twitter: @hk_hooda
https://github.com/composewell/streamly
https://gitter.im/composewell/streamly

Streamly: Concurrent Data Flow Programming

More Related Content

What's hot

Similar to Streamly: Concurrent Data Flow Programming

Recently uploaded

Streamly: Concurrent Data Flow Programming