The document discusses streams and functional programming in JavaScript. It provides three examples of using streams:
1. Playing MP3 files by piping a read stream through a decoder and speaker.
2. Creating a "sound soup" that randomly plays MP3 files from a directory every half second by piping read streams through a decoder and speaker.
3. A brief history of streams in Node.js, from the original Util.pump method to the current Streams API in versions 1-3, highlighting improvements in error handling and backpressure support.
Slides from a talk at HPC Admintech 2019 about containers, a brief review on containers, how to create a container using common linux tools and how to integrate Docker with Slurm.
This is a slightly updated draft of a talk I was planning on giving at Hadoop Summit in 2015. However the abstract was rejected. Rather than toss it, I'm going to share it with all of you on the (almost) 1 year anniversary of the first big commit of this feature!
Keep in mind that this is (currently) locked away in trunk. If you ever want to see this see the light of day, bug your vendors....
With JavaScript being no toy language anymore our demands on a high quality development environment have risen tremendously. One part of a good development environment are build tools, and Gulp.js -- being the JavaScript streaming build system -- is but one of the many choices a developer has nowadays. For many however, it's considered to be the best! In this three hour workshop, we will take a good look into Gulp and its possibilities. After our in-depth 3 hour course you will be able to do the following:
- Know how Gulp's API and the ecosystem of its plugins work
- Create parallel and sequential execution chains to be in total control of your build flow
- Know how 3rd party Node modules evolve around Gulp and how they can be integrated
- Create incremental builds to speed up your build time
- Know how streams work in Node.js
- And use stream arrays and merge streams to plumb together sophisticated pipelines doing all the work for you
Join us and become a build plumber!
More information
http://github.com/frontend-tooling
http://bit.ly/gulp-tool-book
http://fettblog.eu
Slides from a talk at HPC Admintech 2019 about containers, a brief review on containers, how to create a container using common linux tools and how to integrate Docker with Slurm.
This is a slightly updated draft of a talk I was planning on giving at Hadoop Summit in 2015. However the abstract was rejected. Rather than toss it, I'm going to share it with all of you on the (almost) 1 year anniversary of the first big commit of this feature!
Keep in mind that this is (currently) locked away in trunk. If you ever want to see this see the light of day, bug your vendors....
With JavaScript being no toy language anymore our demands on a high quality development environment have risen tremendously. One part of a good development environment are build tools, and Gulp.js -- being the JavaScript streaming build system -- is but one of the many choices a developer has nowadays. For many however, it's considered to be the best! In this three hour workshop, we will take a good look into Gulp and its possibilities. After our in-depth 3 hour course you will be able to do the following:
- Know how Gulp's API and the ecosystem of its plugins work
- Create parallel and sequential execution chains to be in total control of your build flow
- Know how 3rd party Node modules evolve around Gulp and how they can be integrated
- Create incremental builds to speed up your build time
- Know how streams work in Node.js
- And use stream arrays and merge streams to plumb together sophisticated pipelines doing all the work for you
Join us and become a build plumber!
More information
http://github.com/frontend-tooling
http://bit.ly/gulp-tool-book
http://fettblog.eu
It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.
But are build tools really the answer to everything?
In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.
The plumbing metaphor goes back 40 years to the beginning of Unix/Linux and still works today.
In this session the fundamentals of shell scripting will be illustrated through a cumulative example built on pipes, filters, valves, and screens. Environmentally friendly waste cleanup will also be covered.
A Percona Support Engineer Walkthrough on pt-stalkMarcelo Altmann
This presentation shows the approach we use to diagnostic performance issues on the customer. We explain the USE method and how you can relate it with pt-stalk
Redis & ZeroMQ: How to scale your applicationrjsmelo
Presented at #PHPLX 11 July 2013
When you need to do some heavy processing how do you scale you application?
You can use Redis and ZeroMQ to leverage the heavy work for you!
With this presentation we will know more about this two technologies and how they can be used to help solve problems with the performance and scalability of your application.
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club UkraineElixir Club
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga
Slides of Alexei Sholik, Andrew Dryga, at Lightning Talk session at Elixir Club Ukraine, Kyiv, 28.09.2019
Next conference - http://www.elixirkyiv.club/
Follow us on social networks @ElixirClubUA and #ElixirClubUA
Announce and materials from conf - https://www.fb.me/ElixirClubUA
News - https://twitter.com/ElixirClubUA
Photo and free atmosphere - https://www.instagram.com/ElixirClubUA
*Organizer’s channel - https://t.me/incredevly
AMS Node Meetup December presentation Phusion Passengericemobile
Phusion Passenger is an app server for Node.js, Ruby and Python. It simplifies deployment and administration, increases your server's efficiency and helps identifying and solving problems.
In this talk Hongli Lai demonstrates how Passenger simplifies things by integrating with Nginx and by replacing Forever, PM2, Cluster and all sorts of other tools. Hongli also shares what other benefits Passenger has to offer, and what you can expect from future developments.
Slides from my talk at Philly ETE looking at the Lambda Architecture (originating at twitter) critically from the perspective of someone viewing it from the financial (faster, higher volume, spikier data) domain
It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.
But are build tools really the answer to everything?
In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.
The plumbing metaphor goes back 40 years to the beginning of Unix/Linux and still works today.
In this session the fundamentals of shell scripting will be illustrated through a cumulative example built on pipes, filters, valves, and screens. Environmentally friendly waste cleanup will also be covered.
A Percona Support Engineer Walkthrough on pt-stalkMarcelo Altmann
This presentation shows the approach we use to diagnostic performance issues on the customer. We explain the USE method and how you can relate it with pt-stalk
Redis & ZeroMQ: How to scale your applicationrjsmelo
Presented at #PHPLX 11 July 2013
When you need to do some heavy processing how do you scale you application?
You can use Redis and ZeroMQ to leverage the heavy work for you!
With this presentation we will know more about this two technologies and how they can be used to help solve problems with the performance and scalability of your application.
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club UkraineElixir Club
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga
Slides of Alexei Sholik, Andrew Dryga, at Lightning Talk session at Elixir Club Ukraine, Kyiv, 28.09.2019
Next conference - http://www.elixirkyiv.club/
Follow us on social networks @ElixirClubUA and #ElixirClubUA
Announce and materials from conf - https://www.fb.me/ElixirClubUA
News - https://twitter.com/ElixirClubUA
Photo and free atmosphere - https://www.instagram.com/ElixirClubUA
*Organizer’s channel - https://t.me/incredevly
AMS Node Meetup December presentation Phusion Passengericemobile
Phusion Passenger is an app server for Node.js, Ruby and Python. It simplifies deployment and administration, increases your server's efficiency and helps identifying and solving problems.
In this talk Hongli Lai demonstrates how Passenger simplifies things by integrating with Nginx and by replacing Forever, PM2, Cluster and all sorts of other tools. Hongli also shares what other benefits Passenger has to offer, and what you can expect from future developments.
Slides from my talk at Philly ETE looking at the Lambda Architecture (originating at twitter) critically from the perspective of someone viewing it from the financial (faster, higher volume, spikier data) domain
Messaging becomes Data Distributions gets embedded event processing (not complex, made simple) - bending all the rules one benchmark at a time - Push Technology, Waratek and other things
StreamBase - Embedded Erjang - Erlang User Group London - 20th April 2011darach
A presentation delivered to the Erlang User Group in London demonstrating how to embed the erjang implementation of erlang into the StreamBase CEP engine, enabling extending StreamBase with erlang based extensions.
Starting with the system calll "getrusage", this returns synchronous, process-level information, mainly max RSS used. This talk describes the output from getrusage, the rusage formatting utility in ProcStats, and several examples of using it to examine time and memory use.
Optional first & final outputs to give baseline and total status, differencing avoids extraneous output, and user messages allow arbitrary stat's and tracking content.
The combination makes this nice for tracking both long-lived and shorter, more intensive processing.
Node has captured the attention of early adopters by clearly differentiating itself as being asynchronous from the ground up while remaining accessible. Now that server side JavaScript is at the cutting edge of the asynchronous, real time web, it is in a much better position to establish itself as the go to language for also making synchronous, CRUD webapps and gain a stronger foothold on the server.
This talk covers the current state of server side JavaScript beyond Node. It introduces Common Node, a synchronous CommonJS compatibility layer using node-fibers which bridges the gap between the different platforms. We look into Common Node's internals, compare its performance to that of other implementations such as RingoJS and go through some ideal use cases.
Imagine you're tackling one of these evasive performance issues in the field, and your go-to monitoring checklist doesn't seem to cut it. There are plenty of suspects, but they are moving around rapidly and you need more logs, more data, more in-depth information to make a diagnosis. Maybe you've heard about DTrace, or even used it, and are yearning for a similar toolkit, which can plug dynamic tracing into a system that wasn't prepared or instrumented in any way.
Hopefully, you won't have to yearn for a lot longer. eBPF (extended Berkeley Packet Filters) is a kernel technology that enables a plethora of diagnostic scenarios by introducing dynamic, safe, low-overhead, efficient programs that run in the context of your live kernel. Sure, BPF programs can attach to sockets; but more interestingly, they can attach to kprobes and uprobes, static kernel tracepoints, and even user-mode static probes. And modern BPF programs have access to a wide set of instructions and data structures, which means you can collect valuable information and analyze it on-the-fly, without spilling it to huge files and reading them from user space.
In this talk, we will introduce BCC, the BPF Compiler Collection, which is an open set of tools and libraries for dynamic tracing on Linux. Some tools are easy and ready to use, such as execsnoop, fileslower, and memleak. Other tools such as trace and argdist require more sophistication and can be used as a Swiss Army knife for a variety of scenarios. We will spend most of the time demonstrating the power of modern dynamic tracing -- from memory leaks to static probes in Ruby, Node, and Java programs, from slow file I/O to monitoring network traffic. Finally, we will discuss building our own tools using the Python and Lua bindings to BCC, and its LLVM backend.
Brief introduction to DTrace technologies within OpenSolaris/Solaris 10 and DTrace probes within Apache, PHP and MySQL can provide end to end dynamic tracing of your Drupal based web site..
A subject named "Internet Technology and its Applications" subject's introduction taught as part of B.E.IV (Computer), 8th semester students, VNSGU, Surat, Gujarat State, India.
Overview of myHadoop 0.30, a framework for deploying Hadoop on existing high-performance computing infrastructure. Discussion of how to install it, spin up a Hadoop cluster, and use the new features.
myHadoop 0.30's project page is now on GitHub (https://github.com/glennklockwood/myhadoop) and the latest release tarball can be downloaded from my website (glennklockwood.com/files/myhadoop-0.30.tar.gz)
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
3. A <3able Stream
// by @TooTallNate - https://gist.github.com/TooTallNate/3947591
// npm install lame ; npm install speaker ; node mp3player.js
var fs = require('fs');
var lame = require('lame');
var Speaker = require('speaker');
fs.createReadStream(process.argv[2])
.pipe(new lame.Decoder())
.on('format', function (format) {
this.pipe(new Speaker(format));
});
4. A <3able Stream
// by @TooTallNate - https://gist.github.com/TooTallNate/3947591
// npm install lame ; npm install speaker ; node mp3player.js
var fs = require('fs');
var lame = require('lame');
var Speaker = require('speaker');
fs.createReadStream(process.argv[2])
.pipe(new lame.Decoder())
.on('format', function (format) {
this.pipe(new Speaker(format));
});
5. Sound Soup
var fs = require('fs');
var lame = require('lame');
var Speaker = require('speaker');
!
// Cache names of mp3's in cwd
function isMp3(e) { return e.indexOf("mp3") > 1 };
var soundz = fs.readdirSync('.').filter(isMp3);
!
// Every half second, play a random sound
setInterval(function(){
blend() },500);
function rand(min, max) {
var offset = min;
var range = (max - min) + 1;
return Math.floor( Math.random() * range) +
offset;
}
function blend() {
fs.createReadStream(soundz[rand(0,soundz.length-1)])
.pipe(new lame.Decoder())
.on('format', function (format) {
this.pipe(new Speaker(format));
});
}
9. Streams 1
• EventEmitter and a pipe() function.
• pipe() is an instance of the Builder Pattern
• EventEmitter is an instance of Subject/Observer
Pattern
10. Streams 1: Example
var Stream = require(‘stream').Stream
var util = require('util')
function MyStream () {
Stream.call(this)
}
util.inherits(MyStream, Stream)
// stream logic and state management and back-pressure
// you’re on your own (with the rest of the pipeline)
// this was error prone and painful …
11. Streams 2
• EventEmitter and a pipe() function.
• Readable, Writable, Duplex, Transform,
Passthrough (and classic) ‘behaviours'
• New ‘behaviours’ encourage good ‘backpressure'
handling disciplines
12. Streams 2: Using Readable
// From: Stream Handbook, Readable with Push
var Readable = require('stream').Readable;
var rs = new Readable;
rs.push(‘beep ‘);
rs.push(‘boopn’);
rs.push(null);
rs.pipe(process.stdout);
13. Streams 2: Using Readable
// From: Stream Handbook
var Readable = require('stream').Readable;
var rs = Readable();
var c = 97;
rs._read = function () {
rs.push(String.fromCharCode(c++));
if (c > 'z'.charCodeAt(0)) rs.push(null);
};
rs.pipe(process.stdout); // _read called once, prints ‘a-z'
15. Streams 2: Pop Quiz!!!
// From: Stream Handbook (tweaked)
var Readable = require('stream').Readable;
var rs = Readable();
var c = 97;
rs._read = function () {
rs.push(String.fromCharCode(c++));
if (c > 'z'.charCodeAt(0)) rs.push(null);
};
rs.pipe(process.stdout); // _read called once
rs.pipe(process.stdout); // _read called twice
rs.pipe(process.stdout); // _read called three times
rs.pipe(process.stdout); // _read called four times
16. Streams 2: Pop Quiz!!!
$ node readable-popquiz.js
aaaabbbbccccddddeeeeffffgggghhhhiiiijjjjkkkkl
lllmmmmnnnnooooppppqqqqrrrrssssttttuuuuvvvvww
wwxxxxyyyyzzzz%
$
… WAT? (Think about it)
!
18. Streams 2: Using Writable
var Writable = require('stream').Writable;
var src = process.stdin;
var dst = new Writable;
dst._write = function (chunk, enc, next) {
console.log(chunk.toString());
next();
};
src.push("hit any key!");
src.pipe(dst);
19. Streams 2: Transform
var Transform = stream.Transform || require('readable-stream').Transform;
function Hash(algorithm, encoding, options) {
Transform.call(this, options);
this.digester = crypto.createHash(algorithm);
this.encoding = encoding;
}
util.inherits(Hash, Transform);
Hash.prototype._transform = function (chunk, enc, cb) {
var bf = (Buffer.isBuffer(chunk)) ? chunk : new Buffer(chunk, enc);
this.digester.update(bf);
cb();
};
Hash.prototype._flush = function (cb) {
this.push(this.digester.digest(this.encoding));
cb();
};
20. Streams 2: Transform
var hash = require('./hash.js');
var fs = require('fs');
var rs = fs.createReadStream(‘file.js’);
var md5Hash = hash.Hash('md5', 'base64');
// var sha1Hash = hash.Hash('sha1', 'base64');
rs.pipe(md5Hash).pipe(process.stdout);
21. Streams 2: Duplex
* Independent Readable and Writable channels
* Implement _read and _write
* Example use
- network protocols
- serial communications with hardware
- …
22. Know a classic stream
// From: Stream Handbook
process.stdin.on('data', function (buf) {
console.log(buf);
});
process.stdin.on('end', function () {
console.log('__END__');
});
28. Streams … 4IO
! IO Oriented
! Finite, not infinite
! Asynchronous
! Lazy
! Assumes in transit data, not in memory forms
! Great for IO. Not efficient for compute
31. Beams 101
! Sources
! Where data pours in
! Sinks
! Where results pop out
! Operators
! Filters – Drop or pass on data based on a UDF
! Transforms – Mutate data based on a UDF
! Branch with Filters
! Combine with Sinks
32. Beams… 4CPU
! Compute Oriented
! Infinite, not finite
! Synchronous
! Eager
! Assumes in memory form, not encoded or wire data
! Convenient for compute. Not good for IO