Apache Storm 0.9 basic training - VerisignMichael Noll
Apache Storm 0.9 basic training (130 slides) covering:
1. Introducing Storm: history, Storm adoption in the industry, why Storm
2. Storm core concepts: topology, data model, spouts and bolts, groupings, parallelism
3. Operating Storm: architecture, hardware specs, deploying, monitoring
4. Developing Storm apps: Hello World, creating a bolt, creating a topology, running a topology, integrating Storm and Kafka, testing, data serialization in Storm, example apps, performance and scalability tuning
5. Playing with Storm using Wirbelsturm
Audience: developers, operations, architects
Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/
Verisign is a global leader in domain names and internet security.
Tools mentioned:
- Wirbelsturm (https://github.com/miguno/wirbelsturm)
- kafka-storm-starter (https://github.com/miguno/kafka-storm-starter)
Blog post at:
http://www.michael-noll.com/blog/2014/09/15/apache-storm-training-deck-and-tutorial/
Many thanks to the Twitter Engineering team (the creators of Storm) and the Apache Storm open source community!
IDA Vulnerabilities and Bug Bounty by Masaaki ChidaCODE BLUE
IDA Pro is an advanced disassembler software and often used in vulnerability research and malware analysis. IDA Pro is used to analyse software behavior in detail, if there was a vulnerability and the user is attacked not only can it have impact in a social sense but also impact legal proceedings. In this presentation I will discuss the vulnerabilities found and attacks leveraging the vulnerabilities and Hex-rays's remediation process and dialogue I had with them.
http://codeblue.jp/en-speaker.html#MasaakiChida
In this presentation we will examine various scalability options in order to improve the robustness and performance of your Spring Batch applications. We start out with a single threaded Spring Batch application that we will refactor so we can demonstrate how to run it using:
* Concurrent Steps
* Remote Chunking
* AsyncItemProcessor and AsyncItemWriter
* Remote Partitioning
Additionally, we will show how you can deploy Spring Batch applications to Spring XD which provides high availability and failover capabilities. Spring XD also allows you to integrate Spring Batch applications with other Big Data processing needs.
넥슨코리아 사내 발표자료로 왓 스튜디오에서 파이썬으로 《야생의 땅: 듀랑고》 서버를 비롯한 여러가지 도구를 만든 경험을 공유합니다.
- 게임서버와 각종 툴, 테스트/빌드/배포 시스템을 만들 때 사용한 재료
- 파이썬 코드 품질 개선, 디버깅, 프로파일링, 최적화
- 파이썬 오픈소스 생태계와 왓 스튜디오가 하는 오픈소스 활동
Surviving the Java Deserialization Apocalypse // OWASP AppSecEU 2016Christian Schneider
The hidden danger of Java deserialization vulnerabilities – which often lead to remote code execution – has gained extended visibility in the past year. The issue has been known for years; however, it seems that the majority of developers were unaware of it until recent media coverage around commonly used libraries and major products. This talk aims to shed some light about how this vulnerability can be abused, how to detect it from a static and dynamic point of view, and -- most importantly -- how to effectively protect against it. The scope of this talk is not limited to the Java serialization protocol but also other popular Java libraries used for object serialization.
The ever-increasing number of new vulnerable endpoints and attacker-usable gadgets has resulted in a lot of different recommendations on how to protect your applications, including look-ahead deserialization and runtime agents to monitor and protect the deserialization process. Coming at the problem from a developer’s perspective and triaging the recommendations for you, this talk will review existing protection techniques and demonstrate their effectiveness on real applications. It will also review existing techniques and present new gadgets that demonstrates how attackers can actually abuse your application code and classpath to craft a chain of gadgets that will allow them to compromise your servers.
This talk will also present the typical architectural decisions and code patterns that lead to an increased risk of exposing deserialization vulnerabilities. Mapping the typical anti-patterns that must be avoided, through the use of real code examples we present an overview of hardening techniques and their effectiveness. The talk will also show attendees what to search the code for in order to find potential code gadgets the attackers can leverage to compromise their applications. We’ll conclude with action items and recommendations developers should consider to mitigate this threat.
--
This talk was presented by Alvaro Muñoz & Christian Schneider at the OWASP AppSecEU 2016 conference in Rome.
Apache Storm 0.9 basic training - VerisignMichael Noll
Apache Storm 0.9 basic training (130 slides) covering:
1. Introducing Storm: history, Storm adoption in the industry, why Storm
2. Storm core concepts: topology, data model, spouts and bolts, groupings, parallelism
3. Operating Storm: architecture, hardware specs, deploying, monitoring
4. Developing Storm apps: Hello World, creating a bolt, creating a topology, running a topology, integrating Storm and Kafka, testing, data serialization in Storm, example apps, performance and scalability tuning
5. Playing with Storm using Wirbelsturm
Audience: developers, operations, architects
Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/
Verisign is a global leader in domain names and internet security.
Tools mentioned:
- Wirbelsturm (https://github.com/miguno/wirbelsturm)
- kafka-storm-starter (https://github.com/miguno/kafka-storm-starter)
Blog post at:
http://www.michael-noll.com/blog/2014/09/15/apache-storm-training-deck-and-tutorial/
Many thanks to the Twitter Engineering team (the creators of Storm) and the Apache Storm open source community!
IDA Vulnerabilities and Bug Bounty by Masaaki ChidaCODE BLUE
IDA Pro is an advanced disassembler software and often used in vulnerability research and malware analysis. IDA Pro is used to analyse software behavior in detail, if there was a vulnerability and the user is attacked not only can it have impact in a social sense but also impact legal proceedings. In this presentation I will discuss the vulnerabilities found and attacks leveraging the vulnerabilities and Hex-rays's remediation process and dialogue I had with them.
http://codeblue.jp/en-speaker.html#MasaakiChida
In this presentation we will examine various scalability options in order to improve the robustness and performance of your Spring Batch applications. We start out with a single threaded Spring Batch application that we will refactor so we can demonstrate how to run it using:
* Concurrent Steps
* Remote Chunking
* AsyncItemProcessor and AsyncItemWriter
* Remote Partitioning
Additionally, we will show how you can deploy Spring Batch applications to Spring XD which provides high availability and failover capabilities. Spring XD also allows you to integrate Spring Batch applications with other Big Data processing needs.
넥슨코리아 사내 발표자료로 왓 스튜디오에서 파이썬으로 《야생의 땅: 듀랑고》 서버를 비롯한 여러가지 도구를 만든 경험을 공유합니다.
- 게임서버와 각종 툴, 테스트/빌드/배포 시스템을 만들 때 사용한 재료
- 파이썬 코드 품질 개선, 디버깅, 프로파일링, 최적화
- 파이썬 오픈소스 생태계와 왓 스튜디오가 하는 오픈소스 활동
Surviving the Java Deserialization Apocalypse // OWASP AppSecEU 2016Christian Schneider
The hidden danger of Java deserialization vulnerabilities – which often lead to remote code execution – has gained extended visibility in the past year. The issue has been known for years; however, it seems that the majority of developers were unaware of it until recent media coverage around commonly used libraries and major products. This talk aims to shed some light about how this vulnerability can be abused, how to detect it from a static and dynamic point of view, and -- most importantly -- how to effectively protect against it. The scope of this talk is not limited to the Java serialization protocol but also other popular Java libraries used for object serialization.
The ever-increasing number of new vulnerable endpoints and attacker-usable gadgets has resulted in a lot of different recommendations on how to protect your applications, including look-ahead deserialization and runtime agents to monitor and protect the deserialization process. Coming at the problem from a developer’s perspective and triaging the recommendations for you, this talk will review existing protection techniques and demonstrate their effectiveness on real applications. It will also review existing techniques and present new gadgets that demonstrates how attackers can actually abuse your application code and classpath to craft a chain of gadgets that will allow them to compromise your servers.
This talk will also present the typical architectural decisions and code patterns that lead to an increased risk of exposing deserialization vulnerabilities. Mapping the typical anti-patterns that must be avoided, through the use of real code examples we present an overview of hardening techniques and their effectiveness. The talk will also show attendees what to search the code for in order to find potential code gadgets the attackers can leverage to compromise their applications. We’ll conclude with action items and recommendations developers should consider to mitigate this threat.
--
This talk was presented by Alvaro Muñoz & Christian Schneider at the OWASP AppSecEU 2016 conference in Rome.
Most learning materials for web app pentesting focus on “old school” apps. Maybe they have a little jQuery sprinkled in, but most of the heavy-lifting happens server-side. With the dawn of frontend frameworks like AngularJS, Vue, and React and Single-Page Applications, the way web apps are developed is changing, and pentesters need to keep up. This talk runs through common security issues with and approaches to testing these new apps.
OWASP SD: Deserialize My Shorts: Or How I Learned To Start Worrying and Hate ...Christopher Frohoff
Object deserialization is an established but poorly understood attack vector in applications that is disturbingly prevalent across many languages, platforms, formats, and libraries.
In January 2015 at AppSec California, Chris Frohoff and Gabe Lawrence gave a talk on this topic, covering deserialization vulnerabilities across platforms, the many forms they take, and places they can be found. It covered, among other things, somewhat novel techniques using classes in commonly used libraries for attacking Java serialization that were subsequently released in the form of the ysoserial tool. Few people noticed until late 2015, when other researchers used these techniques/tools to exploit well known products such as Bamboo, WebLogic, WebSphere, ApacheMQ, and Jenkins, and then services such as PayPal. Since then, the topic has gotten some long-overdue attention and great work is being done by many to improve our understanding and developer awareness on the subject.
This talk will review the details of Java deserialization exploit techniques and mitigations, as well as report on some of the recent (and future) activity in this area.
http://www.meetup.com/Open-Web-Application-Security-Project-San-Diego-OWASP-SD/events/226242635/
Defending against Java Deserialization VulnerabilitiesLuca Carettoni
Java deserialization vulnerabilities have recently gained popularity due to a renewed interest from the security community. Despite being publicly discussed for several years, a significant number of Java based products are still affected. Whenever untrusted data is used within deserialization methods, an attacker can abuse this simple design anti-pattern to compromise your application. After a quick introduction of the problem, this talk will focus on discovering and defending against deserialization vulnerabilities. I will present a collection of techniques for mitigating attacks when turning off object serialization is not an option, and we will discuss practical recommendations that developers can use to help prevent these attacks.
In this talk, we’ll walk through utilizing one of the most popular web vulnerability testing frameworks BurpSuite. During this presentation we will cover the process of how to conduct a successful web penetration tests, while utilizing BurpSuite's features and tools (Free and Pro Version). This discussion will also cover realistic examples and a brief overview of common vulnerabilities found in web applications.
Secrets of Performance Tuning Java on KubernetesBruno Borges
Java on Kubernetes may seem complicated, but after a bit of YAML and Dockerfiles, you will wonder what all that fuss was. But then the performance of your app in 1 CPU/1 GB of RAM makes you wonder. Learn how JVM ergonomics, CPU throttling, and GCs can help increase performance while reducing costs.
Storm is a realtime fault-tolerant and distributed stream data processing system. Storm is used to run various critical computations in Twitter at scale, and in real-time. This talk was presented at the SIGMOD 2014 conference that accepted our paper Storm@Twitter. It describes the architecture of Storm and its methods for distributed scale-out and fault-tolerance. This presentation also describes how queries (aka. topologies) are executed in Storm, and presents some operational stories based on running Storm at Twitter. We also present results from an empirical evaluation demonstrating the resilience of Storm in dealing with machine failures.
Have a lot of data? Using or considering using Apache HBase (part of the Hadoop family) to store your data? Want to have your cake and eat it too? Phoenix is an open source project put out by Salesforce. Join us to learn how you can continue to use SQL, but get the raw speed of native HBase usage through Phoenix.
Most learning materials for web app pentesting focus on “old school” apps. Maybe they have a little jQuery sprinkled in, but most of the heavy-lifting happens server-side. With the dawn of frontend frameworks like AngularJS, Vue, and React and Single-Page Applications, the way web apps are developed is changing, and pentesters need to keep up. This talk runs through common security issues with and approaches to testing these new apps.
OWASP SD: Deserialize My Shorts: Or How I Learned To Start Worrying and Hate ...Christopher Frohoff
Object deserialization is an established but poorly understood attack vector in applications that is disturbingly prevalent across many languages, platforms, formats, and libraries.
In January 2015 at AppSec California, Chris Frohoff and Gabe Lawrence gave a talk on this topic, covering deserialization vulnerabilities across platforms, the many forms they take, and places they can be found. It covered, among other things, somewhat novel techniques using classes in commonly used libraries for attacking Java serialization that were subsequently released in the form of the ysoserial tool. Few people noticed until late 2015, when other researchers used these techniques/tools to exploit well known products such as Bamboo, WebLogic, WebSphere, ApacheMQ, and Jenkins, and then services such as PayPal. Since then, the topic has gotten some long-overdue attention and great work is being done by many to improve our understanding and developer awareness on the subject.
This talk will review the details of Java deserialization exploit techniques and mitigations, as well as report on some of the recent (and future) activity in this area.
http://www.meetup.com/Open-Web-Application-Security-Project-San-Diego-OWASP-SD/events/226242635/
Defending against Java Deserialization VulnerabilitiesLuca Carettoni
Java deserialization vulnerabilities have recently gained popularity due to a renewed interest from the security community. Despite being publicly discussed for several years, a significant number of Java based products are still affected. Whenever untrusted data is used within deserialization methods, an attacker can abuse this simple design anti-pattern to compromise your application. After a quick introduction of the problem, this talk will focus on discovering and defending against deserialization vulnerabilities. I will present a collection of techniques for mitigating attacks when turning off object serialization is not an option, and we will discuss practical recommendations that developers can use to help prevent these attacks.
In this talk, we’ll walk through utilizing one of the most popular web vulnerability testing frameworks BurpSuite. During this presentation we will cover the process of how to conduct a successful web penetration tests, while utilizing BurpSuite's features and tools (Free and Pro Version). This discussion will also cover realistic examples and a brief overview of common vulnerabilities found in web applications.
Secrets of Performance Tuning Java on KubernetesBruno Borges
Java on Kubernetes may seem complicated, but after a bit of YAML and Dockerfiles, you will wonder what all that fuss was. But then the performance of your app in 1 CPU/1 GB of RAM makes you wonder. Learn how JVM ergonomics, CPU throttling, and GCs can help increase performance while reducing costs.
Storm is a realtime fault-tolerant and distributed stream data processing system. Storm is used to run various critical computations in Twitter at scale, and in real-time. This talk was presented at the SIGMOD 2014 conference that accepted our paper Storm@Twitter. It describes the architecture of Storm and its methods for distributed scale-out and fault-tolerance. This presentation also describes how queries (aka. topologies) are executed in Storm, and presents some operational stories based on running Storm at Twitter. We also present results from an empirical evaluation demonstrating the resilience of Storm in dealing with machine failures.
Have a lot of data? Using or considering using Apache HBase (part of the Hadoop family) to store your data? Want to have your cake and eat it too? Phoenix is an open source project put out by Salesforce. Join us to learn how you can continue to use SQL, but get the raw speed of native HBase usage through Phoenix.
Here's the second version of our big data landscape. Thoughts, questions, comments? We'd love to hear your feedback in the comments section here: http://wp.me/p2dLS7-6A
Apache Kafka 0.8 basic training - VerisignMichael Noll
Apache Kafka 0.8 basic training (120 slides) covering:
1. Introducing Kafka: history, Kafka at LinkedIn, Kafka adoption in the industry, why Kafka
2. Kafka core concepts: topics, partitions, replicas, producers, consumers, brokers
3. Operating Kafka: architecture, hardware specs, deploying, monitoring, P&S tuning
4. Developing Kafka apps: writing to Kafka, reading from Kafka, testing, serialization, compression, example apps
5. Playing with Kafka using Wirbelsturm
Audience: developers, operations, architects
Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/
Verisign is a global leader in domain names and internet security.
Tools mentioned:
- Wirbelsturm (https://github.com/miguno/wirbelsturm)
- kafka-storm-starter (https://github.com/miguno/kafka-storm-starter)
Blog post at:
http://www.michael-noll.com/blog/2014/08/18/apache-kafka-training-deck-and-tutorial/
Many thanks to the LinkedIn Engineering team (the creators of Kafka) and the Apache Kafka open source community!
Development Platform as a Service - erfarenheter efter ett års användning - ...IBM Sverige
Presentation från IBM Smarter Business 2011. Spår: Utveckla produkter och tjänster kostnadseffektivt.
Ta del av Tietos erfarenheter inom implementation av agil utveckling och Application Lifecycle Management med IBM Rationals lösningar. Presentationen visar på ett antal olika exempel på implementationer, och en representant från en svensk kund berättar om sina erfarenheter från ett års användning av IBM och Tietos Cloudbaserad utvecklingsplattform, DpaaS.
Talare: Per Engman, Business Development, Tieto.
Mer information på www.smarterbusiness.se
Big Data Analysis Patterns - TriHUG 6/27/2013boorad
Big Data Analysis Patterns: Tying real world use cases to strategies for analysis using big data technologies and tools.
Big data is ushering in a new era for analytics with large scale data and relatively simple algorithms driving results rather than relying on complex models that use sample data. When you are ready to extract benefits from your data, how do you decide what approach, what algorithm, what tool to use? The answer is simpler than you think.
This session tackles big data analysis with a practical description of strategies for several classes of application types, identified concretely with use cases. Topics include new approaches to search and recommendation using scalable technologies such as Hadoop, Mahout, Storm, Solr, & Titan.
Hadoop Summit 2012 | HBase Consistency and Performance ImprovementsCloudera, Inc.
The latest Apache HBase releases, 0.92 and 0.94, contain many improvements over prior releases in terms of correctness and performance improvements. We discuss a couple of these improvements from a development and operations perspective. For correctness, we discuss the ACID guarantees of HBase, give a case study of problems with earlier releases, and give an overview of the implementation internals that were improved to fix the issues. For performance, we discuss recent improvements in 0.94 and how to monitor the performance of a cluster with new metrics.
How Spotify uses large scale Machine Learning running on top of Hadoop to power music discovery. From the NYC Predictive Analytics meetup: http://www.meetup.com/NYC-Predictive-Analytics/events/129778152/
Towards a software ecosystem for java prolog interoperabiltykim.mens
At SATTOSE2013, Sergio Castro presented his work in progress on developing a software ecosystem for Java-Prolog interoperabilty.
Abstract :
The Prolog community, although continuously accomplishing relevant research results on different aspects of Software Engineering, is fragmented by the lack of a common rich ecosystem comparable to the ones surrounding languages belonging to, for example, the object-oriented programming paradigm. This hinders the agile development of new software and often forces practitioners to reimplement existing artifacts, to accommodate them to the requirements of their specific incompatible environments. Some libraries exist that alleviate this problem by providing bridges to object-oriented languages, such as Java, which are surrounded by rich ecosystems. As such, the logic programming community can access and reuse well proven existing frameworks and libraries from these ecosystems. Unfortunately, these bridge libraries themselves suffer from the symptoms of a fragmented community: they provide user and programming interfaces that cannot easily be reused with other libraries that are targeting essentially the same problem. To solve this problem, we have developed a library that decouples the high level programming interface supporting common Java–Prolog operations, from a provider–dependent library interacting with a Prolog engine. A similar approach has been successfully used in the past in the domain of Java-databases interaction (e.g., JDBC). Our library, in addition to proposing a common high level API, provides a simple and modular mechanism for expressing mappings between Prolog and Java artifacts. In addition, we implement a set of reusable components that can be reused independently of the underlying Prolog engine and bridge library.
We illustrate our approach by means of an application allowing a user to visually query geographical data by means of Prolog queries, that can easily change its Prolog engine and bridge library at runtime. We demonstrate how the client code in our example is completely decoupled from any bridge library serving as a port between the two worlds.
Softshake 2013: Introduction to NoSQL with CouchbaseTugdual Grall
This presentation was delivered during Softshake 2013. Learn why RDBMS are not enought and why NoSQL help developers to scale their applications and provide agility.
Concurrency is hard. Consistency in distributed systems is hard. And then the whole thing should be highly-available and error resilient.
Fear not, there are good news: There exists an awesome tool called ZooKeeper to help you with this. There even exists a plethora of Python libraries for it, but how to know what to use and how?
This talk will walk you through ZooKeeper and how to use it with Python. We’ll be focusing on what I think is the most prominient ZooKeeper library out there for Python: Kazoo.
You’ll see how to do things in ZooKeeper and how to implement them using Kazoo. We’ll also peek in to the recipes Kazoo offers, and if we have enough time, touch a real life application we’ve used Kazoo and ZooKeeper to build at Spotify.
Los nuevos Medios de producción; Prototipado rápido, open source, DIY e impre...Rodger Evans
Estamos adentro de una revolución en nuestra sociedad sin saber. Los productos que usamos diariamente son fabricados mas y mas por sistemas robóticas. Mientras pensamos que estés sistemas cuesta millones de dolares hay salido en los últimos diez años varios versiones de ellos con precios menor que tres mil dolares. Este es muy similar de computadoras en los ochentas cuando el primero vez en historia una familia pueden comprar una computadora para su casa.
Este hay causado cambios en arte, música, video etc.. Gente pueden hacer en su casa resultados que diez años antes solamente una estudio profesional puede hacer. El limite no son los aparatos pero los conocimientos. Con licencias tipo open-source y el cultura DIY (Haga usted mismo) el población están compartiendo estés conocimientos permitiendo grandes avances en todos áreas.
En este platica voy hablar sobre estos temas y dar ejemplos de como todos podemos ser parte de este nuevo mundo.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. what is storm?
storm is a platform for doing analysis on streams of
data as they come in, so you can react to data as it
happens.
Sunday, June 16, 13
3. storm v hadoop
storm & hadoop are complementary!
hadoop => big batch processing
storm => fast, reactive, real time processing
Sunday, June 16, 13
4. origins
• originated at backtype, acquired by twitter
in 2011.
• to vastly simplify dealing with queues &
workers.
Sunday, June 16, 13
9. what does storm
provide?
• at least once message processing.
• horizontal scalability.
• no intermediate queues.
• less operational overhead.
• “just works”.
Sunday, June 16, 13
13. typical spouts
• read from a kestrel/kafka queue. {tuples = events}
• read from a http server log. {tuples = http requests}
• read from twitter streaming api. {tuples = tweets}
Sunday, June 16, 13
14. bolts
process input stream - A
produce output stream - B
A A A A A A A A B B B B B B B B
Sunday, June 16, 13
15. bolts
• filtering tuples in a stream.
• aggregation of tuples.
• joining multiple streams.
• arbitrary functions on streams.
• communication with external caches/
dbs.
Sunday, June 16, 13
21. recap
• worker - process that executes a
subset of a topology.
• executor - a thread spawned by a
worker.
• task - performs the actual data
processing.
Sunday, June 16, 13
22. stream grouping
• shuffle grouping - random distribution
of tuples.
• field grouping - groups tuples by a field.
• all grouping - replicates to all tasks.
• global grouping - sends the entire
stream to one task.
Sunday, June 16, 13
23. streaming word-count
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("tweet_spout", new RandomTweetSpout(), 5);
builder.setBolt("parse_bolt", new ParseTweetBolt(), 8)
.shuffleGrouping("tweet_spout")
.setNumTasks(2);
builder.setBolt("count_bolt", new WordCountBolt(), 12)
.fieldsGrouping("parse_bolt", new Fields("word"));
Config config = new Config();
config.setNumWorkers(3);
StormSubmitter.submitTopology(“demo”, config, builder.createTopology());
Sunday, June 16, 13
24. tweet spout
class RandomTweetSpout extends BaseRichSpout {
SpoutOutputCollector collector;
Random rand;
String[] tweets = new String[] {
"@jkrums:There’s a plane in the Hudson. I’m on the ferry to pick up people. Crazy",
"@barackobama: Four more years. pic.twitter.com/bAJE6Vom",
...
};
....
@Override
public void nextTuple() {
Utils.sleep(100);
String tweet = tweets[rand.nextInt(tweets.length)];
collector.emit(new Values(tweet));
}
}
Sunday, June 16, 13
25. parse bolt
class ParseTweetBolt extends BaseBasicBolt {
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
String tweet = tuple.getString(0);
for (String word : tweet.split(" ")) {
collector.emit(new Values(word));
}
}
@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("word"));
}
}
Sunday, June 16, 13
26. word count bolt
class WordCountBolt extends BaseBasicBolt {
Map<String, Integer> counts = new HashMap<String, Integer>();
@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
String word = tuple.getString(0);
Integer count = counts.get(word);
count = (count == null) ? 1 : count + 1;
counts.put(word, count);
collector.emit(new Values(word, count));
}
@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("word", "count"));
}
}
Sunday, June 16, 13
28. how do we run storm
@twitter ?
Sunday, June 16, 13
29. storm on mesos
node node node node
mesos
we run multiple instances of storm on
the same cluster via mesos.
storm
(production)
storm
(dev) provides efficient
resource isolation and
sharing across distributed
frameworks such as
storm.
Sunday, June 16, 13
30. topology isolation
isolation scheduler solves the problem of
multi-tenancy – avoiding resource contention
between topologies, by providing full isolation
between topologies.
Sunday, June 16, 13
31. topology isolation
• shared pool - multiple topologies can
run on the same host.
• isolated pool - dedicated set of hosts to
run a single topology.
Sunday, June 16, 13
39. numbers
• benchmarked at a million tuples
processed per second per node.
• running 30 topologies in a 200 node
cluster..
• processing 50 billion messages a day
with an average complete latency under 50
ms.
Sunday, June 16, 13
42. current use-cases
• discovery of emerging topics/stories.
• online learning of tweet features for search
result ranking.
• realtime analytics for ads.
• internal log processing.
Sunday, June 16, 13
43. tweet scoring pipeline
tweets
data streams
impressions
interactions
storm
topology
graph
store
metadata
store
join: tweets, impressions
join: tweets, interactions
last 7 days of:
tweet ->
feature_val,
feature_type,
timestamp
persistent
store:
tweet ->
feature_val,
feature_type,
timestamp
thrift
service
cassandra
twemcache
input: tweet id
output: score
write tweet
features
Sunday, June 16, 13