PHP 5.3 has many new features that allow very different paradigms of software development, that may be unfamiliar to many PHP developers. If you want to learn more about functional or aspect-oriented programming, or how to organize your PHP libraries according to the new de facto PHP namespacing standard, don't miss this talk.
If your not using an ORM (object relational mapper) and are still writing SQL by hand, here's what you need to know.
An introduction into DBIx::Class and some of the concepts and goodies you should be aware off.
Slides for my talk in Spark Summit 2017 SF : https://spark-summit.org/2017/events/hive-bucketing-in-apache-spark/
Bucketing is a partitioning technique that can improve performance in certain data transformations by avoiding data shuffling and sorting. The general idea of bucketing is to partition, and optionally sort, the data based on a subset of columns while it is written out (a one-time cost), while making successive reads of the data more performant for downstream jobs if the SQL operators can make use of this property. Bucketing can enable faster joins (i.e. single stage sort merge join), the ability to short circuit in FILTER operation if the file is pre-sorted over the column in a filter predicate, and it supports quick data sampling.
In this session, you’ll learn how bucketing is implemented in both Hive and Spark. In particular, Patil will describe the changes in the Catalyst optimizer that enable these optimizations in Spark for various bucketing scenarios. Facebook’s performance tests have shown bucketing to improve Spark performance from 3-5x faster when the optimization is enabled. Many tables at Facebook are sorted and bucketed, and migrating these workloads to Spark have resulted in a 2-3x savings when compared to Hive. You’ll also hear about real-world applications of bucketing, like loading of cumulative tables with daily delta, and the characteristics that can help identify suitable candidate jobs that can benefit from bucketing.
These are slides from our Big Data Warehouse Meetup in April. We talked about NoSQL databases: What they are, how they’re used and where they fit in existing enterprise data ecosystems.
Mike O’Brian from 10gen, introduced the syntax and usage patterns for a new aggregation system in MongoDB and give some demonstrations of aggregation using the new system. The new MongoDB aggregation framework makes it simple to do tasks such as counting, averaging, and finding minima or maxima while grouping by keys in a collection, complementing MongoDB’s built-in map/reduce capabilities.
For more information, visit our website at http://casertaconcepts.com/ or email us at info@casertaconcepts.com.
Enough suffering, fix your architecture!Luís Cobucci
You aim for a simple change to deliver a feature, but the outcome is chaotic: coupled code, unrelated things breaking, one tiny change results on a humongous changeset. Aren't you tired of this and think that software development should be simple and make people happy? Do not despair we're here to help you!
In this talk, we take a look at software anatomy, laugh at some mistakes we constantly make, and see a proof of concept using modern PHP tools. Then you will finally be able to break this cycle and find your way out of this madness.
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Sca...Thoughtworks
3 years ago, Springer decided to use Scala on a large, strategic project. This talk is about the journey the development teams made. Why did they choose Scala in the first place? Did they get what they hoped for? What challenges and surprises did they encounter along the way? And, most importantly, are they still happy with their choice?
In this session we covered Classes, Modules and Mixins in more depth. Ruby's inheritance mechanism was covered, as well as the uses of instance and class variables.
SPARQL introduction and training (130+ slides with exercices)Thomas Francart
Full SPARQL training
Covers all SPARQL : basic graph patterns, FILTERs, functions, property paths, optional, negation, assignation, aggregation, subqueries, federated queries.
Does not cover except SPARQL updates.
Includes exercices on DBPedia.
CC BY license
PHP 5.3 has many new features that allow very different paradigms of software development, that may be unfamiliar to many PHP developers. If you want to learn more about functional or aspect-oriented programming, or how to organize your PHP libraries according to the new de facto PHP namespacing standard, don't miss this talk.
If your not using an ORM (object relational mapper) and are still writing SQL by hand, here's what you need to know.
An introduction into DBIx::Class and some of the concepts and goodies you should be aware off.
Slides for my talk in Spark Summit 2017 SF : https://spark-summit.org/2017/events/hive-bucketing-in-apache-spark/
Bucketing is a partitioning technique that can improve performance in certain data transformations by avoiding data shuffling and sorting. The general idea of bucketing is to partition, and optionally sort, the data based on a subset of columns while it is written out (a one-time cost), while making successive reads of the data more performant for downstream jobs if the SQL operators can make use of this property. Bucketing can enable faster joins (i.e. single stage sort merge join), the ability to short circuit in FILTER operation if the file is pre-sorted over the column in a filter predicate, and it supports quick data sampling.
In this session, you’ll learn how bucketing is implemented in both Hive and Spark. In particular, Patil will describe the changes in the Catalyst optimizer that enable these optimizations in Spark for various bucketing scenarios. Facebook’s performance tests have shown bucketing to improve Spark performance from 3-5x faster when the optimization is enabled. Many tables at Facebook are sorted and bucketed, and migrating these workloads to Spark have resulted in a 2-3x savings when compared to Hive. You’ll also hear about real-world applications of bucketing, like loading of cumulative tables with daily delta, and the characteristics that can help identify suitable candidate jobs that can benefit from bucketing.
These are slides from our Big Data Warehouse Meetup in April. We talked about NoSQL databases: What they are, how they’re used and where they fit in existing enterprise data ecosystems.
Mike O’Brian from 10gen, introduced the syntax and usage patterns for a new aggregation system in MongoDB and give some demonstrations of aggregation using the new system. The new MongoDB aggregation framework makes it simple to do tasks such as counting, averaging, and finding minima or maxima while grouping by keys in a collection, complementing MongoDB’s built-in map/reduce capabilities.
For more information, visit our website at http://casertaconcepts.com/ or email us at info@casertaconcepts.com.
Enough suffering, fix your architecture!Luís Cobucci
You aim for a simple change to deliver a feature, but the outcome is chaotic: coupled code, unrelated things breaking, one tiny change results on a humongous changeset. Aren't you tired of this and think that software development should be simple and make people happy? Do not despair we're here to help you!
In this talk, we take a look at software anatomy, laugh at some mistakes we constantly make, and see a proof of concept using modern PHP tools. Then you will finally be able to break this cycle and find your way out of this madness.
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Sca...Thoughtworks
3 years ago, Springer decided to use Scala on a large, strategic project. This talk is about the journey the development teams made. Why did they choose Scala in the first place? Did they get what they hoped for? What challenges and surprises did they encounter along the way? And, most importantly, are they still happy with their choice?
In this session we covered Classes, Modules and Mixins in more depth. Ruby's inheritance mechanism was covered, as well as the uses of instance and class variables.
SPARQL introduction and training (130+ slides with exercices)Thomas Francart
Full SPARQL training
Covers all SPARQL : basic graph patterns, FILTERs, functions, property paths, optional, negation, assignation, aggregation, subqueries, federated queries.
Does not cover except SPARQL updates.
Includes exercices on DBPedia.
CC BY license
This is a survey about Dialog System, Question and Answering, including the 03 generations: (1) Symbolic Rule/Template Based QA; (2) Data Driven, Learning; (3) Data-Driven Deep Learning. It also presents the available Frameworks and Datas for Dialog Systems.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking.
📕 Curious on our agenda? Wait no more!
10:00 Welcome note - UiPath Community in Dubai
Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank
10:20 A UiPath cross-region MEA overview
Ashraf El Zarka, VP and Managing Director MEA, UiPath
10:35: Customer Success Journey
Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank
11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more
Boris Krumrey, Global VP, Automation Innovation, UiPath
12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services.
Brendan Lingam, Director of Sales and Business Development, Marc Ellis
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
The Metaverse and AI: how can decision-makers harness the Metaverse for their...Jen Stirrup
The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives?
How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
2. In this lecture
• Summary of XQuery
• Default data type mapping(
http://en.wikipedia.org/wiki/XQuery_API_for_Java)
• FLWOR expressions
• FOR and LET expressions
• Collections and sorting
Resources
W3C recommendation: www.w3.org/TR/xquery/
Google
5. XQuery
Find all book titles published after 1995:
FOR $x IN doc ("books")/bib/book
WHERE $x/year > 1995
RETURN $x/title
FOR $x IN doc ("books")/bib/book
WHERE $x/year > 1995
RETURN $x/title
Result:
<title> abc </title>
<title> def </title>
<title> ghi </title>
6. XQuery
For each author of a book by Morgan
Kaufmann, list all books she published:
FOR $a IN distinct-values(doc ("books.xml")
/bib/book[publisher=“Morgan Kaufmann”]/author)
RETURN <result>
$a,
FOR $t IN /bib/book[author=$a]/title
RETURN $t
</result>
FOR $a IN distinct-values(doc ("books.xml")
/bib/book[publisher=“Morgan Kaufmann”]/author)
RETURN <result>
$a,
FOR $t IN /bib/book[author=$a]/title
RETURN $t
</result>
distinct = a function that eliminates duplicates
8. XQuery
• FOR $x in expr -- binds $x to each element
in the list expr
• LET $x = expr -- binds $x to the entire list
expr
– Useful for common subexpressions and for
aggregations
9. XQuery
count = a (aggregate) function that returns the number of elms
<big_publishers>
FOR $p IN distinct-values(doc ("books.xml")//publisher)
LET $b := doc ("books.xml")/book[publisher = $p]
WHERE count($b) > 100
RETURN $p
</big_publishers>
<big_publishers>
FOR $p IN distinct-values(doc ("books.xml")//publisher)
LET $b := doc ("books.xml")/book[publisher = $p]
WHERE count($b) > 100
RETURN $p
</big_publishers>
10. XQuery
Find books whose price is larger than
average:
LET $a=avg(doc(“books.xml")/bib/book/@price)
FOR $b in doc("books.xml")/bib/book
WHERE $b/@price > $a
RETURN $b
LET $a=avg(doc(“books.xml")/bib/book/@price)
FOR $b in doc("books.xml")/bib/book
WHERE $b/@price > $a
RETURN $b
12. FOR v.s. LET
FOR
• Binds node variables iteration
LET
• Binds collection variables one value
13. FOR v.s. LET
FOR $x IN doc ("books.xml")/bib/book
RETURN <result> $x </result>
FOR $x IN doc ("books.xml")/bib/book
RETURN <result> $x </result>
Returns:
<result> <book>...</book></result>
<result> <book>...</book></result>
<result> <book>...</book></result>
...
LET $x := doc ("books.xml")/bib/book
RETURN <result> $x </result>
LET $x := doc ("books.xml")/bib/book
RETURN <result> $x </result>
Returns:
<result> <book>...</book>
<book>...</book>
<book>...</book>
...
</result>
14. Collections in XQuery
• Ordered and unordered collections
– /bib/book/author = an ordered collection
– Distinct-values(/bib/book/author) = an unordered
collection
• LET $a = /bib/book $a is a collection
• $b/author a collection (several authors...)
RETURN <result> $b/author </result>RETURN <result> $b/author </result>
Returns:
<result> <author>...</author>
<author>...</author>
<author>...</author>
...
</result>
15. Collections in XQuery
What about collections in expressions ?
• $b/@price list of n prices
• $b/@price * 0.7 list of n numbers
• $b/@price * $b/@quantity list of n x m numbers ??
• $b/@price * ($b/@quant1 + $b/@quant2) ≠
$b/@price * $b/@quant1 + $b/@price * $b/@quant2
!!
16. Sorting in XQuery
<publisher_list>
FOR $p IN distinct-values(doc("books.xml")//publisher)
RETURN <publisher> <name> $p/text() </name> ,
FOR $b IN doc("books.xml")//book[publisher = $p]
RETURN <book>
$b/title ,
$b/@price
</book> SORTBY(price DESCENDING)
</publisher> SORTBY(name)
</publisher_list>
<publisher_list>
FOR $p IN distinct-values(doc("books.xml")//publisher)
RETURN <publisher> <name> $p/text() </name> ,
FOR $b IN doc("books.xml")//book[publisher = $p]
RETURN <book>
$b/title ,
$b/@price
</book> SORTBY(price DESCENDING)
</publisher> SORTBY(name)
</publisher_list>
17. Sorting in XQuery
• Sorting arguments: refer to the name space
of the RETURN clause, not the FOR clause
• To sort on an element you don’t want to
display, first return it, then remove it with
an additional query.
18. If-Then-Else
FOR $h IN //holding
RETURN <holding>
$h/title,
IF $h/@type = "Journal"
THEN $h/editor
ELSE $h/author
</holding> SORTBY (title)
FOR $h IN //holding
RETURN <holding>
$h/title,
IF $h/@type = "Journal"
THEN $h/editor
ELSE $h/author
</holding> SORTBY (title)
19. Other Stuff in XQuery
• BEFORE and AFTER
– for dealing with order in the input
• FILTER
– deletes some edges in the result tree
• Recursive functions
– Currently: arbitrary recursion
– Perhaps more restrictions in the future ?
20. Group-By in Xquery ??
• No GROUPBY currently in XQuery
• A recent proposal (next)
– What do YOU think ?
21. Group-By in Xquery ??
FOR $b IN doc ("http://www.bn.com")/bib/book,
$y IN $b/@year
WHERE $b/publisher="Morgan Kaufmann"
RETURN GROUPBY $y
WHERE count($b) > 10
IN <year> $y </year>
FOR $b IN doc ("http://www.bn.com")/bib/book,
$y IN $b/@year
WHERE $b/publisher="Morgan Kaufmann"
RETURN GROUPBY $y
WHERE count($b) > 10
IN <year> $y </year>
SELECT year
FROM Bib
WHERE Bib.publisher="Morgan Kaufmann"
GROUPBY year
HAVING count(*) > 10
SELECT year
FROM Bib
WHERE Bib.publisher="Morgan Kaufmann"
GROUPBY year
HAVING count(*) > 10
with GROUPBY
Equivalent SQL
22. Group-By in Xquery ??
FOR $b IN doc("http://www.bn.com")/bib/book,
$a IN $b/author,
$y IN $b/@year
RETURN GROUPBY $a, $y
IN <result> $a,
<year> $y </year>,
<total> count($b) </total>
</result>
FOR $b IN doc("http://www.bn.com")/bib/book,
$a IN $b/author,
$y IN $b/@year
RETURN GROUPBY $a, $y
IN <result> $a,
<year> $y </year>,
<total> count($b) </total>
</result>
FOR $Tup IN distinct-values(FOR $b IN doc("http://www.bn.com")/bib,
$a IN $b/author,
$y IN $b/@year
RETURN <Tup> <a> $a </a> <y> $y </y> </Tup>),
$a IN $Tup/a/node(),
$y IN $Tup/y/node()
LET $b = doc("http://www.bn.com")/bib/book[author=$a,@year=$y]
RETURN <result> $a,
<year> $y </year>,
<total> count($b) </total>
</result>
FOR $Tup IN distinct-values(FOR $b IN doc("http://www.bn.com")/bib,
$a IN $b/author,
$y IN $b/@year
RETURN <Tup> <a> $a </a> <y> $y </y> </Tup>),
$a IN $Tup/a/node(),
$y IN $Tup/y/node()
LET $b = doc("http://www.bn.com")/bib/book[author=$a,@year=$y]
RETURN <result> $a,
<year> $y </year>,
<total> count($b) </total>
</result>
with GROUPBY
Without GROUPBY
23. Group-By in Xquery ??
FOR $b IN doc("http://www.bn.com")/bib/book,
$a IN $b/author,
$y IN $b/@year,
$t IN $b/title,
$p IN $b/publisher
RETURN
GROUPBY $p, $y
IN <result> $p,
<year> $y </year>,
GROUPBY $a
IN <authorEntry>
$a,
GROUPBY $t
IN $t
<authorEntry>
</result>
Nested GROUPBY’s