Midway in our life's journey, I went astray from the straight imperative road and woke to find myself alone in a dark declarative wood.
My guide out of this dark declarative wood was a familiar friend, SQL, who showed me the way to wrap a context of a window to push through using Window Functions to escape the Inferno.
Next I found myself somewhere in-between running up hill with one foot in front of the other advancing so as the leading foot was always above the ground running with my friend LINQ, I was able to wrap the context of a collection around my data to advance my journey through Purgatorio.
My last guide into the blinding brilliant light of Paradiso was from the Dutch Caribbean, who taught me how to wrap my computations into a context and move my data through leading me into brilliant bliss.
Join me on my divine data comedy.
Overview of a few ways to group and summarize data in R using sample airfare data from DOT/BTS's O&D Survey.
Starts with naive approach with subset() & loops, shows base R's tapply() & aggregate(), highlights doBy and plyr packages.
Presented at the March 2011 meeting of the Greater Boston useR Group.
Python for R developers and data scientistsLambda Tree
This is an introductory talk aimed at data scientists who are well versed with R but would like to work with Python as well. I will cover common workflows in R and how they translate into Python. No Python experience necessary.
This Tutorial is designed for the HDF5 users with some HDF5 experience. It will cover advanced features of the HDF5 library for achieving better I/O performance and efficient storage. The following HDF5 features will be discussed: partial I/O, compression and other filters including new n-bit and scale+offset filters, and data storage options. Significant time will be devoted to the discussion of complex HDF5 datatypes such as strings, variable-length, array and compound datatypes. Participants will work with the Tutorial examples and exercises during the hands-on sessions.
Overview of a few ways to group and summarize data in R using sample airfare data from DOT/BTS's O&D Survey.
Starts with naive approach with subset() & loops, shows base R's tapply() & aggregate(), highlights doBy and plyr packages.
Presented at the March 2011 meeting of the Greater Boston useR Group.
Python for R developers and data scientistsLambda Tree
This is an introductory talk aimed at data scientists who are well versed with R but would like to work with Python as well. I will cover common workflows in R and how they translate into Python. No Python experience necessary.
This Tutorial is designed for the HDF5 users with some HDF5 experience. It will cover advanced features of the HDF5 library for achieving better I/O performance and efficient storage. The following HDF5 features will be discussed: partial I/O, compression and other filters including new n-bit and scale+offset filters, and data storage options. Significant time will be devoted to the discussion of complex HDF5 datatypes such as strings, variable-length, array and compound datatypes. Participants will work with the Tutorial examples and exercises during the hands-on sessions.
Learn to manipulate strings in R using the built in R functions. This tutorial is part of the Working With Data module of the R Programming Course offered by r-squared.
Stefan Hochdörfer - The NoSQL Store everyone ignores: PostgreSQL - NoSQL matt...NoSQLmatters
PostgreSQL is well known being an object-relational database management system. In it`s core PostgreSQL is schema-aware dealing with fixed database tables and column types. However, recent versions of PostgreSQL made it possible to deal with schema-free data. Learn which new features PostgreSQL supports and how to use those features in your application.
This talk was prepared for the November 2013 DataPhilly Meetup: Data in Practice ( http://www.meetup.com/DataPhilly/events/149515412/ )
Map Reduce: Beyond Word Count by Jeff Patti
Have you ever wondered what map reduce can be used for beyond the word count example you see in all the introductory articles about map reduce? Using Python and mrjob, this talk will cover a few simple map reduce algorithms that in part power Monetate's information pipeline
Bio: Jeff Patti is a backend engineer at Monetate with a passion for algorithms, big data, and long walks on the beach. Prior to working at Monetate he performed software R&D for Lockheed Martin, where he worked on projects ranging from social network analysis to robotics.
Data warehouse or conventional database: Which is right for you?Data Con LA
Data Con LA 2020
Description
Developers have a plethora of choice for application data stores. In this talk we'll explore the differences between transaction processing systems like MySQL and analytic databases like ClickHouse to help you make the best choice for your application. Confused about when to use a data warehouse vs a traditional relational database? Open source has so many choices! Using MySQL and ClickHouse as examples, we'll work through use cases to see where each shines. Along the way we'll explore key technical differences like:
* row vs. column storage
* indexing and compression
* query parallelization
* concurrency support
* transaction models.
Finally we'll discuss how to handle use cases that require capabilities of both. Listeners will leave with clear criteria and and deeper understanding of database internals that enable them to make the right choice(s) for their own use cases.
Speaker
Robert Hodges, Altinity, Inc, CEO
Stata cheat sheet: programming. Co-authored with Tim Essam (linkedin.com/in/timessam). See all cheat sheets at http://bit.ly/statacheatsheets. Updated 2016/06/04
Learn to manipulate strings in R using the built in R functions. This tutorial is part of the Working With Data module of the R Programming Course offered by r-squared.
Stefan Hochdörfer - The NoSQL Store everyone ignores: PostgreSQL - NoSQL matt...NoSQLmatters
PostgreSQL is well known being an object-relational database management system. In it`s core PostgreSQL is schema-aware dealing with fixed database tables and column types. However, recent versions of PostgreSQL made it possible to deal with schema-free data. Learn which new features PostgreSQL supports and how to use those features in your application.
This talk was prepared for the November 2013 DataPhilly Meetup: Data in Practice ( http://www.meetup.com/DataPhilly/events/149515412/ )
Map Reduce: Beyond Word Count by Jeff Patti
Have you ever wondered what map reduce can be used for beyond the word count example you see in all the introductory articles about map reduce? Using Python and mrjob, this talk will cover a few simple map reduce algorithms that in part power Monetate's information pipeline
Bio: Jeff Patti is a backend engineer at Monetate with a passion for algorithms, big data, and long walks on the beach. Prior to working at Monetate he performed software R&D for Lockheed Martin, where he worked on projects ranging from social network analysis to robotics.
Data warehouse or conventional database: Which is right for you?Data Con LA
Data Con LA 2020
Description
Developers have a plethora of choice for application data stores. In this talk we'll explore the differences between transaction processing systems like MySQL and analytic databases like ClickHouse to help you make the best choice for your application. Confused about when to use a data warehouse vs a traditional relational database? Open source has so many choices! Using MySQL and ClickHouse as examples, we'll work through use cases to see where each shines. Along the way we'll explore key technical differences like:
* row vs. column storage
* indexing and compression
* query parallelization
* concurrency support
* transaction models.
Finally we'll discuss how to handle use cases that require capabilities of both. Listeners will leave with clear criteria and and deeper understanding of database internals that enable them to make the right choice(s) for their own use cases.
Speaker
Robert Hodges, Altinity, Inc, CEO
Stata cheat sheet: programming. Co-authored with Tim Essam (linkedin.com/in/timessam). See all cheat sheets at http://bit.ly/statacheatsheets. Updated 2016/06/04
A good data model is key to getting the best performance from Apache Cassandra. The Log Structured Storage Engine and it's distributed architecture mean we cannot rely on a paradigm such as Normal Form to evaluate a model. Instead we need to design data models that support the read path of the application. In this talk Aaron Morton will walk through the key principles and patterns of a good CQL3 data model using simple examples.
AWS July Webinar Series: Amazon Redshift Optimizing PerformanceAmazon Web Services
Amazon Redshift is a fast, petabyte-scale data warehouse that makes it easy to analyze your data for a fraction of the cost of traditional data warehouses.
By following a few best practices for schema design and cluster design, you can unleash the high performance capabilties of Amazon Redshift. This webinar is a deep dive into performance tuning techniques based on real-world use cases.
Learning Objectives:
Learn how to get the best performance from your Redshift cluster
Design Amazon Redshift clusters based on real world use cases
See sample tuning scripts to diagnose and maximize cluster performance
Learn about increasing query performance using interleaved sorting
This is a brief introduction to how R can be useful in the manufacturing sector to calculate the frequency of faults and then developing the model so that preventive maintenance can be done
UKOUG Tech14 - Using Database In-Memory Column Store with Complex DatatypesMarco Gralike
Presentation used during the UKOUG Tech14 conference in Liverpool (UK) discussing possibilities of the use of the 12.1.0.2 In-Memory Column Store option with XMLType data type storage options common to the Oracle 12.1 database
Designing a database like an archaeologistyoavrubin
Talk given at the Israel Clojure meetup group, based on the chapter in "500 Lines or less" http://aosabook.org/en/500L/an-archaeology-inspired-database.html
Desk reference for data wrangling, analysis, visualization, and programming in Stata. Co-authored with Tim Essam(@StataRGIS, linkedin.com/in/timessam). See all cheat sheets at http://bit.ly/statacheatsheets. Updated 2016/06/03
S, K, and I combinators with example from C# using MoreLINQ's pipe (K combinator). Implementations of S, K, and I combinators in F# and JavaScript. Ends with proof that SKK = I.
Great Scott! C# 7 is almost out!
Time to hop into the DeLorean with Doc Brown. If my calculations are correct, when this baby hits 88 miles per hour, we'll be traveling back to the future with C# 7. Switches? Where we're going, we don't need Switches we got Pattern Matching. Just hold on for one second. Let's get this straight, in the future Tuples will be usable! This will beg the questions. Where are we? When are we? We are in the future and we now have Local Functions and Records.
You'll walk away with a sense of where C# is going and how you can learn about its new features today by looking back to the future. This is heavy.
Hiking through the Functional Forest with Fizz BuzzMike Harris
FizzBuzz the interview question which we all laugh at. What if I told you that FizzBuzz can be used as a tool to navigate your way through the functional forest? Join me on a short hike through the functional forest using FizzBuzz to navigate our way. We’ll look at Union Types, Pattern Matching, and Higher Order Functions all while staying in the domain of FizzBuzz. You’ll never look at FizzBuzz the same way again.
Using the work of Dr. Graham Hutton as our guide, we'll look at how to satisfy all of your list processing needs with one function, fold. First we'll start off simple by finding the length of a list, then we'll reverse a list, followed by and-ing and or-ing a list; all using fold. Next we'll look at implementing the higher order functions of: map, filter, and zip. Last we'll look at fold in action by using it on the Coin Changer kata. You'll never look at fold the same way again.
Using the work of Dr. Graham Hutton as our guide, we'll look at how to satisfy all of your list processing needs with one function, fold. First we'll start off simple by finding the length of a list, then we'll reverse a list, followed by and-ing and or-ing a list; all using fold. Next we'll look at implementing the higher order functions of: map, filter, and zip. Last we'll look at fold in action by using it on the Coin Changer kata. You'll never look at fold the same way again.
He often used to say there was only one Road; that it was like a great river: its springs were at every doorstep and every path was its tributary. "It's a dangerous business, Frodo, going out of your door," he used to say. "You step into the Road, and if you don't keep your feet, there is no telling where you might be swept off to."
-- Frodo Baggins in The Fellowship of the Ring
When you go off and learn something new there is no telling what you'll bring back. Join me on my travels in the Land of Clojure and see how ideas from Clojure have effected my day-to-day programming in other languages like JavaScript. See how ideas from ES 6 and Lodash can be traced to functional languages like Clojure. You'll never look at the way you program the same way again.
The Marvelous Land of Higher Order FunctionsMike Harris
From my talk at LCNUG.
http://www.lcnug.org/News/14-12-16/LCNUG_January_8th_-_The_Marvelous_Land_of_Higher_Order_Functions.aspx
Looking to be more pragmatic with LINQ? Want to see the connection between the C# you use every day and Functional Programming? Want to stop using for loops?
Join me on a trip to the marvelous Land of Higher Order Functions.
We’ll travel to the forest of Maps, Filters, and Folding—oh my!
Then we’ll visit the towns of Zip and Mapcat.
We’ll end our trip with a sight of the City of FP.
Testing: the final frontier. These are the voyages of Software Craftsmanship. Its continuing mission: to explore strange new worlds, to seek out new code and new patterns, to boldly go where no one has gone before. Looking to take your testing to new worlds? Want to verify your code meetings your user's requirements? Come learn about behavior and property testing in .Net. We'll look at how SpecFlow can get everyone on the same page and how to find more edge cases with property testing using FsCheck.
Learn You a Functional JavaScript for Great GoodMike Harris
We have been told by just about everyone that we should learning functional programming, let this session be your introduction using a language you already know JavaScript.
Yep, JavaScript. No endless amounts of parentheses. No monadic arrows. Just good old JavaScript.
We'll look take a look at the functional JavaScript landscape see how to use:
Currying
Combinators
Multimethods
We'll be using Underscore.js, allong.es, and bilby.js to help us.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload.
Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
Advanced Flow Concepts Every Developer Should KnowPeter Caitens
Tim Combridge from Sensible Giraffe and Salesforce Ben presents some important tips that all developers should know when dealing with Flows in Salesforce.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
Modern design is crucial in today's digital environment, and this is especially true for SharePoint intranets. The design of these digital hubs is critical to user engagement and productivity enhancement. They are the cornerstone of internal collaboration and interaction within enterprises.
6. –Dante (tr. Longfellow), Inferno
Canto I, Lines 1-3
Midway upon the journey of our life
I found myself within a forest dark,
For the straightforward pathway had been lost.
7. Canto
• Data Flow in a System
• The Language of Data Flow Manipulations
• Data Flow in Context
8. Canto
• Data Flow in a System
• The Language of Data Flow Manipulations
• Data Flow in Context
9. Example Data Set
create table stock_price (
symbol varchar(5) not null
,price_date date not null
,price money not null
);
insert into stock_price
(symbol, price_date, price)
values
('ZVZZT', '2017-01-03', 10.02)
,('CBO' , '2017-01-03', 18.99)
,('ZVZZT', '2017-01-04', 9.99)
,('CBO' , '2017-01-04', 19.01)
;
10. Example Data Set
Symbol Price Date Price
ZVZZT 2017-01-03 10.02
CBO 2017-01-03 18.99
ZVZZT 2017-01-04 9.99
CBO 2017-01-04 19.01
11. SQL Statement
select symbol, price
from stock_price
where symbol = 'ZVZZT'
order by price_date
;
Symbol Price Date Price
ZVZZT 2017-01-03 10.02
ZVZZT 2017-01-04 9.99
12. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
13. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
Extract
Aggregate
Map
Filter
Order
14. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
16. SQL Statement
select symbol, price
from stock_price
where symbol = 'ZVZZT'
order by price_date
;
Symbol Price Date Price
ZVZZT 2017-01-03 10.02
ZVZZT 2017-01-04 9.99
2
1
3
4
23. Group By Folded SQL Statement
select
symbol
,max(price_date) as price_date
from stock_price
group by symbol
;
Symbol Price Date
CBO 2017-01-04
ZVZZT 2017-01-04
24. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
25. Group By Folded SQL Statement Data Flow
from
group
by
select max
26. Group By Folded SQL Statement
select
symbol
,max(price_date) as price_date
from stock_price
group by symbol
;
Symbol Price Date
CBO 2017-01-04
ZVZZT 2017-01-04
2
1
3
28. Window Folded SQL Statement
select distinct
symbol
,max(price_date) over(
partition by symbol) as price_date
from stock_price
;
Symbol Price Date
CBO 2017-01-04
ZVZZT 2017-01-04
29. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
30. Window SQL Statement Data View
max(price_date) over(
partition by symbol) as price_date
Symbol Price_Date
CBO 2017-01-03
CBO 2017-01-04
Price_Date
2017-01-04
2017-01-04
32. Window Folded SQL Statement
select distinct
symbol
,max(price_date) over(
partition by symbol) as price_date
from stock_price
;
Symbol Price Date
CBO 2017-01-04
ZVZZT 2017-01-04
1
2
34. Window SQL Statement
select
symbol
,price
,lag(price, 1) over (
partition by symbol
order by price_date) as prev_price
,price - lag(price, 1) over (
partition by symbol
order by price_date) as price_change
,lead(price, 1) over (
partition by symbol
order by price_date) as next_price
from stock_price
;
36. Window SQL Statement Data View
lag(price, 1) over (
partition by symbol
order by price_date) as prev_price
Symbol Price_Date Price
CBO 2017-01-03 18.99
CBO 2017-01-04 19.01
prev_price
(null)
18.99
37. Window SQL Statement Data View
lead(price, 1) over (
partition by symbol
order by price_date) as next_price
Symbol Price_Date Price
CBO 2017-01-03 18.99
CBO 2017-01-04 19.01
next_price
19.01
(null)
38. Logical Order of a T-SQL Statement
1. FROM
2. ON
3. JOIN
4. WHERE
5. GROUP BY
6. WITH ROLLUP
7. HAVING
8. SELECT
9. DISTINCT
10.ORDER BY
11.TOP
40. Window SQL Statement
select
symbol
,price
,lag(price, 1) over (
partition by symbol
order by price_date) as prev_price
,price - lag(price, 1) over (
partition by symbol
order by price_date) as price_change
,lead(price, 1) over (
partition by symbol
order by price_date) as next_price
from stock_price
;
1
2
41. Canto
• Data Flow in a System
• The Language of Data Flow Manipulations
• Data Flow in Context
42. Imperative Example
var s = "Midway in our life's journey, I went astray";
var count = 0;
foreach(var word in s.Split(' '))
{
var stripped =
Regex.Replace(word, "('s)|W+", "");
if (stripped.Length > 2)
count++;
}
44. Nested Example
var s = "Midway in our life's journey, I went astray";
var count = Enumerable.Count(
Enumerable.Where(
Enumerable.Select(
s.Split(' '),
word => Regex.Replace(
word, "('s)|W+", "")),
stripped => stripped.Length > 2));
48. The Language of Data Flow Manipulations
• map
• bind
• filter
• fold
• mutate
• group by
• order by
49. The Language of Data Flow Manipulations
• map: Select
• bind: SelectMany
• filter: Where
• fold: Aggregate
• mutate: ForEach
• group by: GroupBy
• order by: OrderBy
50. The Language of Data Flow Manipulations
• map
• bind
• filter
• fold
• mutate
• group by
• order by
60. Bind Example
var list = new [] {
"Midway in our life's journey,",
"I went astray"
}
.SelectMany(s => s.Split(' '))
.Select(w => Regex.Replace(w, "('s)|W+", ""));
output>
new [] { "Midway", "in", “our",
"life", "journey", “I",
“went", "astray" }
61. Bind Query Syntax Example
var list =
from sentences in
new [] {
"Midway in our life's journey,",
"I went astray"
}
from words in sentences.Split(' ')
select Regex.Replace(words, "('s)|W+", "");
output>
new [] { "Midway", "in", “our",
"life", "journey", “I",
“went", "astray" }
64. SelectMany Source Code
public override bool MoveNext()
{
switch (_state)
{
case 1:
// Retrieve the source enumerator.
_sourceEnumerator = _source.GetEnumerator();
_state = 2;
goto case 2;
case 2:
// Take the next element from the source enumerator.
https://github.com/dotnet/corefx/blob/7a673547563dc92f1f4cad802d7b57483cdf7500/src/System.Linq/src/System/Linq/SelectMany.cs#L127-L223
65. SelectMany Source Code
case 2:
// Take the next element from the source enumerator.
if (!_sourceEnumerator.MoveNext())
{
break;
}
TSource element = _sourceEnumerator.Current;
// Project it into a sub-collection and get its enumerator.
_subEnumerator = _selector(element).GetEnumerator();
_state = 3;
goto case 3;
case 3:
// Take the next element from the sub-collection and yield.
https://github.com/dotnet/corefx/blob/7a673547563dc92f1f4cad802d7b57483cdf7500/src/System.Linq/src/System/Linq/SelectMany.cs#L127-L223
66. SelectMany Source Code
case 3:
// Take the next element from the sub-collection and yield.
if (!_subEnumerator.MoveNext())
{
_subEnumerator.Dispose();
_subEnumerator = null;
_state = 2;
goto case 2;
}
_current = _subEnumerator.Current;
return true;
https://github.com/dotnet/corefx/blob/7a673547563dc92f1f4cad802d7b57483cdf7500/src/System.Linq/src/System/Linq/SelectMany.cs#L127-L223
67. Bind Example
var list = new [] {
"Midway in our life's journey, I went astray"
}
.SelectMany(s => s.Split(' '))
.Select(w => Regex.Replace(w, "('s)|W+", ""));
output>
new [] { "Midway", "in", “our",
"life", "journey", “I",
“went", "astray" }
68. The Language of Data Flow Manipulations
• map
• bind
• filter
• fold
• mutate
• group by
• order by
85. ForEach Source Code
public void ForEach(Action<T> action)
{
// ... assert we have something
int version = _version;
for (int i = 0; i < _size; i++)
{
if (version != _version)
{
break;
}
action(_items[i]);
}
// cannot alter list
if (version != _version)
throw new InvalidOperationException(
SR.InvalidOperation_EnumFailedVersion);
}
https://github.com/dotnet/corefx/blob/bffef76f6af208e2042a2f27bc081ee908bb390b/src/System.Collections/src/System/Collections/Generic/
List.cs#L598-L618
86. Mutate Example
var visited = new List<int>();
new [] {3, 9, 10, 33, 100}
.Where(n => n % 3 == 0)
.ToList()
.ForEach(n => visited.Add(n));
output>
//void
var visited = new [] {3, 9, 33}
87. The Language of Data Flow Manipulations
• map
• bind
• filter
• fold
• mutate
• group by
• order by
89. Group By Example
var groups = new [] {3, 9, 10, 33, 100}
.GroupBy(n => n % 2 == 0)
.ToDictionary(g => g.Key, g => g.ToList());
output>
new Dictionary<bool, List<int>>()
{
{true, new List<int>() {10, 100}},
{false, new List<int>() {3, 9, 33}}
}
92. Group By Example
var groups = new [] {3, 9, 10, 33, 100}
.GroupBy(n => n % 2 == 0)
.ToDictionary(g => g.Key, g => g.ToList());
output>
new Dictionary<bool, List<int>>()
{
{true, new List<int>() {10, 100}},
{false, new List<int>() {3, 9, 33}}
}
93. The Language of Data Flow Manipulations
• map
• bind
• filter
• fold
• mutate
• group by
• order by
98. Order By Example
var unordered = new [] {33, 9, 100, 3, 10}
.OrderBy(n => n);
output>
new [] {3, 9, 10, 33, 100}
99. Canto
• Data Flow in a System
• The Language of Data Flow Manipulations
• Data Flow in Context
100. –Dante (tr. Barolini), Paradiso
Canto XVII, Lines 55-60
You shall leave everything you love most dearly:
this is the arrow that the bow of exile
shoots first. You are to know the bitter taste
of others’ bread, how salt it is, and know
how hard a path it is for one who goes
descending and ascending others’ stairs.
119. Next Steps
• StrangeLoop (conference)
https://www.thestrangeloop.com/
• Brian Lonsdorf - Professor Frisby Introduces Composable Functional JavaScript
(video series)
https://egghead.io/courses/professor-frisby-introduces-composable-functional-
javascript
• Aditya Bhargava - Functors, Applicatives, And Monads In Pictures (blog post)
http://adit.io/posts/2013-04-17-
functors,_applicatives,_and_monads_in_pictures.html
• Scott Wlaschin - F# for Fun and Profit (blog series)
http://fsharpforfunandprofit.com/series/thinking-functionally.html
• Dixin Yan - LINQ and Functional Programming via C# (blog series)
https://weblogs.asp.net/dixin/linq-via-csharp
122. Biography
• Barolini, Teodolinda. Commento Baroliniano, Digital Dante. New York, NY: Columbia University Libraries, 2017.
https://digitaldante.columbia.edu/dante/divine-comedy/
• Ben-Gan, Itzik. Microsoft SQL Server 2012 high-performance T-SQL using Window functions: Microsoft Press, 2012.
• Byham, Rick. Microsoft Docs. https://docs.microsoft.com/en-us/sql/t-sql/queries/select-transact-sql
• Cook, William R. and Herzman, Ronald B. Dante's Divine Comedy. The Great Courses. http://
www.thegreatcourses.com/courses/dante-s-divine-comedy.html
• Equity Regulatory Alert #2016 - 3 Guidance On Test Stock Usage https://www.nasdaqtrader.com/MicroNews.aspx?
id=ERA2016-3
• Petricek, Tomas. Beyond the Monad fashion (I.): Writing idioms in LINQ- http://tomasp.net/blog/idioms-in-linq.aspx/
• Securities Industry Automation Corporation. New York, 10 September 2015. https://www.nyse.com/publicdocs/
ctaplan/notifications/trader-update/CTS_CQS%20%20NEW%20DEDICATED%20TEST%20SYMBOL_09102015.pdf
• Wickham, Hadley, and Garrett Grolemund. R for data science : import, tidy, transform, visualize, and model data.
Sebastopol, CA: O'Reilly Media, 2016. http://r4ds.had.co.nz/
123. Images
• By Domenico di Michelino - Jastrow, Self-photographed, Public Domain, https://
commons.wikimedia.org/w/index.php?curid=970608
• By Sailko - Own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=49841035
• By Lua - Sotheby's, London, 17 December 2015, lot 22, Public Domain, https://
commons.wikimedia.org/w/index.php?curid=32782393
• By Sailko - Own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=56266161
• By William Blake - http://www.blakearchive.org/exist/blake/archive/work.xq?
workid=but812&java=no, Public Domain, https://commons.wikimedia.org/w/index.php?
curid=27118518
• By Salvatore Postiglione - http://www.hampel-auctions.com/en/92-325/onlinecatalog-detail-
n229.html, Public Domain, https://commons.wikimedia.org/w/index.php?curid=26540023
• Di Giovanni Stradano - Opera propria, 2007-10-25, Pubblico dominio, https://
commons.wikimedia.org/w/index.php?curid=2981981