Presentation given at OSCON 2009 and PostgreSQL West 09. Describes SQL solutions to a selection of object-oriented problems:
- Extensibility
- Polymorphism
- Hierarchies
- Using ORM in MVC application architecture
These slides are excerpted from another presentation, "SQL Antipatterns Strike Back."
The document discusses using JSON in MySQL. It begins by introducing the speaker and outlining topics to be covered, including why JSON is useful, loading JSON data into MySQL, performance considerations when querying JSON data, using generated columns with JSON, and searching multi-valued attributes in JSON. The document then dives into examples demonstrating loading sample data from XML to JSON in MySQL, issues that can arise, and techniques for optimizing JSON queries using generated columns and indexes.
Clean Pragmatic Architecture - Avoiding a MonolithVictor Rentea
Talk built based on several of my trainings: http://www.victorrentea.ro/#training
Covers: Clean/Onion/Hexagonal Architecture, Domain Entities, Value Objects, Repository, Extract When it Grows Principle, Dependency Inversion Principle, Clean Code and Design Patterns.
These are the backing slides of the talks given at JPoint 2017 and Devoxx PL 2017: https://www.youtube.com/embed/4-4ahz7zDiQ
This document discusses various techniques for optimizing MySQL queries, including queries for exclusion joins, random selection, and greatest per group. For a query seeking movies without directors, solutions using NOT EXISTS, NOT IN, and outer joins are examined. The outer join solution performed best by taking advantage of a "not exists" optimization. For random selection of a movie, an initial naive solution using ORDER BY RAND() is shown to be inefficient, prompting discussion of alternative approaches.
This document provides an overview of database anti-patterns including:
- The entity-attribute-value pattern, which stores data in a type-value format and cannot efficiently model constraints. Other options include seeking proper relational models or a "poor man's EAV" approach.
- Materialized path trees, which reference the full parent path for each child and violate normalization rules but allow retrieving the entire tree or subtrees with a single query. Moving subtrees only requires updating the path column.
- Surrogate keys, which are artificially added unique identifiers that can make queries harder to read but help ensure primary keys remain immutable.
Streaming Operational Data with MariaDB MaxScaleMariaDB plc
MariaDB experts explain how to stream data using MariaDB MaxScale, a database proxy that can vastly improve your server's transactional data processing without sacrificing scalability, security or speed. In this webinar, learn how to use MaxScale to convert data to JSON documents or AVRO objects, and watch as MariaDB's senior software engineers do a live demo of how to use the Kafka producer.
Watch the webinar here: https://mariadb.com/resources/webinars/streaming-operational-data-mariadb-maxscale
Kotlin and Domain-Driven Design: A perfect match - Kotlin Meetup MunichFlorian Benz
Kotlin helps when pushing for Domain-Driven Design. A common complaint from Java developers is that wrapping and especially mapping values adds too much boilerplate. I rarely hear this from Kotlin developers. This talk presents some of our experience with Kotlin and Domain-Driven Design.
The following article was the basis for this talk:
https://dev.to/flbenz/kotlin-and-domain-driven-design-value-objects-4m32
Presentation given at OSCON 2009 and PostgreSQL West 09. Describes SQL solutions to a selection of object-oriented problems:
- Extensibility
- Polymorphism
- Hierarchies
- Using ORM in MVC application architecture
These slides are excerpted from another presentation, "SQL Antipatterns Strike Back."
The document discusses using JSON in MySQL. It begins by introducing the speaker and outlining topics to be covered, including why JSON is useful, loading JSON data into MySQL, performance considerations when querying JSON data, using generated columns with JSON, and searching multi-valued attributes in JSON. The document then dives into examples demonstrating loading sample data from XML to JSON in MySQL, issues that can arise, and techniques for optimizing JSON queries using generated columns and indexes.
Clean Pragmatic Architecture - Avoiding a MonolithVictor Rentea
Talk built based on several of my trainings: http://www.victorrentea.ro/#training
Covers: Clean/Onion/Hexagonal Architecture, Domain Entities, Value Objects, Repository, Extract When it Grows Principle, Dependency Inversion Principle, Clean Code and Design Patterns.
These are the backing slides of the talks given at JPoint 2017 and Devoxx PL 2017: https://www.youtube.com/embed/4-4ahz7zDiQ
This document discusses various techniques for optimizing MySQL queries, including queries for exclusion joins, random selection, and greatest per group. For a query seeking movies without directors, solutions using NOT EXISTS, NOT IN, and outer joins are examined. The outer join solution performed best by taking advantage of a "not exists" optimization. For random selection of a movie, an initial naive solution using ORDER BY RAND() is shown to be inefficient, prompting discussion of alternative approaches.
This document provides an overview of database anti-patterns including:
- The entity-attribute-value pattern, which stores data in a type-value format and cannot efficiently model constraints. Other options include seeking proper relational models or a "poor man's EAV" approach.
- Materialized path trees, which reference the full parent path for each child and violate normalization rules but allow retrieving the entire tree or subtrees with a single query. Moving subtrees only requires updating the path column.
- Surrogate keys, which are artificially added unique identifiers that can make queries harder to read but help ensure primary keys remain immutable.
Streaming Operational Data with MariaDB MaxScaleMariaDB plc
MariaDB experts explain how to stream data using MariaDB MaxScale, a database proxy that can vastly improve your server's transactional data processing without sacrificing scalability, security or speed. In this webinar, learn how to use MaxScale to convert data to JSON documents or AVRO objects, and watch as MariaDB's senior software engineers do a live demo of how to use the Kafka producer.
Watch the webinar here: https://mariadb.com/resources/webinars/streaming-operational-data-mariadb-maxscale
Kotlin and Domain-Driven Design: A perfect match - Kotlin Meetup MunichFlorian Benz
Kotlin helps when pushing for Domain-Driven Design. A common complaint from Java developers is that wrapping and especially mapping values adds too much boilerplate. I rarely hear this from Kotlin developers. This talk presents some of our experience with Kotlin and Domain-Driven Design.
The following article was the basis for this talk:
https://dev.to/flbenz/kotlin-and-domain-driven-design-value-objects-4m32
MySQL 8 introduces support for ANSI SQL recursive queries with common table expressions, a powerful method for working with recursive data references. Until now, MySQL application developers have had to use workarounds for hierarchical data relationships. It's time to write SQL queries in a more standardized way, and be compatible with other brands of SQL implementations. But as always, the bottom line is: how does it perform? This presentation will briefly describe how to use recursive queries, and then test the performance and scalability of those queries against other solutions for hierarchical queries.
The Secret Life of a Bug Bounty Hunter – Frans Rosén @ Security Fest 2016Frans Rosén
Frans Rosén has reported hundreds of security issues using his big white hat since 2012. He have recieved the biggest bounty ever paid on HackerOne, and is one of the highest ranked bug bounty researchers of all time. He's been bug bounty hunting with an iPhone in Thailand, in a penthouse suite in Las Vegas and without even being present using automation. He'll share his stories about how to act when a company's CISO is screaming "SH******T F*CK" in a phone call 02:30 a Friday night, what to do when companies are sending him money without any reason and why Doctors without Borders are trying to hunt him down.
Don't Be Mocked by your Mocks - Best Practices using MocksVictor Rentea
Do you ❤️ Mocks? When you write your first unit tests, especially on older codebases, mocking foreign code is key to survival. But as you grow older in the craft, you start piling up hours and days wasted to refactor fragile tests or to fix bugs that those heavy mock-based tests didn't catch. And so you start looking at Mocks differently.
Let's go through the key factors to consider to strike the optimal balance between what needs to be mocked away and what code should be tested in integration. There's sometimes a fine line there, often interwoven with strong emotions:
"Why am I testing this?"
"Argh… these tests take too long"
"Can this ever really break?"
etc...
Among the points that we'll touch on:
- Mocks vs Refactoring
- Mocks vs Reliability
- Fine vs Coarse Mocks
- Reproducibility
- Partial Mocks
- Mocking Statics
- Alternatives to Mocks
Speakers: Victor Rentea
Victor is a Java Champion and Independent Trainer with an impressive experience: thousands of developers in dozens of companies trained in dedicated company sessions. He is the founder of one of the largest developer communities in Romania, Bucharest Software Craftsmanship Community and a top international conference speaker.
To find more about him, join a live masterclass or call him in for a company dedicated training: victorrentea.ro
This presentation was given at the November 2012 chapter meeting of the Memphis ISSA. In the presentation, I discuss various methods of exploiting common SQL Injection vulnerabilities, as well as present a specialized technique known as Time-Based Blind SQL Injection. Related to the latter, I give a scenario in which other common forms of SQL Injection would fail to produce results for a penetration tester or attacker, and show how one may overcome this situation by using the specialized technique. The scenario given, along with the sample code, is NOT a contrived example, but instead is closely based on a real-world application that I encountered as part of an assessment.
A live demonstration of the common forms of SQL Injection was also given which utilized the OWASP Broken Web Apps VM, DVWA, Burp Proxy and SQL Power Injector. To demo a real-world time-based blind injection, I created and locally hosted a new application which closely mimicked the real-world application mentioned above.
A New Era of SSRF - Exploiting URL Parser in Trending Programming Languages! ...CODE BLUE
We propose a new exploit technique that brings a whole-new attack surface to bypass SSRF (Server Side Request Forgery) protections. This is a very general attack approach, in which we used in combination with our own fuzzing tool to discover many 0days in built-in libraries of very widely-used programming languages, including Python, PHP, Perl, Ruby, Java, JavaScript, Wget and cURL. The root cause of the problem lies in the inconsistency of URL parsers and URL requesters.
Being a very fundamental problem that exists in built-in libraries, sophisticated web applications such as WordPress (27% of the Web), vBulletin, MyBB and GitHub can also suffer, and 0days have been discovered in them via this technique. This general technique can also adapt to various code contexts and lead to protocol smuggling and SSRF bypassing. Several scenarios will be demonstrated to illustrate how URL parsers can be exploited to bypass SSRF protection and achieve RCE (Remote Code Execution), which is the case in our GitHub Enterprise demo.
Understanding the basics of this technique, the audience won’t be surprised to know that more than 20 vulnerabilities have been found in famous programming languages and web applications aforementioned via this technique.
Trees In The Database - Advanced data structuresLorenzo Alberton
Storing tree structures in a bi-dimensional table has always been problematic. The simplest tree models are usually quite inefficient, while more complex ones aren't necessarily better. In this talk I briefly go through the most used models (adjacency list, materialized path, nested sets) and introduce some more advanced ones belonging to the nested intervals family (Farey algorithm, Continued Fractions, and other encodings). I describe the advantages and pitfalls of each model, some proprietary solutions (e.g. Oracle's CONNECT BY) and one of the SQL Standard's upcoming features, Common Table Expressions.
Zeronights 2016 | A blow under the belt. How to avoid WAF/IPS/DLP | Удар ниже...Дмитрий Бумов
There are regular and smart firewalls. The regular ones are quite clear, it was a good report on the latest Black Hat. However, we need a completely different approach to come round advanced protection.
Есть фаерволы на регулярных выражениях, а есть умные. Если с первыми все понятно — был отличный доклад на последнем Black Hat, то чтобы обойти современную защиту, нужно иметь совершенно другой подход!
This document discusses indexing strategies in MySQL to improve performance and concurrency. It covers how indexes can help avoid lock contention on tables by enabling concurrent queries to access and modify different rows. However, indexes can also cause deadlocks in some situations. The document outlines several cases exploring how indexes impact locking, covering indexes, sorting and query plans.
XSS is much more than just <script>alert(1)</script>. Thousands of unique vectors can be built and more complex payloads to evade filters and WAFs. In these slides, cool techniques to bypass them are described, from HTML to javascript. See also http://brutelogic.com.br/blog
Is it easier to add functional programming features to a query language, or to add query capabilities to a functional language? In Morel, we have done the latter.
Functional and query languages have much in common, and yet much to learn from each other. Functional languages have a rich type system that includes polymorphism and functions-as-values and Turing-complete expressiveness; query languages have optimization techniques that can make programs several orders of magnitude faster, and runtimes that can use thousands of nodes to execute queries over terabytes of data.
Morel is an implementation of Standard ML on the JVM, with language extensions to allow relational expressions. Its compiler can translate programs to relational algebra and, via Apache Calcite’s query optimizer, run those programs on relational backends.
In this talk, we describe the principles that drove Morel’s design, the problems that we had to solve in order to implement a hybrid functional/relational language, and how Morel can be applied to implement data-intensive systems.
(A talk given by Julian Hyde at Strange Loop 2021, St. Louis, MO, on October 1st, 2021.)
Garage4Hackers Ranchoddas Webcast Series - Bypassing Modern WAF's Exemplified...Garage4hackers.com
Presentation slides of Garage4Hackers Ranchoddas Webcast Series - Bypassing Modern WAF's Exemplified At XSS by Rafay Baloch. Ask all your question's related to the webcast here http://goo.gl/Vv10hJ. Don't forget to leave you feedback here https://goo.gl/YrBeic.
Presentation made at GTA meetup in 2012-02-07.
Object Calisthenics is a set of exercise rules to reach better code, maintainable, testable and readable.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
Whitebox testing of Spring Boot applicationsYura Nosenko
This document discusses whitebox testing of Spring Boot applications. It begins with introductions and backgrounds, then discusses issues with existing testing frameworks like TestNG and JUnit 4. It proposes alternatives like Spock and JUnit 5, highlighting advantages of each. It also provides an overview of Spring Boot testing capabilities, focusing on integration testing support, transaction handling, main components, and reactive support. It concludes with examples of setting up Spring Boot testing with Spock and JUnit 5.
This document summarizes a presentation about hacking Ansible to make it more customizable. It discusses how Ansible's plugin system allows it to be extended through modules, filters, lookups, callbacks and caches. Examples are provided of extending Ansible's core functionality by modifying files in the lib directory and writing custom plugins. The presentation also outlines how Ansible's object model works and provides an overview of its growth in modules and plugins over time.
How should a professional software developer behave in code? What guidelines should one follow? How should we name our constructs? What about OOP principles? What's their real use?
This classic training module in my training curricula is the cornerstone of my professionalism. These are my conduit guidelines at work. I've held this training > 10 times, including at Voxxed Days Bucharest 2016 and at a Bucharest Java User Group meetup.
Slides of the talk held at JEEConf, Kiev and jPrime, Sofia. A personal view on the classic topics from the Uncle Bob's Clean Code bible, with some personal additions and tips&tricks. This topic actually represents the core of the training sessions that I provide as an independent trainer (www.victorrentea.ro)
When does InnoDB lock a row? Multiple rows? Why would it lock a gap? How do transactions affect these scenarios? Locking is one of the more opaque features of MySQL, but it’s very important for both developers and DBA’s to understand if they want their applications to work with high performance and concurrency. This is a creative presentation to illustrate the scenarios for locking in InnoDB and make these scenarios easier to visualize. I'll cover: key locks, table locks, gap locks, shared locks, exclusive locks, intention locks, insert locks, auto-inc locks, and also conditions for deadlocks.
PHX - Session #4 Treating Databases as First-Class Citizens in DevelopmentSteve Lange
The document discusses treating databases as first-class citizens in development by managing schemas and data through database projects and tools. It addresses questions around where the truth of a schema resides, how to version databases, generate test data, perform unit testing, and manage changes. The key points are using database projects to represent the truth of the schema, version control to manage versions, test data generators for testing, and schema/data comparison tools to facilitate refactoring and managing changes.
Session #4: Treating Databases as First-Class Citizens in DevelopmentSteve Lange
The document discusses treating databases as first-class citizens in development by managing schemas and data through database projects and tools. It addresses questions around where the truth of a schema resides, how to version databases, generate test data, perform unit testing, and manage changes. The key points are using database projects to represent the truth of the schema, version control to manage versions, test data generators for testing, and tools for schema/data compares and refactoring to facilitate change management.
MySQL 8 introduces support for ANSI SQL recursive queries with common table expressions, a powerful method for working with recursive data references. Until now, MySQL application developers have had to use workarounds for hierarchical data relationships. It's time to write SQL queries in a more standardized way, and be compatible with other brands of SQL implementations. But as always, the bottom line is: how does it perform? This presentation will briefly describe how to use recursive queries, and then test the performance and scalability of those queries against other solutions for hierarchical queries.
The Secret Life of a Bug Bounty Hunter – Frans Rosén @ Security Fest 2016Frans Rosén
Frans Rosén has reported hundreds of security issues using his big white hat since 2012. He have recieved the biggest bounty ever paid on HackerOne, and is one of the highest ranked bug bounty researchers of all time. He's been bug bounty hunting with an iPhone in Thailand, in a penthouse suite in Las Vegas and without even being present using automation. He'll share his stories about how to act when a company's CISO is screaming "SH******T F*CK" in a phone call 02:30 a Friday night, what to do when companies are sending him money without any reason and why Doctors without Borders are trying to hunt him down.
Don't Be Mocked by your Mocks - Best Practices using MocksVictor Rentea
Do you ❤️ Mocks? When you write your first unit tests, especially on older codebases, mocking foreign code is key to survival. But as you grow older in the craft, you start piling up hours and days wasted to refactor fragile tests or to fix bugs that those heavy mock-based tests didn't catch. And so you start looking at Mocks differently.
Let's go through the key factors to consider to strike the optimal balance between what needs to be mocked away and what code should be tested in integration. There's sometimes a fine line there, often interwoven with strong emotions:
"Why am I testing this?"
"Argh… these tests take too long"
"Can this ever really break?"
etc...
Among the points that we'll touch on:
- Mocks vs Refactoring
- Mocks vs Reliability
- Fine vs Coarse Mocks
- Reproducibility
- Partial Mocks
- Mocking Statics
- Alternatives to Mocks
Speakers: Victor Rentea
Victor is a Java Champion and Independent Trainer with an impressive experience: thousands of developers in dozens of companies trained in dedicated company sessions. He is the founder of one of the largest developer communities in Romania, Bucharest Software Craftsmanship Community and a top international conference speaker.
To find more about him, join a live masterclass or call him in for a company dedicated training: victorrentea.ro
This presentation was given at the November 2012 chapter meeting of the Memphis ISSA. In the presentation, I discuss various methods of exploiting common SQL Injection vulnerabilities, as well as present a specialized technique known as Time-Based Blind SQL Injection. Related to the latter, I give a scenario in which other common forms of SQL Injection would fail to produce results for a penetration tester or attacker, and show how one may overcome this situation by using the specialized technique. The scenario given, along with the sample code, is NOT a contrived example, but instead is closely based on a real-world application that I encountered as part of an assessment.
A live demonstration of the common forms of SQL Injection was also given which utilized the OWASP Broken Web Apps VM, DVWA, Burp Proxy and SQL Power Injector. To demo a real-world time-based blind injection, I created and locally hosted a new application which closely mimicked the real-world application mentioned above.
A New Era of SSRF - Exploiting URL Parser in Trending Programming Languages! ...CODE BLUE
We propose a new exploit technique that brings a whole-new attack surface to bypass SSRF (Server Side Request Forgery) protections. This is a very general attack approach, in which we used in combination with our own fuzzing tool to discover many 0days in built-in libraries of very widely-used programming languages, including Python, PHP, Perl, Ruby, Java, JavaScript, Wget and cURL. The root cause of the problem lies in the inconsistency of URL parsers and URL requesters.
Being a very fundamental problem that exists in built-in libraries, sophisticated web applications such as WordPress (27% of the Web), vBulletin, MyBB and GitHub can also suffer, and 0days have been discovered in them via this technique. This general technique can also adapt to various code contexts and lead to protocol smuggling and SSRF bypassing. Several scenarios will be demonstrated to illustrate how URL parsers can be exploited to bypass SSRF protection and achieve RCE (Remote Code Execution), which is the case in our GitHub Enterprise demo.
Understanding the basics of this technique, the audience won’t be surprised to know that more than 20 vulnerabilities have been found in famous programming languages and web applications aforementioned via this technique.
Trees In The Database - Advanced data structuresLorenzo Alberton
Storing tree structures in a bi-dimensional table has always been problematic. The simplest tree models are usually quite inefficient, while more complex ones aren't necessarily better. In this talk I briefly go through the most used models (adjacency list, materialized path, nested sets) and introduce some more advanced ones belonging to the nested intervals family (Farey algorithm, Continued Fractions, and other encodings). I describe the advantages and pitfalls of each model, some proprietary solutions (e.g. Oracle's CONNECT BY) and one of the SQL Standard's upcoming features, Common Table Expressions.
Zeronights 2016 | A blow under the belt. How to avoid WAF/IPS/DLP | Удар ниже...Дмитрий Бумов
There are regular and smart firewalls. The regular ones are quite clear, it was a good report on the latest Black Hat. However, we need a completely different approach to come round advanced protection.
Есть фаерволы на регулярных выражениях, а есть умные. Если с первыми все понятно — был отличный доклад на последнем Black Hat, то чтобы обойти современную защиту, нужно иметь совершенно другой подход!
This document discusses indexing strategies in MySQL to improve performance and concurrency. It covers how indexes can help avoid lock contention on tables by enabling concurrent queries to access and modify different rows. However, indexes can also cause deadlocks in some situations. The document outlines several cases exploring how indexes impact locking, covering indexes, sorting and query plans.
XSS is much more than just <script>alert(1)</script>. Thousands of unique vectors can be built and more complex payloads to evade filters and WAFs. In these slides, cool techniques to bypass them are described, from HTML to javascript. See also http://brutelogic.com.br/blog
Is it easier to add functional programming features to a query language, or to add query capabilities to a functional language? In Morel, we have done the latter.
Functional and query languages have much in common, and yet much to learn from each other. Functional languages have a rich type system that includes polymorphism and functions-as-values and Turing-complete expressiveness; query languages have optimization techniques that can make programs several orders of magnitude faster, and runtimes that can use thousands of nodes to execute queries over terabytes of data.
Morel is an implementation of Standard ML on the JVM, with language extensions to allow relational expressions. Its compiler can translate programs to relational algebra and, via Apache Calcite’s query optimizer, run those programs on relational backends.
In this talk, we describe the principles that drove Morel’s design, the problems that we had to solve in order to implement a hybrid functional/relational language, and how Morel can be applied to implement data-intensive systems.
(A talk given by Julian Hyde at Strange Loop 2021, St. Louis, MO, on October 1st, 2021.)
Garage4Hackers Ranchoddas Webcast Series - Bypassing Modern WAF's Exemplified...Garage4hackers.com
Presentation slides of Garage4Hackers Ranchoddas Webcast Series - Bypassing Modern WAF's Exemplified At XSS by Rafay Baloch. Ask all your question's related to the webcast here http://goo.gl/Vv10hJ. Don't forget to leave you feedback here https://goo.gl/YrBeic.
Presentation made at GTA meetup in 2012-02-07.
Object Calisthenics is a set of exercise rules to reach better code, maintainable, testable and readable.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
Whitebox testing of Spring Boot applicationsYura Nosenko
This document discusses whitebox testing of Spring Boot applications. It begins with introductions and backgrounds, then discusses issues with existing testing frameworks like TestNG and JUnit 4. It proposes alternatives like Spock and JUnit 5, highlighting advantages of each. It also provides an overview of Spring Boot testing capabilities, focusing on integration testing support, transaction handling, main components, and reactive support. It concludes with examples of setting up Spring Boot testing with Spock and JUnit 5.
This document summarizes a presentation about hacking Ansible to make it more customizable. It discusses how Ansible's plugin system allows it to be extended through modules, filters, lookups, callbacks and caches. Examples are provided of extending Ansible's core functionality by modifying files in the lib directory and writing custom plugins. The presentation also outlines how Ansible's object model works and provides an overview of its growth in modules and plugins over time.
How should a professional software developer behave in code? What guidelines should one follow? How should we name our constructs? What about OOP principles? What's their real use?
This classic training module in my training curricula is the cornerstone of my professionalism. These are my conduit guidelines at work. I've held this training > 10 times, including at Voxxed Days Bucharest 2016 and at a Bucharest Java User Group meetup.
Slides of the talk held at JEEConf, Kiev and jPrime, Sofia. A personal view on the classic topics from the Uncle Bob's Clean Code bible, with some personal additions and tips&tricks. This topic actually represents the core of the training sessions that I provide as an independent trainer (www.victorrentea.ro)
When does InnoDB lock a row? Multiple rows? Why would it lock a gap? How do transactions affect these scenarios? Locking is one of the more opaque features of MySQL, but it’s very important for both developers and DBA’s to understand if they want their applications to work with high performance and concurrency. This is a creative presentation to illustrate the scenarios for locking in InnoDB and make these scenarios easier to visualize. I'll cover: key locks, table locks, gap locks, shared locks, exclusive locks, intention locks, insert locks, auto-inc locks, and also conditions for deadlocks.
PHX - Session #4 Treating Databases as First-Class Citizens in DevelopmentSteve Lange
The document discusses treating databases as first-class citizens in development by managing schemas and data through database projects and tools. It addresses questions around where the truth of a schema resides, how to version databases, generate test data, perform unit testing, and manage changes. The key points are using database projects to represent the truth of the schema, version control to manage versions, test data generators for testing, and schema/data comparison tools to facilitate refactoring and managing changes.
Session #4: Treating Databases as First-Class Citizens in DevelopmentSteve Lange
The document discusses treating databases as first-class citizens in development by managing schemas and data through database projects and tools. It addresses questions around where the truth of a schema resides, how to version databases, generate test data, perform unit testing, and manage changes. The key points are using database projects to represent the truth of the schema, version control to manage versions, test data generators for testing, and tools for schema/data compares and refactoring to facilitate change management.
TechDays Tunisia - Visual Studio & SQL Server, Better Together - Ayman El-HattabAyman El-Hattab
The document describes an approach for managing database schema changes using source control and deployment scripts. It involves reverse engineering the database schema into DDL artifacts, storing them in a database project, and using tools like TFS to track changes. Incremental deployment scripts are generated to update the target databases without disrupting existing data or applications. This allows database changes to be developed, tested, and deployed using a similar process as application code.
The document discusses Test4pass, an expert in IT certification exams. It lists popular IT certifications from Microsoft, Cisco, IBM, and CompTIA and provides sample exam questions and answers related to Microsoft SQL Server 2008 database development.
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/tt9w50A5g7u
Jeffrey will cover ways to avoid people thinking your code is bad, some common coding fallacies, and presents two case studies on rewriting bad SQL. The first case study contains three iterations of code written by the instructor, the second case study comes from an eCommerce site that had a great idea but horrible execution.
This document provides an overview of SQL queries and functions. It discusses the SELECT statement for building queries, aggregate functions like COUNT and MIN/MAX, and scalar functions. It also covers queries for adding, updating, deleting records and creating tables. Examples are provided for each SQL statement type. The document concludes with an exercise to recreate the queries in your own database and questions to test understanding of SQL standards, query creation, and query types in Microsoft Access.
Unit Testing like a Pro - The Circle of PurityVictor Rentea
Best practices on designing unit tests, designing testable production code, a glimpse of TDD, using mocks and isolating pure functions for easy testing. Talk distilled from http://victorrentea.ro/#unit-testing
Held at VoxxedDays Bucharest in March 2019.
Avoiding cursors with sql server 2005 tech republicKaing Menglieng
The document discusses how to avoid using cursors in SQL Server 2005 when executing queries. It presents a scenario where cursors would traditionally be used to loop through inventory transaction records and calculate the remaining inventory each day. It then shows two methods using new SQL 2005 features like common table expressions and window functions to solve the problem with a single query instead of cursors. Avoiding cursors improves performance since sets are processed at once rather than row-by-row.
This document discusses productive debugging techniques for software development. It begins by explaining that debugging is an important part of development that makes apps better. It then outlines the canonical debugging process of choosing a focus, stopping before suspect paths, stepping through live code, and inspecting data. The document provides tips for using assertions, logging, static analysis, unit tests, and debug gauges. It also explains how to set breakpoints, use LLDB commands, navigate threads and queues in the debugger, and inspect data. Overall, it promotes debugging approaches that avoid long investigations and help validate assumptions.
The document discusses stored procedures and triggers in databases, noting that stored procedures are reusable SQL code stored on the database that can increase performance, while triggers automatically run SQL code in response to changes made to a database table, such as inserts, updates or deletes. Both stored procedures and triggers can help with tasks like validation, auditing, and increasing performance by reducing traffic between applications and databases.
Return all employee IDs who sold to a customer based in London, of customer demographic ‘x’. The query traverses the hypergraph to match an employee entity with a given employee ID, a company entity located in London, and a sale relation connecting them as the seller and customer respectively. The employee ID is then returned.
SQL Server Development Tools & Processes Using Visual Studio 2010 Ayman El-Hattab
The document discusses database development using database projects in Visual Studio. It covers topics like:
1. Using database projects to model schemas offline and perform schema refactoring and incremental deployments.
2. Reverse engineering existing databases into schema models and using model diff to determine changes for deployment.
3. Integrating database projects with source control and work item tracking in Team Foundation Server (TFS) for development workflows.
4. Automating builds, deployments, testing and change management using TFS for database projects.
This document provides an overview and programming tips for using SQL procedural language (SQL PL) stored procedures on DB2 for z/OS. It discusses various features and enhancements for SQL PL including compound blocks, templates, dynamic SQL, XML support, array data types, global variables, and autonomous transactions. The document also provides examples and best practices for writing SQL procedures, including handling naming resolution, using templates for readability, and working with arrays and dynamic SQL.
Any structure expected to stand the test of time and change needs a strong foundation! Software is no exception. Engineering your code to grow in a stable and effective way is critical to your ability to rapidly meet the growing demands of users, new features, technologies, and platform capabilities. Join us to obtain architect-level design patterns for use in your Apex code to keep it well factored, easy to maintain, and in line with platform best practices. You'll follow a Force.com interpretation of Martin Fowler's Enterprise Architecture Application patterns, and the practice of Separation of Concerns.
This document discusses advanced index tuning techniques in SQL Server, including:
- Using DMVs (dynamic management views) to passively tune indexes by observing performance and removing or adding indexes.
- Active tuning techniques such as avoiding over-application of tuning wizard recommendations and giving indexes smart names for ongoing maintenance.
- Using data compression for indexes in SQL Server 2008 to reduce storage requirements.
- Addressing database fragmentation as a "silent performance killer" and using online reindexing techniques to defragment indexes without taking tables offline.
The most massive crime of identity theft in history was perpetrated in 2007 by exploiting an SQL Injection vulnerability. This issue is one of the most common and most serious threats to web application security. In this presentation, you'll see some common myths busted and you'll get a better understanding of defending against SQL injection.
SQL injection attacks pose a major security threat by allowing attackers to alter intended database queries or commands through injection of malicious SQL code. Effective defenses include whitelisting input, escaping special characters, and using prepared statements with bind variables to separate data from SQL commands. These techniques help prevent attacks that could compromise user data, modify critical database information, or grant unauthorized access to attackers.
PNWPHP -- What are Databases so &#%-ing DifficultDave Stokes
This document discusses why databases can be difficult. It begins by noting that databases are selfish, want entire systems to themselves, are messy and suck up resources. It then compares databases to toddlers. It identifies problems like most PHP developers lacking SQL training. It provides quizzes and discusses concepts like joins, indexes, foreign keys, transactions and query plans. It offers programming advice like checking return codes and scrubbing data. Finally, it recommends books and invites questions.
1. The document discusses the code review process and provides guidance on how to effectively conduct code reviews. It suggests reviewing code from a top-down perspective, starting with high-level architecture and design before examining lower-level implementation details.
2. Key aspects that should be reviewed include problem solution, public APIs, database schema, object-oriented design, method signatures, classes and methods, views, test coverage, and code style. Areas like performance, security, and vulnerabilities should also be evaluated.
3. An effective code review process establishes a review culture that is polite, acknowledges good work, and saves reviewers' and authors' time.
Sql vs no sql diponkar paul-april 2020-Toronto PASSDiponkar Paul
NoSQL database have grown popularity in recent years due to the flexibility of data modeling and scaling up capabilities. NoSQL database also have been using in big data landscape. The demo rich session will elaborate difference between SQL and NoSQL.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
2. Design Patterns
• A solution to a recurring problem, in a context
• Abstracts a recurring design structure
• Distills design experience
• Pattern consists of a problem and a solution
3. Antipatterns
• A solution to a recurring problem, in a context
• Generates problems than solution
• Go from problem to bad solution
• Antipattern consists of two solutions
• One which is beneficial
• One which is problematic
4. Why learning antipatterns?
“Identifying bad practices can be as valuable as
identifying good practices” - Ward Cunningham
Antipatterns identify and categorize the common
mistakes in software practice
“If debugging is the process of removing bugs,
then programming must be the process of
putting them in.” – Edsger W. Dijkstra
5. SQL Antipatterns
Database Design Database Creation
CREATE TABLE BugsProducts
(
bug_id INTEGER REFERENCES Bugs,
product VARCHAR(100) REFERENCES
Products,
PRIMARY KEY (bug_id, product)
);
Query
SELECT b.product, COUNT(*)
FROM BugsProducts AS b
GROUP BY b.product;
Application
SELECT b.product, COUNT(*)
FROM BugsProducts AS b
GROUP BY b.product;
6. SQL Antipatterns
Design Creation Query Application
Jaywalking
Cross walking
Jaywalking
Example Problem
Storing list of authors for a book…
Book Name Authors
Code Complete Steve McConnell
Design Patterns Erich Gamma, Richard Helm, Ralph Johnson, John
Vlissides
Intro. to Automata Theory Jeffrey Ullman, John Hopcroft
Assigning role based menu access…
Menu Name Role Accessibility
Reset Password Admin
Account Heads Senior Accountant, Accounts Manager
YTD Sales Report CFO, Sales Manager, Sales Rep
9. SQL Antipatterns
Design Creation Query Application
Jaywalking
New problems…
Violation of First normal form
Cannot chose a right data type – You can enter Admin, Sales Rep, 1, 2, banana
Cannot use foreign key constraints – No referential integrity
Cannot enforce uniqueness – 1, 2, 3, 4, 3, 3, 2
Cannot easily modify the values in the column
What happens if the length of values exceeds the column limit?
How do you search for menus accessible for Admin and CFO? No indexing is possible
How many roles have access to Current Year Budget Reports? Aggregation is difficult
How do you join the table with Role table?
Storing integer number as character take space than storing as integer?
Workarounds include splitting functions that will be inefficient?
10. SQL Antipatterns
Design Creation Query Application
Jaywalking
Solution : Create an intersection table
Menu MenuAccess Role
Intersection table
CREATE TABLE dbo.Menu
(
MenuID INT,
Menu VARCHAR(50),
Module VARCHAR(50)
CONSTRAINT PK_Menu PRIMARY KEY
CLUSTERED
(
MenuID
)
)
GO
CREATE TABLE dbo.Role
(
RoleID INT,
Name VARCHAR(50),
CreatedDate DATETIME,
CONSTRAINT PK_Role PRIMARY KEY
CLUSTERED
(
RoleID
)
)
GO
CREATE TABLE dbo.MenuAccess
(
MenuAccessID INT,
MenuID INT,
RoleID INT,
CONSTRAINT FK_MenuAccess_Menu
FOREIGN KEY (MenuID)
REFERENCES Menu (MenuID),
CONSTRAINT FK_MenuAccess_Role
FOREIGN KEY (RoleID)
REFERENCES dbo.Role (RoleID)
)
GO
11. SQL Antipatterns
Design Creation Query Application
Keyless Entry
Example Problem
I need a Supplier and a Invoice table…
Invoice table stores number from Supplier table, to
identify the supplier; however…..
I don’t want to make the design complex….
The sentence I’m about to tell you is a secret ! …
I don’t like keys ! Keys and Constraints limit flexibility!
13. SQL Antipatterns
Design Creation Query Application
Keyless Entry
Implementation
Create tables without any FOREIGN KEY constraints
14. SQL Antipatterns
Design Creation Query Application
Keyless Entry
New problems…
Breaking the foundation of relational database - Constraints
Introduce meaningless data – Authors without any books or Orders without customers
Cannot utilize the query optimizations due to constraints – Some RDBMS utilizes
FOREIGN KEY and CHECK constraints to optimize queries
Need to implement custom solutions to check integrity on a later stage
Forced to implement periodic checks to find orphan rows
15. SQL Antipatterns
Design Creation Query Application
Keyless Entry
Solution : Implement referential integrity through FOREIGN KEY constraints
Use cascading referential integrity constraints.
CREATE TABLE dbo.MenuAccess
(
MenuAccessID INT,
MenuID INT,
RoleID INT,
CONSTRAINT FK_MenuAccess_Menu
FOREIGN KEY (MenuID)
REFERENCES dbo.Menu (MenuID)
ON DELETE CASCADE
ON UPDATE CASCADE,
CONSTRAINT FK_MenuAccess_Role
FOREIGN KEY (RoleID)
REFERENCES dbo.Role (RoleID)
ON DELETE SET NULL
ON UPDATE CASCADE,
)
GO
You can even disable and enable FOREIGN KEY,
if needed.
-- Disable FOREIGN KEY
ALTER TABLE MenuAccess
NOCHECK CONSTRAINT FK_MenuAccess_Menu;
-- Enable FOREIGN KEY
ALTER TABLE MenuAccess
WITH CHECK
CHECK CONSTRAINT FK_MenuAccess_Menu;
16. SQL Antipatterns
Design Creation Query Application
‘31 Flavors’
Example Problem
Our bug tracking software supports only three bug
statuses
NEW
INPROGRESS
FIXED
This will never change (I guess!)
17. SQL Antipatterns
Design Creation Query Application
‘31 Flavors’
Objective
1. Restrict a column’s values to a fixed set of values
2. Column never contains an invalid entry
3. Simplifies usage and query development
19. SQL Antipatterns
Design Creation Query Application
‘31 Flavors’
New problems…
What are the available status values?
SELECT DISTINCT BugStatus FROM dbo.Bugs;
After few months, the QA team decides to add a new status ‘Duplicate’. How do you do
that?
Later the team has instructed to change ‘Fixed’ status to ‘Resolved’
20. SQL Antipatterns
Design Creation Query Application
‘31 Flavors’
Solution : Create a lookup table and use DRI
CREATE TABLE dbo.BugStatus
(
BugStatusID INT,
Status VARCHAR(20),
Description VARCHAR(100),
CONSTRAINT PK_BugStatus
PRIMARY KEY CLUSTERED
(BugStatusID)
)
GO
CREATE TABLE dbo.Bugs
(
-- Other columns
BugStatusID INT,
CONSTRAINT FK_Bugs_BugStatus
FOREIGN KEY (BugStatusID)
REFERENCES dbo.BugStatus
(BugStatusID)
);
GO
INSERT INTO dbo.BugStatus (BugStatusID, Status, Description) VALUES
(1, 'NEW', 'Once again a new bug reported'),
(2, 'IN PROGRESS', 'Team is working hard on this'),
(3, 'FIXED', 'The issues fixed for the time being!');
21. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
"There are things known, and there are things unknown, And
in between are the Doors." ~ Jim Morrison
Example Problem
Our library stores information on books and periodicals
Both have a name, publisher, language, and number of pages
However ISBN is applicable to books and ISSN is for
periodicals.
22. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
Objective
Storing values that are not available or not applicable
23. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
Implementation
CREATE TABLE dbo.Collection
(
ID INT NOT NULL,
Name VARCHAR(100) NOT NULL,
Year INT NOT NULL,
PublisherID INT NOT NULL,
-- Applicable to Books
Edition INT NULL,
ISBN VARCHAR(20) NULL,
Binding VARCHAR(20) NULL
CHECK
Binding IN ('HARDCOVER', 'PAPERBACK')),
-- Applicable to periodicals
FrequencyID INT NULL
REFERENCES dbo.Frequency (FrequencyID),
Volume INT NULL,
Issue INT NULL,
ISSN VARCHAR(10) NULL
)
GO
24. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
New problems…
ID Name Year PublisherID Edition ISBN Binding FrequencyID Volume Issue ISSN
1 Introduction to Algorithms 1990 2 3978-0-262-03384-8 PAPERBACK
2Code Complete 1993 5 1 978-1-55615-484-3 PAPERBACK
3 Dr. Dobb's Journal 2009 12 3 34 2 1044-789X
4The C Programming Language 1978 1 10-262-51087-2 PAPERBACK
5 SQL Server Pro 1999 22 3 7 3 1522-2187
What is the result of SELECT ID FROM dbo.Collection WHERE FrequencyID != 3;?
Ans. (0) rows affected
What is the result of
SELECT
Name + ‘,‘ + Edition + ‘(‘ + ISSN + ‘)’
FROM dbo.Collection WHERE ID = 2;
Ans. NULL
What is the result of SELECT COUNT(Volume) FROM dbo.Collection;?
Ans. 2
25. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
Understanding NULL
… in expression
… in Boolean expression
Expression Expected Actual
NULL = 0 TRUE Unknown
NULL = 12345 FALSE Unknown
NULL <> 12345 TRUE Unknown
NULL + 12345 12345 Unknown
NULL + ‘ABC’ ABC Unknown
NULL = NULL TRUE Unknown
NULL <> NULL FALSE Unknown
Expression Expected Actual
NULL AND TRUE FALSE Unknown
NULL AND FALSE FALSE FALSE
NULL OR FALSE FALSE Unknown
NULL OR TRUE TRUE TRUE
NOT (NULL) TRUE Unknown
26. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
Understanding NULL
NULL is not a value in a column
NULL is a marker/construct in SQL for representing missing information
"...it follows that a “type” that “contains a null” isn’t a type,
a “tuple” that “contains a null” isn’t a tuple,
a “relation” that “contains a null” isn’t a relation, and
a “relvar” that contains a null isn’t a relvar.
It follows further that nulls do serious violence to the relational model, and this
dictionary therefore has very little to say regarding most null-related concepts“
– C . J. Date – The Relational Database Dictionary
27. SQL Antipatterns
Design Creation Query Application
Fear of Unknown
Solution: Avoid usage of NULL as far as possible
There are three reasons for NULL creep in to database table…
Inapplicable NULL
The ISSN of periodical is
inapplicable to a book
Solution:
Design specialized tables
with applicable columns.
Create a Book table and a
Periodical table
Inapplicable NULL should
be avoided
Not yet applicable NULL
The Number of copies of
the book is currently not
available, but will soon be
available
Solution:
Give a default value until
the value is available
e.g.: ISO scheme for sex:
0 = UNKNOWN
1 = MALE
2 = FEMALE
9 = NOT APPLICABLE
Nice to know NULL
The comment for a book
has no business value and
will only used for
reporting
Solution:
This column can contain
the marker NULL.
28. SQL Antipatterns
Design Creation Query Application
Pseudo Key Neat-Freak
…D oor ythoius plikaett tehrins… p?attern…?
1 2 3
89 4 144 5 233
6
21 7 34 8 55
9
13 8 5
10 11 12
Example Problem
I used auto generated pseudo key for BookID in Book table
Today morning when I query the table I found gaps in BookID numbers…!
Books are missing!
How the database miss it!
Is that a bug!
Clean it up and make it sequential! Born of a Pseudo Key Neat-Freak
29. SQL Antipatterns
Design Creation Query Application
Pseudo Key Neat-Freak
Objective
1. Pseudo key numbers should always be in sequential order
2. There should not be any gaps between two adjacent pseudo key numbers
30. SQL Antipatterns
Design Creation Query Application
Pseudo Key Neat-Freak
Implementation
1. Do not use auto generating numbers; instead write custom
queries to find the next number for the table
2. Periodically run maintenance scripts to fill the gaps
Gaps and Islands – Itzik Ben-Gan (MVP Deep Dives Vol. 1)
10,000,000 rows with 10,000 gaps in every 1,000 rows
31. SQL Antipatterns
Design Creation Query Application
Pseudo Key Neat-Freak
New problems…
Any custom solution will cause locking and concurrency issues in application
Gap filling algorithms are time consuming and resource intensive
What about missed the propagation of renumbered ID to child tables? You finally
introduce ‘Keyless Entry’ Antipattern. Think about a query that returns “Art of
computer programming Vol 1” by Chetan Bhagat by joining Book and Author
tables
Nightmare starts if almost all tables have pseudo keys
32. SQL Antipatterns
Design Creation Query Application
Pseudo Key Neat-Freak
Solution
Use natural keys as far as possible
Get over it: Don’t try to renumber a pseudo key. Train the mind to take the
‘Pseudo Key’ as ‘Pseudo’; it has no business value
Another argument for reusing the numbers:
“After three years the ID will reach the maximum number! I don’t want to
waste values…”
BIGINT (8 Byte), 263 – 1 = Max. value 9,223,372,036,854,775,807
1 Day = 86400 Seconds
1,000 insertions/sec = 106751991167 days ~ 292,471,208 Years
Note: Pseudo keys are row identifiers; not row numbers
33. SQL Antipatterns
References
SQL Antipatterns – Bill Karwin Applied Mathematics for
Database Professional –
Lex de Haan & Toon
Koppelaars