Specql is a Clojure library that combines Clojure spec, PostgreSQL, and macros to provide a cleaner way of writing SQL queries and managing database interactions. It defines tables and columns as Clojure data, allowing column sets and where clauses to be easily manipulated. This avoids issues like duplicate code and unnecessary data fetching. Specql returns query results as Clojure data with namespaced keywords, handling joins and relationships between tables. It aims to provide benefits like type safety, upsert support, and integration with both Clojure and ClojureScript. The author is looking for others to try it out and provide feedback to improve the library.
Getting Creative with WordPress Queries, AgainDrewAPicture
This is an update to a talk I gave three years ago at WordCamp Cape Town 2015. We’re at a point now where we have these incredibly powerful query classes in WordPress core that allow you to really tailor down to whatever criterion you want. In this talk, I take you through the outermost abstraction layers and deep into the SQL that WordPress actually uses to query for stuff on your site – it’s very much a “sky’s the limit” kind of situation.
Queries are really interesting and powerful, and a lot of people are intimidated by advanced queries, even with the abstraction layers that WordPress has put in place.
This document summarizes Peter Eisentraut's presentation on XML support in PostgreSQL. It discusses the types of XML support including an XML data type, XML export format, and mapping XML documents to SQL databases. It also outlines specifications like SQL/XML, features of the XML data type, functions for input, output, and querying XML, and the status and future of XML support in PostgreSQL.
Introduction to the basics of Information Retrieval (IR) with an emphasis on Apache Solr/Lucene. A lecture I gave during the JOSA Data Science Bootcamp.
This session is for those who are excited by the great power of Apache Solr search for Drupal and want to take things even further. Do you want take complete control over your search interface and offer more than the default features? Have you ever wondered what it takes to add data to your search index? Curious about defining facets, custom sorting, or making cool new widgets for filtering and faceting? Join us for a technical deep dive into the world of Solr search.
The general topics of this presentation will overlap with those covered at Drupalcon SF for the Drupal 6 version, but we will focus on use of the API as found in the Drupal 7 version.
Introducing the Solr index
* Learn about Solr fields, and how to map Drupal data onto them
* See how to add data to the search index
* Execute a search in PHP code and use the results
Using the API for custom search paths and interfaces
* How to use the prepare and alter hooks for the query object, and why they differ.
* Make use facing changes, or add filters that are transparent to the user.
Build custom facets based on node fields
* What comes OOTB
* Hooks to add facets for additional field types
Database is a permanent location to store application data in a tabular format. Normalization is used to store the data and involves decomposing schemas into subschemas. Primary and foreign keys play an important role in normalization and are used to create relationships between parent and child tables. A primary key is used in parent tables to prevent duplicate and null values, while a foreign key in child tables references the parent table. For example, an Emp table may have a Dept child table, with Emp No as the foreign key referencing the primary key in Emp. Procedures in databases are used to execute multiple SQL statements in a single request. A DB adapter allows executing SQL commands and procedures by providing configuration like the JNDI name of
Slick is a modern database query and access library for Scala that allows working with stored data like Scala collections while controlling database access. It provides easy, concise, scalable, and safe database access. Slick supports many databases and can be set up by adding dependencies to a build file. Queries are processed by compiling them to SQL and lifting query types for translation. Queries support filtering, dropping rows, deletion, and creation.
You find a column named EntityNum in a table you manage, but what data belongs in this column? Not every detail of usage is clear from just SQL data type and constraints. What is the sensible range of values? Unit of measure? How is the column used by applications? Who in the world knows? We need a way to add comments to the database schema, just as we would write comments in application code to document how programmers should use it. But comments are useful only if they're correct and current, and if they're easy to read and to update. Schemadoc is an experimental tool to help in these goals.
1) SQL is a language used to manage data in relational database management systems. It allows users to define, manipulate, and control access to data.
2) SQL statements like SELECT, INSERT, UPDATE, DELETE allow users to retrieve, add, modify and remove data from database tables.
3) SQL also includes statements for defining the structure of a database through data definition language commands like CREATE DATABASE, CREATE TABLE.
Getting Creative with WordPress Queries, AgainDrewAPicture
This is an update to a talk I gave three years ago at WordCamp Cape Town 2015. We’re at a point now where we have these incredibly powerful query classes in WordPress core that allow you to really tailor down to whatever criterion you want. In this talk, I take you through the outermost abstraction layers and deep into the SQL that WordPress actually uses to query for stuff on your site – it’s very much a “sky’s the limit” kind of situation.
Queries are really interesting and powerful, and a lot of people are intimidated by advanced queries, even with the abstraction layers that WordPress has put in place.
This document summarizes Peter Eisentraut's presentation on XML support in PostgreSQL. It discusses the types of XML support including an XML data type, XML export format, and mapping XML documents to SQL databases. It also outlines specifications like SQL/XML, features of the XML data type, functions for input, output, and querying XML, and the status and future of XML support in PostgreSQL.
Introduction to the basics of Information Retrieval (IR) with an emphasis on Apache Solr/Lucene. A lecture I gave during the JOSA Data Science Bootcamp.
This session is for those who are excited by the great power of Apache Solr search for Drupal and want to take things even further. Do you want take complete control over your search interface and offer more than the default features? Have you ever wondered what it takes to add data to your search index? Curious about defining facets, custom sorting, or making cool new widgets for filtering and faceting? Join us for a technical deep dive into the world of Solr search.
The general topics of this presentation will overlap with those covered at Drupalcon SF for the Drupal 6 version, but we will focus on use of the API as found in the Drupal 7 version.
Introducing the Solr index
* Learn about Solr fields, and how to map Drupal data onto them
* See how to add data to the search index
* Execute a search in PHP code and use the results
Using the API for custom search paths and interfaces
* How to use the prepare and alter hooks for the query object, and why they differ.
* Make use facing changes, or add filters that are transparent to the user.
Build custom facets based on node fields
* What comes OOTB
* Hooks to add facets for additional field types
Database is a permanent location to store application data in a tabular format. Normalization is used to store the data and involves decomposing schemas into subschemas. Primary and foreign keys play an important role in normalization and are used to create relationships between parent and child tables. A primary key is used in parent tables to prevent duplicate and null values, while a foreign key in child tables references the parent table. For example, an Emp table may have a Dept child table, with Emp No as the foreign key referencing the primary key in Emp. Procedures in databases are used to execute multiple SQL statements in a single request. A DB adapter allows executing SQL commands and procedures by providing configuration like the JNDI name of
Slick is a modern database query and access library for Scala that allows working with stored data like Scala collections while controlling database access. It provides easy, concise, scalable, and safe database access. Slick supports many databases and can be set up by adding dependencies to a build file. Queries are processed by compiling them to SQL and lifting query types for translation. Queries support filtering, dropping rows, deletion, and creation.
You find a column named EntityNum in a table you manage, but what data belongs in this column? Not every detail of usage is clear from just SQL data type and constraints. What is the sensible range of values? Unit of measure? How is the column used by applications? Who in the world knows? We need a way to add comments to the database schema, just as we would write comments in application code to document how programmers should use it. But comments are useful only if they're correct and current, and if they're easy to read and to update. Schemadoc is an experimental tool to help in these goals.
1) SQL is a language used to manage data in relational database management systems. It allows users to define, manipulate, and control access to data.
2) SQL statements like SELECT, INSERT, UPDATE, DELETE allow users to retrieve, add, modify and remove data from database tables.
3) SQL also includes statements for defining the structure of a database through data definition language commands like CREATE DATABASE, CREATE TABLE.
The document discusses using databases to store information persistently. It begins by explaining that variables in a program disappear after execution, so some data needs to be stored externally. Initially, simply writing to a file is proposed, but there are issues with this approach like handling concurrent access and updates.
The document then introduces database management systems (DBMS) like MySQL that can handle these complexities. It provides SQL commands to create and manage databases and tables within MySQL, including inserting, selecting, updating, and deleting data in tables. Primary keys are discussed as a way to uniquely identify table rows. Overall the document serves as an introduction to using a DBMS like MySQL to persistently store and manage data through the use of databases
The document discusses using databases to store information persistently. It begins by explaining that variables in a program disappear after execution, so some data needs to be stored externally. Initially, simply writing to a file is proposed, but there are issues with this approach like handling concurrent access and updates. The document then introduces database management systems (DBMS) like MySQL that can address these problems. It provides basic examples of using SQL commands to CREATE, INSERT, SELECT, UPDATE and DELETE data from databases. Primary keys are also introduced as a way to uniquely identify table rows.
Oracle Developer Day, 20 October 2009, Oracle De Meern, Holland: Oracle Datab...Marco Gralike
The document summarizes new features in Oracle XML Database 11g Release 2. Key features include improvements to the XMLIndex such as support for structured XML indexing of highly structured XML content and binary XML partitioning using range, list, and hash methods. Other improvements include enhanced DBMS_METADATA support for object-relational XMLType structures, improved XBRL support, and RAC protocol server support.
Apache Solr is an open-source enterprise search platform that provides fast, scalable, and reliable full-text search functionality. It powers the search capabilities of many large websites and applications. Some key features of Solr include fast indexing and search, faceted search, autocomplete, geospatial search, and integration with various databases and applications.
Make an object be searchable is difficult? Taking control of search result's ranking is even more challenging.
In BarCamp Bangkhen 2015 (Nov 22, 2015), Suparit Krityakien, Co-founder & Software Architecture of Wongnai (http://www.wongnai.com) shared his experiences on applying Wongnai ranking algorithm to take control of our search engine's scoring systematically as well as what can be tuned to make it smarter.
This presentation is to furnish you with fundamental knowledge required when tweaking ranking on your search.
This document provides an overview of PostgreSQL, an open-source object-relational database management system. It discusses differences between PostgreSQL and MySQL, how to install PostgreSQL with PHP, unique features like PL/pgSQL procedural languages for writing functions and stored procedures, and table inheritance. Examples of PL/pgSQL code are provided to demonstrate functions, stored procedures, and returning complex data types.
SQL is a standard language for storing, manipulating and retrieving data in databases. It is used by many database systems like Oracle, Microsoft SQL Server, MySQL, and PostgreSQL. SQL allows users to define data structures, manipulate data, and control access to databases. The language includes commands like SELECT to query data, INSERT to add new data, UPDATE to modify existing data, and DELETE to remove data from databases.
supporting t-sql scripts for Heap vs clustered tableMahabubur Rahaman
The document contains SQL scripts that:
1. Perform various diagnostic checks on indexes such as retrieving index status, enabling/disabling tracing, and viewing index usage statistics.
2. Create tables with different indexing configurations like no indexes, non-clustered indexes, and clustered indexes to test query performance.
3. Query system catalog views and dynamic management functions to retrieve detailed metadata about indexes including column definitions, storage details, usage statistics, and unused indexes.
Starting with JSON Path Expressions in Oracle 12.1.0.2Marco Gralike
This document discusses new features in Oracle Database 12c that allow it to be used as a JSON document store. Key features include the ability to store and index JSON documents, access and query JSON data via SQL operators and functions, load JSON documents into the database using SQL*Loader, and index JSON columns to enable faster querying. The document provides details on the JSON operators, functions, indexing options, and how to validate and retrieve JSON content from the database.
Slick: Bringing Scala’s Powerful Features to Your Database Access Rebecca Grenier
This talk will teach you how to use Slick in practice, based on our experience at EatingWell Media Group. Slick is a totally different (and better!) relational database mapping tool that brings Scala’s powerful features to your database interactions, namely: static-checking, compile-time safety, and compositionality.
Here at EatingWell, we have learned quite a bit about Slick over the past two years as we transitioned from a PHP website to Scala. I will share with you tips and tricks we have learned, as well as everything you need to get started using Slick in your Scala application.
I will begin with Slick fundamentals: how to get started making your connection, the types of databases it can access, how to actually create table objects and make queries to and from them. We will using these fundamentals to demonstrate the powerful features inherited from the Scala language itself: static-checking, compile-time safety, and compositionality. And throughout I will share plenty of tips that will help you in everything from getting started to connection pooling options and configuration for use at scale.
UKOUG Tech14 - Getting Started With JSON in the DatabaseMarco Gralike
Presentation used during the UKOUG Tech14 conference in Liverpool (UK) discussing possibilities of the use of, and explaining, the new JSON database functionality in the Oracle 12.1.0.2 database
Introduction to Solr. A brief introduction to Solr for the resources who wants to get trained on Solr.
1. Introduction to Solr
2. Solr Terminologies
3.Installation and Configuration
4. Configuration files schema.xml and solrconfig.xml
5. Features of SOLR
a. Hit Highlighting
Auto Complete / Suggester
Stop words
Synonyms
SpellCheck
Geo Spatial Search
Result Grouping
Query Syntax
Query Boosting
Content Spotlighting
Block Record / Remove URL Feature
Content Spotlighting / Merchandising / Banner / Elevate
Block Record / Remove URL Feature
6. Indexing the Data
7. Search Queries
8. DataImportHandler - DIH
9. Plugins to index various types of Data (XML, CSV, DB, Filesystem)
10. Solr Client APIs
11. Overview of SOLRJ API
12. Running Solr on Tomcat
13. Enabling SSL on Solr
14. Zookeeper Configuration
15. Solr Cloud Deployment
16. Production Indexing Architecture
17. Production Serving Architecture
18. Solr Upgradation
19. References
The document summarizes Simon Willison's presentation on advanced Django topics including unit testing, newforms, Ajax, and OpenID. Key points include:
- Unit testing in Django is encouraged through features like fixtures, doctests, test client, and email capture. Tests should be written before code using test-driven development.
- Newforms provide a declarative way to create forms with fields, validation, rendering, and conversion to Python types. Forms make it easy to display, validate, and process user submissions.
- The test client allows simulating a browser to test views and templates. Ajax functionality can return HTML, XML, or JSON and be integrated with JavaScript libraries like jQuery.
- Form validation
The Query the Whole Query and Nothing but the QueryChris Olbekson
The document discusses WordPress queries and the $wp_query object. It explains that $wp_query represents the main query and stores information about the current request. It advises against using query_posts() to alter the main query, as it overrides $wp_query and can cause issues. Instead, it recommends using pre_get_posts to modify the main query or creating a secondary query with WP_Query or get_posts() for other queries. It also notes some best practices for resetting post data when performing secondary queries.
The document discusses MySQL query optimization. It covers the query optimizer, principles of optimization like using EXPLAIN and profiling, indexes, JOIN optimization, and ORDER BY/GROUP BY optimization. The key points are to identify bottlenecks, use indexes on frequently filtered fields, avoid indexes on fields that change often or contain many duplicates, and consider composite indexes to cover multiple queries.
We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you practical, step-by-step knowledge to turn a performance bottleneck into an epic win!
Solr is an open source enterprise search platform built on Apache Lucene. It provides full-text search, hit highlighting, faceted search, and handles various document formats. Ajax-Solr is a JavaScript library that offers an autocomplete feature searching multiple fields and faceted search using tag clouds to interface with Solr. It follows the MVC pattern and can be deployed by customizing the configuration and fields used in examples/reuters-requirejs.
PDO (PHP Data Objects) provides a common interface for accessing databases in PHP. It uses prepared statements to separate SQL structures from user-supplied input, improving security and performance. PDO supports databases like MySQL, PostgreSQL, SQLite, and Oracle. It offers flexible fetching of query results as arrays, objects, or callbacks. PDO also includes features like transactions, metadata retrieval, and error handling via exceptions.
Django's ORM is extremely powerful, allowing you to manage your data without ever going near a line of SQL and hiding a multitude of complexities. But its power can sometimes be a curse rather than a blessing, multiplying queries without your knowledge and bringing your database to its knees.
In this session I explain what's going on behind the scenes and present some techniques to make your ORM use more efficient, showing how to monitor what's going on and how to better deal with relationships, indexes and more.
This talk was presented at Europython 2010 in Birmingham.
The document provides an overview of Elasticsearch including that it is easy to install, horizontally scalable, and highly available. It discusses Elasticsearch's core search capabilities using Lucene and how data can be stored and retrieved. The document also covers Elasticsearch's distributed nature, plugins, scripts, custom analyzers, and other features like aggregations, filtering and sorting.
GreenDao is an ORM library that provides high performance for CRUD operations on SQLite databases in Android apps. It uses code generation to map objects to database tables, allowing data to be accessed and queried using objects rather than raw SQL. Some key features include object mapping, query building, caching, and bulk operations. The documentation provides examples of how to set up GreenDao in a project, define entity classes, perform queries, inserts, updates and deletes on objects.
The document discusses using databases to store information persistently. It begins by explaining that variables in a program disappear after execution, so some data needs to be stored externally. Initially, simply writing to a file is proposed, but there are issues with this approach like handling concurrent access and updates.
The document then introduces database management systems (DBMS) like MySQL that can handle these complexities. It provides SQL commands to create and manage databases and tables within MySQL, including inserting, selecting, updating, and deleting data in tables. Primary keys are discussed as a way to uniquely identify table rows. Overall the document serves as an introduction to using a DBMS like MySQL to persistently store and manage data through the use of databases
The document discusses using databases to store information persistently. It begins by explaining that variables in a program disappear after execution, so some data needs to be stored externally. Initially, simply writing to a file is proposed, but there are issues with this approach like handling concurrent access and updates. The document then introduces database management systems (DBMS) like MySQL that can address these problems. It provides basic examples of using SQL commands to CREATE, INSERT, SELECT, UPDATE and DELETE data from databases. Primary keys are also introduced as a way to uniquely identify table rows.
Oracle Developer Day, 20 October 2009, Oracle De Meern, Holland: Oracle Datab...Marco Gralike
The document summarizes new features in Oracle XML Database 11g Release 2. Key features include improvements to the XMLIndex such as support for structured XML indexing of highly structured XML content and binary XML partitioning using range, list, and hash methods. Other improvements include enhanced DBMS_METADATA support for object-relational XMLType structures, improved XBRL support, and RAC protocol server support.
Apache Solr is an open-source enterprise search platform that provides fast, scalable, and reliable full-text search functionality. It powers the search capabilities of many large websites and applications. Some key features of Solr include fast indexing and search, faceted search, autocomplete, geospatial search, and integration with various databases and applications.
Make an object be searchable is difficult? Taking control of search result's ranking is even more challenging.
In BarCamp Bangkhen 2015 (Nov 22, 2015), Suparit Krityakien, Co-founder & Software Architecture of Wongnai (http://www.wongnai.com) shared his experiences on applying Wongnai ranking algorithm to take control of our search engine's scoring systematically as well as what can be tuned to make it smarter.
This presentation is to furnish you with fundamental knowledge required when tweaking ranking on your search.
This document provides an overview of PostgreSQL, an open-source object-relational database management system. It discusses differences between PostgreSQL and MySQL, how to install PostgreSQL with PHP, unique features like PL/pgSQL procedural languages for writing functions and stored procedures, and table inheritance. Examples of PL/pgSQL code are provided to demonstrate functions, stored procedures, and returning complex data types.
SQL is a standard language for storing, manipulating and retrieving data in databases. It is used by many database systems like Oracle, Microsoft SQL Server, MySQL, and PostgreSQL. SQL allows users to define data structures, manipulate data, and control access to databases. The language includes commands like SELECT to query data, INSERT to add new data, UPDATE to modify existing data, and DELETE to remove data from databases.
supporting t-sql scripts for Heap vs clustered tableMahabubur Rahaman
The document contains SQL scripts that:
1. Perform various diagnostic checks on indexes such as retrieving index status, enabling/disabling tracing, and viewing index usage statistics.
2. Create tables with different indexing configurations like no indexes, non-clustered indexes, and clustered indexes to test query performance.
3. Query system catalog views and dynamic management functions to retrieve detailed metadata about indexes including column definitions, storage details, usage statistics, and unused indexes.
Starting with JSON Path Expressions in Oracle 12.1.0.2Marco Gralike
This document discusses new features in Oracle Database 12c that allow it to be used as a JSON document store. Key features include the ability to store and index JSON documents, access and query JSON data via SQL operators and functions, load JSON documents into the database using SQL*Loader, and index JSON columns to enable faster querying. The document provides details on the JSON operators, functions, indexing options, and how to validate and retrieve JSON content from the database.
Slick: Bringing Scala’s Powerful Features to Your Database Access Rebecca Grenier
This talk will teach you how to use Slick in practice, based on our experience at EatingWell Media Group. Slick is a totally different (and better!) relational database mapping tool that brings Scala’s powerful features to your database interactions, namely: static-checking, compile-time safety, and compositionality.
Here at EatingWell, we have learned quite a bit about Slick over the past two years as we transitioned from a PHP website to Scala. I will share with you tips and tricks we have learned, as well as everything you need to get started using Slick in your Scala application.
I will begin with Slick fundamentals: how to get started making your connection, the types of databases it can access, how to actually create table objects and make queries to and from them. We will using these fundamentals to demonstrate the powerful features inherited from the Scala language itself: static-checking, compile-time safety, and compositionality. And throughout I will share plenty of tips that will help you in everything from getting started to connection pooling options and configuration for use at scale.
UKOUG Tech14 - Getting Started With JSON in the DatabaseMarco Gralike
Presentation used during the UKOUG Tech14 conference in Liverpool (UK) discussing possibilities of the use of, and explaining, the new JSON database functionality in the Oracle 12.1.0.2 database
Introduction to Solr. A brief introduction to Solr for the resources who wants to get trained on Solr.
1. Introduction to Solr
2. Solr Terminologies
3.Installation and Configuration
4. Configuration files schema.xml and solrconfig.xml
5. Features of SOLR
a. Hit Highlighting
Auto Complete / Suggester
Stop words
Synonyms
SpellCheck
Geo Spatial Search
Result Grouping
Query Syntax
Query Boosting
Content Spotlighting
Block Record / Remove URL Feature
Content Spotlighting / Merchandising / Banner / Elevate
Block Record / Remove URL Feature
6. Indexing the Data
7. Search Queries
8. DataImportHandler - DIH
9. Plugins to index various types of Data (XML, CSV, DB, Filesystem)
10. Solr Client APIs
11. Overview of SOLRJ API
12. Running Solr on Tomcat
13. Enabling SSL on Solr
14. Zookeeper Configuration
15. Solr Cloud Deployment
16. Production Indexing Architecture
17. Production Serving Architecture
18. Solr Upgradation
19. References
The document summarizes Simon Willison's presentation on advanced Django topics including unit testing, newforms, Ajax, and OpenID. Key points include:
- Unit testing in Django is encouraged through features like fixtures, doctests, test client, and email capture. Tests should be written before code using test-driven development.
- Newforms provide a declarative way to create forms with fields, validation, rendering, and conversion to Python types. Forms make it easy to display, validate, and process user submissions.
- The test client allows simulating a browser to test views and templates. Ajax functionality can return HTML, XML, or JSON and be integrated with JavaScript libraries like jQuery.
- Form validation
The Query the Whole Query and Nothing but the QueryChris Olbekson
The document discusses WordPress queries and the $wp_query object. It explains that $wp_query represents the main query and stores information about the current request. It advises against using query_posts() to alter the main query, as it overrides $wp_query and can cause issues. Instead, it recommends using pre_get_posts to modify the main query or creating a secondary query with WP_Query or get_posts() for other queries. It also notes some best practices for resetting post data when performing secondary queries.
The document discusses MySQL query optimization. It covers the query optimizer, principles of optimization like using EXPLAIN and profiling, indexes, JOIN optimization, and ORDER BY/GROUP BY optimization. The key points are to identify bottlenecks, use indexes on frequently filtered fields, avoid indexes on fields that change often or contain many duplicates, and consider composite indexes to cover multiple queries.
We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you practical, step-by-step knowledge to turn a performance bottleneck into an epic win!
Solr is an open source enterprise search platform built on Apache Lucene. It provides full-text search, hit highlighting, faceted search, and handles various document formats. Ajax-Solr is a JavaScript library that offers an autocomplete feature searching multiple fields and faceted search using tag clouds to interface with Solr. It follows the MVC pattern and can be deployed by customizing the configuration and fields used in examples/reuters-requirejs.
PDO (PHP Data Objects) provides a common interface for accessing databases in PHP. It uses prepared statements to separate SQL structures from user-supplied input, improving security and performance. PDO supports databases like MySQL, PostgreSQL, SQLite, and Oracle. It offers flexible fetching of query results as arrays, objects, or callbacks. PDO also includes features like transactions, metadata retrieval, and error handling via exceptions.
Django's ORM is extremely powerful, allowing you to manage your data without ever going near a line of SQL and hiding a multitude of complexities. But its power can sometimes be a curse rather than a blessing, multiplying queries without your knowledge and bringing your database to its knees.
In this session I explain what's going on behind the scenes and present some techniques to make your ORM use more efficient, showing how to monitor what's going on and how to better deal with relationships, indexes and more.
This talk was presented at Europython 2010 in Birmingham.
The document provides an overview of Elasticsearch including that it is easy to install, horizontally scalable, and highly available. It discusses Elasticsearch's core search capabilities using Lucene and how data can be stored and retrieved. The document also covers Elasticsearch's distributed nature, plugins, scripts, custom analyzers, and other features like aggregations, filtering and sorting.
GreenDao is an ORM library that provides high performance for CRUD operations on SQLite databases in Android apps. It uses code generation to map objects to database tables, allowing data to be accessed and queried using objects rather than raw SQL. Some key features include object mapping, query building, caching, and bulk operations. The documentation provides examples of how to set up GreenDao in a project, define entity classes, perform queries, inserts, updates and deletes on objects.
The latest version of my PostgreSQL introduction for IL-TechTalks, a free service to introduce the Israeli hi-tech community to new and interesting technologies. In this talk, I describe the history and licensing of PostgreSQL, its built-in capabilities, and some of the new things that were added in the 9.1 and 9.2 releases which make it an attractive option for many applications.
Talk given for the #phpbenelux user group, March 27th in Gent (BE), with the goal of convincing developers that are used to build php/mysql apps to broaden their horizon when adding search to their site. Be sure to also have a look at the notes for the slides; they explain some of the screenshots, etc.
An accompanying blog post about this subject can be found at http://www.jurriaanpersyn.com/archives/2013/11/18/introduction-to-elasticsearch/
PostgreSQL - It's kind've a nifty databaseBarry Jones
This presentation was given to a company that makes software for churches that is considering a migration from SQL Server to PostgreSQL. It was designed to give a broad overview of features in PostgreSQL with an emphasis on full-text search, various datatypes like hstore, array, xml, json as well as custom datatypes, TOAST compression and a taste of other interesting features worth following up on.
Oracle Database 12c - New Features for Developers and DBAsAlex Zaballa
Oracle Database 12c includes over 500 new features designed to support cloud computing, big data, security, and availability. Key features include support for up to 4096 pluggable databases, hot cloning without placing the source database in read-only mode, sharding capabilities, in-memory column storage, application containers, improved resource management isolation, and AWR support on Active Data Guard databases. Other notable features include enhanced JSON support, data redaction for security, row limits and offsets for queries, invisible columns, SQL text expansion, PL/SQL from SQL, session-level sequences, extended data types up to 32K, multiple indexes on the same columns, READ privileges without row locking ability, session private statistics for global temporary tables,
Oracle Database 12c - New Features for Developers and DBAsAlex Zaballa
This document summarizes a presentation about new features in Oracle Database 12c for developers and DBAs. It introduces JSON support, data redaction, SQL query row limits and offsets, invisible columns, extended data types, session level sequences, and more. Demo sections are included to illustrate several of the new features.
To scale or not to scale: Key/Value, Document, SQL, JPA – What’s right for my...Uri Cohen
This presentation will focuses on the various data and querying models available in today’s distributed data stores landscape. It reviews what models and APIs are available and discusses the capabilities each of them provides, the applicable use cases and what it means for your application’s performance and scalability.
I will show how to use Go's database/sql package, with MySQL as an example. Although the documentation is good, it's dense. I'll discuss idiomatic database/sql code, and cover some topics that can save you time and frustration, and perhaps even prevent serious mistakes.
Oracle Database 12c New Features for Developers and DBAs - OTN TOUR LA 2015Alex Zaballa
Oracle Database 12c includes over 500 new features designed for cloud computing, big data, security, and availability. Key features include Multitenant architecture which allows multiple databases to share a single database instance, In-Memory which stores frequently accessed data in memory for faster queries, and new security features like data redaction and encryption. Additional features improve performance, manageability, and flexibility of the database like JSON support, heat map statistics, and identity columns.
This document provides an overview of an introductory training session on SQLite, a popular database for Internet of Things (IoT) applications. The agenda covers installing and configuring SQLite, basic commands like .tables and .schema, accessing databases using ATTACH and DETACH, data types, operators, and SQL statements like SELECT, INSERT, UPDATE, and DELETE. The session teaches the basics of using SQLite through examples of commands, queries, and making changes to databases.
PostgreSQL (or Postgres) began its life in 1986 as POSTGRES, a research project of the University of California at Berkeley.
PostgreSQL isn't just relational, it's object-relational.it's object-relational. This gives it some advantages over other open source SQL databases like MySQL, MariaDB and Firebird.
MYSQL Query Anti-Patterns That Can Be Moved to SphinxPythian
This document provides an overview and summary of MySQL and Sphinx search capabilities. It discusses some limitations of MySQL for certain queries and how Sphinx can help address those limitations by offloading search queries and enabling features like full-text search and geospatial search. The document also covers how to install, configure, and query Sphinx including indexing data from MySQL, running the Sphinx daemon, and connecting to it via SphinxQL or APIs.
This document summarizes a talk about making Django and NoSQL databases like MongoDB play nicely together. Currently, Django's ORM is optimized for SQL databases and makes assumptions that don't always apply to NoSQL databases. The talk proposes some changes to address this, including having the Query object do less database-specific work and pushing more of that down to the individual database compilers. This would make the Query more agnostic and allow the compilers to generate queries optimized for their specific databases. An example backend for MongoDB would be built to demonstrate this approach.
The document provides an overview of using the Drupal database API for interacting with the Drupal database. It covers basics of db_query and dynamic queries using db_select. Key points include using placeholders in queries, working with result sets, and more advanced topics like joins, sorting, conditional statements, and query tagging. The document suggests considering the database API as an alternative to Views when custom queries or aggregated data are needed that may require complex Views configuration.
10 Reasons to Start Your Analytics Project with PostgreSQLSatoshi Nagayasu
PostgreSQL provides several advantages for analytics projects:
1) It allows connecting to external data sources and performing analytics queries across different data stores using features like foreign data wrappers.
2) Features like materialized views, transactional DDLs, and rich SQL capabilities help build effective data warehouses and data marts for analytics.
3) Performance optimizations like table partitioning, BRIN indexes, and parallel queries enable PostgreSQL to handle large datasets and complex queries efficiently.
This document provides an overview of Apache Tajo, an open source distributed SQL query engine for large-scale data analytics on Hadoop. It discusses that Tajo supports ANSI SQL, batch and interactive queries, various data formats/storage including HDFS, HBase and S3. It also summarizes Tajo's architecture, use cases, SQL features like DDL, queries, indexes and storage integration with HBase.
This document discusses using the Drupal database API to interact with the database in Drupal. It covers the basics of db_query and dynamic queries using db_select. It discusses how to add conditions, joins, sorting, and other clauses to dynamic queries. It also covers how to work with result sets and how to perform inserts, updates, and deletes using the API. Finally, it discusses when to use the database API versus views and provides a case study comparing the two approaches.
This document provides an overview of the relational database language SQL. It begins by defining basic data types in SQL like numbers, characters, dates and times. It then discusses the SQL Data Definition Language (DDL) which is used to define and modify database schemas and objects. Specific DDL commands like CREATE, DROP, ALTER, TRUNCATE and RENAME are described. The document also covers the SQL Data Manipulation Language (DML) including commands for queries, inserts, updates and deletes. Additional topics discussed include constraints, indexes, views and the advantages of SQL.
Similar to Scrap your query boilerplate with specql (20)
8 Best Automated Android App Testing Tool and Framework in 2024.pdfkalichargn70th171
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
Mobile app Development Services | Drona InfotechDrona Infotech
Drona Infotech is one of the Best Mobile App Development Company In Noida Maintenance and ongoing support. mobile app development Services can help you maintain and support your app after it has been launched. This includes fixing bugs, adding new features, and keeping your app up-to-date with the latest
Visit Us For :
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsPeter Muessig
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
Transform Your Communication with Cloud-Based IVR SolutionsTheSMSPoint
Discover the power of Cloud-Based IVR Solutions to streamline communication processes. Embrace scalability and cost-efficiency while enhancing customer experiences with features like automated call routing and voice recognition. Accessible from anywhere, these solutions integrate seamlessly with existing systems, providing real-time analytics for continuous improvement. Revolutionize your communication strategy today with Cloud-Based IVR Solutions. Learn more at: https://thesmspoint.com/channel/cloud-telephony
Unveiling the Advantages of Agile Software Development.pdfbrainerhub1
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Microservice Teams - How the cloud changes the way we workSven Peters
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
2. WHAT IS SPECQL?
› The combination of clojure.spec, PostgreSQL and the power of
macros
clojure.spec PostgreSQL
λ
3. SOME BACKGROUND
› Yesql/Jeesql/Hugsql are all fine and specql does not try to ”solve”
SQL so that you don’t have to use it anymore
› When you have multiple slightly different queries, there’s
duplication or fetching too much
› Adding dynamic WHERE clauses leads to ugly NULL checking of
bound parameters
4. EXAMPLE
› SELECT it.foo, it.bar, it.baz
› FROM interesting_table it
› WHERE it.archived = FALSE
› AND it.organization = :user_org_id
› AND it.category IN (:categories)
5. EXAMPLE
› SELECT it.foo, it.bar, it.baz
› FROM interesting_table it
› WHERE it.archived = FALSE
› AND it.organization = :user_org_id
› AND it.category IN (:categories)
How to make
variants without
duplication or
selecting too much?
6. EXAMPLE
› SELECT it.foo, it.bar, it.baz
› FROM interesting_table it
› WHERE it.archived = FALSE
› AND it.organization = :user_org_id
› AND it.category IN (:categories)
How to
add/remove
clauses
dynamically?
7. EXAMPLE
› SELECT it.foo, it.bar, it.baz,
› o.name AS organization_name
› FROM interesting_table it
› JOIN organization o ON it.organization_id = o.id
› WHERE it.archived = FALSE
› AND it.organization = :user_org_id
› AND it.category IN (:categories)
8. EXAMPLE
› SELECT it.foo, it.bar, it.baz,
› o.name AS organization_name
› FROM interesting_table it
› JOIN organization o ON it.organization_id = o.id
› WHERE it.archived = FALSE
› AND it.organization = :user_org_id
› AND it.category IN (:categories)
How to cleanly add
these without code
duplication?
9. THE SPECQL WAY: DEFINE
› (define-tables db
› [”interesting_table” ::it/interesting
› {::it/organization (rel/has-one
› ::it/organization_id
› ::org/organization
› ::org/id)}])
10. THE SPECQL WAY: DEFINE
› (define-tables db
› [”interesting_table” ::it/interesting
› {::it/organization (rel/has-one
› ::it/organization_id
› ::org/organization
› ::org/id)}])
The main macro
for defining things
11. THE SPECQL WAY: DEFINE
› (define-tables db
› [”interesting_table” ::it/interesting
› {::it/organization (rel/has-one
› ::it/organization_id
› ::org/organization
› ::org/id)}])
The name of the
table in the
database
12. THE SPECQL WAY: DEFINE
› (define-tables db
› [”interesting_table” ::it/interesting
› {::it/organization (rel/has-one
› ::it/organization_id
› ::org/organization
› ::org/id)}])
The namespaced
keyword for the
table
13. THE SPECQL WAY: DEFINE
› (define-tables db
› [”interesting_table” ::it/interesting
› {::it/organization (rel/has-one
› ::it/organization_id
› ::org/organization
› ::org/id)}])
Additional
definitions for
columns
22. THE SPECQL WAY: JOIN
› ;; => ({::it/foo 1
› ::it/bar ”example”
› ::it/baz true
› ::it/organization {::org/id 1
› ::org/name ”Acme Inc”}
› …)
Joined entity is
available in a nested
map
23. SPECQL BENEFITS
› Clojure data!
› The column set is just data and can be easily manipulated
• Even given as parameters from the frontend
› Where clauses can be easily added and are (mostly) just data as well
› Every table and column has a single ns keyword definition
• No more ”order” vs ”order-id” vs ”ord” differences in returned query
results
• Namespaced keys are the way of the future
24. SPECQL BENEFITS
› Clojure data!
› The column set is just data and can be easily manipulated
• Even given as parameters from the frontend
› Where clauses can be easily added and are (mostly) just data as well
› Every table and column has a single ns keyword definition
• No more ”order” vs ”order-id” vs ”ord” differences in returned query
results
• Namespaced keys are the way of the future HERE TO STAY
25. SPECQL BENEFITS
› Works with ClojureScript
• The spec generation part, put your specs in .cljc files and enjoy the same
specs and have your db definitions drive your frontend as well
› Works well with ”typed document storage” pattern
• A column can be a user defined type or array
› Provides generic upsert!
• No need to branch code, let the db handle it
› Works with database VIEWs
26. STATUS
› The github project page (https://github.com/tatut/specql) still says
EXPERIMENTAL
• Current 0.7 alpha stage, should be ready this year
• We are already using it in production in Harja
› No concrete promises before the experimental flag is gone, but the
API should only grow
• The test suite is comprehensive and should not break
27. STATUS
› 0.7 is coming and adds support for stored procedures
• Macro for defining a stored procedure as a function
• Better JOIN handling
• Currently fetching multiple ”has many” collections at once doesn’t work properly
› How to help
• Please try it out if you are using PostgreSQL
• Report rough edges, I try to provide good error messages
Specql is a new library for using PostgreSQL from a Clojure application. It introspects your database at compile time (with the help of some macro magic) and provides clojure.spec specs for your tables and their columns.
In addition to creating specs, Specql also retains runtime information so that it can provide generic query and update operations. More on those later.
We have found that slightly different queries are painful to maintain.
Hand written SQL also suffers from the problem of having different keys for different table columns. One query might use :order and another might use :orderid. There is no single truth of what a keyword means.
Let's compare by an example. Consider some table called interesting_table that we want to query from.
How can we easily vary the set of columns we fetch without having multiple almost identical queries or fetching everything and selecting the wanted keys on the caller.
These two problems, in our experience, lead to making similar queries or make the queries you write awkwardly complicated.
And we haven’t even talked about JOINs yet. What if I have two similar queries, but the other one needs to join a table?
Adding columns from a JOINed table further changes how our SQL query looks. Making it difficult to even see that they are conceptually the same.
First we define our tables so specql can generate the specs and record information about it.
First we define our tables so specql can generate the specs and record information about it.
First we define our tables so specql can generate the specs and record information about it.
The spec for the table (a keys spec) is generated for the keyword specified here. All column specs will be generated in the same namespace.
The column options can be used to do transformations for the columns (like converting an enum to keywords on the fly).
In this example we are defining a JOIN specification. The organization virtual column is a one to one JOIN to the organization table.
Specql provides a function called fetch that can be used to make queries against any previously defined table.
The table is simply the keyword we registered the table under in a previous call to define-tables
The columns is just a Clojure set of the namespaced keywords of the table columns
The where map is interesting. The keys are columns of the table being queried the values can either be values (which are simple equality checks) or operators. The basic SQL operators are provided (equality, less than, greater than, in, like) and there is a protocol for you to define more.
Multiple maps of where clauses can be combined with op/and and op/or operators.
Finally the result is a sequence of maps which contain the namespaced keys.
How do we add the JOIN?
We simply add a vector to the column set. The first element is the join definition we defined in define-tables and the second element is a set of columns to retrieve from the join table.
We simply add a vector to the column set. The first element is the join definition we defined in define-tables and the second element is a set of columns to retrieve from the join table.
We simply add a vector to the column set. The first element is the join definition we defined in define-tables and the second element is a set of columns to retrieve from the join table.
ClojureScript: speql understands some of the constraints on fields: NOT NULL and string lengths.
PostgreSQL columns can be arbitrarily complex objects. You can have an array of a user defined type as a column and specql will work great with that.
Upsert:
Upsert, which came in pg 9.5, is fully supported.
Upsert takes the db, the table keyword, a record map to insert and a where clause for the update
Optionally the unique column set to upsert on can also be specified, and defaults to the primary key
Views: you can write your complicated queries as views that and spec those like any table.
Although it is a very young library, we are already using it in production in Harja, which is a large public sector application for road infastructure maintenance.
We started adding specql to our new features and it has been helpful.