PostgreSQL is the World’s Most Advanced Open Source Relational Database and by the end of this talk you will understand what that means for you, an application developer. What kind of problems PostgreSQL can solve for you, and how much you can rely on PostgreSQL in your daily activities, including unit-testing.
Python and PostgreSQL: Let's Work Together! | PyConFr 2018 | Dimitri FontaineCitus Data
Python is often used to maintain application backends. When the backend should implement user oriented workflows, it may rely on a RDBMS component to take care of the system's integrity.
PostgreSQL is the world's most advanced open source relational database, and is very good at taking care of your system's integrity. PostgreSQL also comes with a ton of data processing power, and in many cases a simple enough SQL statement may replace hundreds of lines of code written in Python.
In this talk, we learn advanced SQL techniques and how to reason about which part of the backend code should be done in the database, and which parf of the backend code is so easier to write as a SQL query.
(KO) <<2019 데이터야놀자>> 에서 발표한 내용입니다.
사회적으로 문제가 되고 있는 어뷰징 분석과 이를 방지할 수 있는 방법에 대해 정리해보았습니다.
(EN) Presented in Datayanolja 2019 (Domestic data conference).
Dealing with one of the big social issues: manipulation of public opinions in online news platform.
Main contributions are two-fold: 1) identifying manipulators 2) suggesting possible 3 solutions
* notice: The material is written in Korean.
A talk on Data Science in Piano, contains the following:
1. Tips on how to make sure your data are analysis-friendly
2. A short introduction into how to do data science with a for loop (partially stolen from https://goo.gl/wHwZKv)
3. A brief look on output evolution for paywall health check for our clients (publishers)
4. A sneak peek into challenges we face currently
Python and PostgreSQL: Let's Work Together! | PyConFr 2018 | Dimitri FontaineCitus Data
Python is often used to maintain application backends. When the backend should implement user oriented workflows, it may rely on a RDBMS component to take care of the system's integrity.
PostgreSQL is the world's most advanced open source relational database, and is very good at taking care of your system's integrity. PostgreSQL also comes with a ton of data processing power, and in many cases a simple enough SQL statement may replace hundreds of lines of code written in Python.
In this talk, we learn advanced SQL techniques and how to reason about which part of the backend code should be done in the database, and which parf of the backend code is so easier to write as a SQL query.
(KO) <<2019 데이터야놀자>> 에서 발표한 내용입니다.
사회적으로 문제가 되고 있는 어뷰징 분석과 이를 방지할 수 있는 방법에 대해 정리해보았습니다.
(EN) Presented in Datayanolja 2019 (Domestic data conference).
Dealing with one of the big social issues: manipulation of public opinions in online news platform.
Main contributions are two-fold: 1) identifying manipulators 2) suggesting possible 3 solutions
* notice: The material is written in Korean.
A talk on Data Science in Piano, contains the following:
1. Tips on how to make sure your data are analysis-friendly
2. A short introduction into how to do data science with a for loop (partially stolen from https://goo.gl/wHwZKv)
3. A brief look on output evolution for paywall health check for our clients (publishers)
4. A sneak peek into challenges we face currently
The consumer product landscape, particularly among e-commerce firms, includes a bevy of subscription-based business models. Internet and mobile phone subscriptions are now commonplace and joining the ranks are dietary supplements, meals, clothing, cosmetics and personal grooming products.
Standard metrics to diagnose a healthy consumer-brand relationship typically include customer purchase frequency and ultimately, retention of the customer demonstrated by regular purchases. If a brand notices that a customer isn’t purchasing, it may consider targeting the customer with discount offers or deploying a tailored messaging campaign in the hope that the customer will return and not “churn”.The churn diagnosis, however, becomes more complicated for subscription-based products, many of which offer multiple delivery frequencies and the ability to pause a subscription. Brands with subscription-based products need to have some reliable measure of churn propensity so they can further isolate the factors that lead to churn and preemptively identify at-risk customers.
CARTO en 5 Pasos: del Dato a la Toma de Decisiones [CARTO]CARTO
En este webinar repasamos - mediante una demostración con el mercado de Real Estate de Los Angeles como ejemplo - cada uno de los cinco pasos que la plataforma de CARTO sigue para una toma de decisiones eficaz basada en los datos.
Watch it now at: https://go.carto.com/carto-pasos-dato-toma-decisiones-recorded
KDD Analytics provides expertise in marketing predictive analytics and insightful dashboards for management striving for better data driven solutions. KDD is pioneering the use of ai-one's Analyst Toolbox for conversion of unstructured text documents into exciting BI visualizations.
Alex Shaw III - Information Technology PortfolioAlexShawIII
My portfolio showcases my work in technology, particularly data modeling and presentation technologies. I also love and include game development and STEAM tools.
Public Construction Company Overviews Q1 2017Jonathan Hunt
The latest in Star America Capital Advisors' summaries of U.S. based publicly traded construction companies. This presentation includes Q1 2017 financial highlights and news from the following contractors: Tutor Perini, Granite, Sterling, AECOM, Emcor, and MYR Group.
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Citus Data
A story about powering a 1.5 petabyte internal analytics application at Microsoft with 2816 cores and 18.7 TB of memory in the Citus cluster.
The internal RQV analytics dashboard at Microsoft helps the Windows team to assess the quality of upcoming Windows releases. The system tracks 20,000 diagnostic and quality metrics, digests data from 800 million Windows devices and currently supports over 6 million queries per day, with hundreds of concurrent users. The RQV analytics dashboard relies on Postgres—along with the Citus extension to Postgres to scale out horizontally—and is deployed on Microsoft Azure.
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Citus Data
As a developer using PostgreSQL one of the most important tasks you have to deal with is modeling the database schema for your application. In order to achieve a solid design, it’s important to understand how the schema is then going to be used as well as the trade-offs it involves.
As Fred Brooks said: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.”
In this talk we're going to see practical normalisation examples and their benefits, and also review some anti-patterns and their typical PostgreSQL solutions, including Denormalization techniques thanks to advanced Data Types.
More Related Content
Similar to The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine
The consumer product landscape, particularly among e-commerce firms, includes a bevy of subscription-based business models. Internet and mobile phone subscriptions are now commonplace and joining the ranks are dietary supplements, meals, clothing, cosmetics and personal grooming products.
Standard metrics to diagnose a healthy consumer-brand relationship typically include customer purchase frequency and ultimately, retention of the customer demonstrated by regular purchases. If a brand notices that a customer isn’t purchasing, it may consider targeting the customer with discount offers or deploying a tailored messaging campaign in the hope that the customer will return and not “churn”.The churn diagnosis, however, becomes more complicated for subscription-based products, many of which offer multiple delivery frequencies and the ability to pause a subscription. Brands with subscription-based products need to have some reliable measure of churn propensity so they can further isolate the factors that lead to churn and preemptively identify at-risk customers.
CARTO en 5 Pasos: del Dato a la Toma de Decisiones [CARTO]CARTO
En este webinar repasamos - mediante una demostración con el mercado de Real Estate de Los Angeles como ejemplo - cada uno de los cinco pasos que la plataforma de CARTO sigue para una toma de decisiones eficaz basada en los datos.
Watch it now at: https://go.carto.com/carto-pasos-dato-toma-decisiones-recorded
KDD Analytics provides expertise in marketing predictive analytics and insightful dashboards for management striving for better data driven solutions. KDD is pioneering the use of ai-one's Analyst Toolbox for conversion of unstructured text documents into exciting BI visualizations.
Alex Shaw III - Information Technology PortfolioAlexShawIII
My portfolio showcases my work in technology, particularly data modeling and presentation technologies. I also love and include game development and STEAM tools.
Public Construction Company Overviews Q1 2017Jonathan Hunt
The latest in Star America Capital Advisors' summaries of U.S. based publicly traded construction companies. This presentation includes Q1 2017 financial highlights and news from the following contractors: Tutor Perini, Granite, Sterling, AECOM, Emcor, and MYR Group.
Architecting peta-byte-scale analytics by scaling out Postgres on Azure with ...Citus Data
A story about powering a 1.5 petabyte internal analytics application at Microsoft with 2816 cores and 18.7 TB of memory in the Citus cluster.
The internal RQV analytics dashboard at Microsoft helps the Windows team to assess the quality of upcoming Windows releases. The system tracks 20,000 diagnostic and quality metrics, digests data from 800 million Windows devices and currently supports over 6 million queries per day, with hundreds of concurrent users. The RQV analytics dashboard relies on Postgres—along with the Citus extension to Postgres to scale out horizontally—and is deployed on Microsoft Azure.
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Citus Data
As a developer using PostgreSQL one of the most important tasks you have to deal with is modeling the database schema for your application. In order to achieve a solid design, it’s important to understand how the schema is then going to be used as well as the trade-offs it involves.
As Fred Brooks said: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.”
In this talk we're going to see practical normalisation examples and their benefits, and also review some anti-patterns and their typical PostgreSQL solutions, including Denormalization techniques thanks to advanced Data Types.
JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...Citus Data
When do you use jsonb, and when don’t you? How do you make it fast? What operators are available, and what can they do? How will this change? These are all very good questions, but jsonb support in Postgres moves so fast that it’s hard to keep up.
In this talk, you will get details on these topics, complete with practical examples and real-world stories:
- When to use jsonb, what it’s good for, and when to not use it
- Operators and how to use them effectively
- Indexing, operator support for indexes, and the tradeoffs involved
- Postgres 12 improvements and new features
Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...Citus Data
One of the strongest features of any database is its extensibility and PostgreSQL comes with a rich extension API. It allows you to define new functions, types, and operators. It even allows you to modify some of its core parts like planner, executor or storage engine. You read it right, you can even change the behavior of PostgreSQL planner. How cool is that?
Such freedom in extensibility created strong extension community around PostgreSQL and made way for a vast amount of extensions such as pg_stat_statements, citus, postgresql-hll and many more.
In this tutorial, we will look at how you can create your own PostgreSQL extension. We will start with more common stuff like defining new functions and types but gradually explore less known parts of the PostgreSQL's extension API like C level hooks which lets you change the behavior of planner, executor and other core parts of the PostgreSQL. We will see how to code, debug, compile and test our extension. After that, we will also look into how to package and distribute our extension for other people to use.
To get the best benefit from the tutorial, C and SQL knowledge would be beneficial. Some knowledge on PostgreSQL internals would also be useful but we will cover the necessary details, so it is not necessary.
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
Postgres is a powerful database, it continues to improve in terms of performance, extensibility, and more broadly in features. However it is not perfect.
Here I'll cover a highly opinionated view of all the areas Postgres falls flat, with some rough thought ideas on how we can make it better. Opinions are all informed by 10 years of interacting with customers running literally millions of databases for users.
When it all goes wrong | PGConf EU 2019 | Will LeinweberCitus Data
You're woken up in the middle of the night to your phone. Your app is down and you're on call to fix it. Eventually you track it down to "something with the db," but what exactly is wrong? And of course, you're sure that nothing changed recently…
Knowing what to fix, and even where to start looking, is a skill that takes a long time to develop. Especially since Postgres normally works very well for months at a time, not letting you get practice!
In this talk, I'll share not only the more common failure cases and how to fix them, but also a general approach to efficiently figuring out what's wrong in the first place.
Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise GrandjoncCitus Data
SQL can seem like an obscure and complex but powerful language. Learning it can be intimidating. As a developer, we can easily be tempted using basic SQL provided by the ORM. But did you know that you can use window functions in some ORMs? Same goes for a lot of other fun SQL functionalities.
In this talk we will explore some advanced SQL features that you might find useful. We will discover the wonderful world of joins (lateral, cross…), subqueries, grouping sets, window functions, common table expressions.
But most importantly this talk is not only a talk to show you how great SQL is. This talk is here to show you how to use it in real life. What are the features supported by your ORM? And how can you use them if they don’t support them?
Wether you know SQL or not, whether you are a developer or a DBA working with developers, you might learn a lot about SQL, ORMs, and application development using Postgres.
What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...Citus Data
Many people have asked us: “Why did Microsoft acquire Citus Data?” and “What do you plan to do with the Citus open source extension to Postgres?” Come join us to see the exciting work we are doing with Postgres and open source at Microsoft.
Deep Postgres Extensions in Rust | PGCon 2019 | Jeff DavisCitus Data
Postgres relies heavily on an extension ecosystem, but that is almost 100% dependent on C; which cuts out developers, libraries, and ideas from the world of Postgres. postgres-extension.rs changes that by supporting development of extensions in Rust. Rust is a memory-safe language that integrates nicely in any environment, has powerful libraries, a vibrant ecosystem, and a prolific developer community.
Rust is a unique language because it supports high-level features but all the magic happens at compile-time, and the resulting code is not dependent on an intrusive or bulky runtime. That makes it ideal for integrating with postgres, which has a lot of its own runtime, like memory contexts and signal handlers. postgres-extension.rs offers this integration, allowing the development of extensions in rust, even if deeply-integrated into the postgres internals, and helping handle tricky issues like error handling. This is done through a collection of Rust function declarations, macros, and utility functions that allow rust code to call into postgres, and safely handle resulting errors.
Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...Citus Data
I spent the early part of my career working on developer tools, operating systems, high-speed file systems, and scale-out storage. Not databases. Frankly, I always thought that databases were a bit boring. So almost 2 years in to my new job at a Postgres company, I continue to be amazed at the enthusiasm of the PostgreSQL developer community and users. I mean, people’s eyes light up when you ask them why they love Postgres. Sure, a lot of us get animated when talking about our newest gadget, or Ronaldo’s phenomenal free-kick goal in the World Cup, or mint chip gelato from La Strega Nocciola—but most platform software simply doesn’t trigger this kind of passion. So why does Postgres? Why is this open source database having such a “moment”? Well, I’ve been trying to understand, looking at this “Postgres moment” from a few different angles. In this talk I’ll share what I’ve observed to be the top 10 business, technology, and community reasons so many of you have so much affection for PostgreSQL.
A story on Postgres index types | PostgresLondon 2019 | Louise GrandjoncCitus Data
Want to know everything about indexes in postgres? Here are the slides for a postgresql talk, and if you want to know more, you can read articles on www.louisemeta.com.
Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...Citus Data
Many in today’s developer world look down on marketing. I mean, after all, the marketing team is usually “not technical.” And they’re not developers. It’s 2019 and while we try to promote inclusiveness of all types, inclusiveness doesn’t seem to apply to marketers. Why? Is that OK? Who does that hurt? I grew up in engineering and spent the first 15 years of my career as a developer or an engineering manager of some type. So now that I’m in marketing, it surprised me when one of my engineering colleagues blurted out “But it’s a technical conference!” when he learned one of my talks was accepted to a technical conference.
This keynote is about why developers really need marketing. About how good marketing managers can make it so visitors to your website don’t leave empty-handed, confused about what your technology actually does or why it matters. About how the ability to translate technology into what-users-actually-care-about can make your project be the one that takes off. About why Dormain Drewitz said at Monktoberfest: “I work in product marketing. My preferred programming language is English.” Finally, this talk explores how to be sensitive to the bias against marketing that pervades some of our teams—and how to instead embrace teamwork best practices employed by sailors, where everyone in the boat has an important role to play if you are to win the race.
The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri FontaineCitus Data
PostgreSQL is the World’s Most Advanced Open Source Relational Database and by the end of this talk you will understand what that means for you, an application developer. What kind of problems PostgreSQL can solve for you, and how much you can rely on PostgreSQL in your daily activities, including unit-testing.
Optimizing your app by understanding your Postgres | RailsConf 2019 | Samay S...Citus Data
I’m a Postgres person. Period. After talking to many Rails developers about their application performance, I realized many performance issues can be solved by understanding your database a bit better. So I thought I’d share the statistics Postgres captures for you and how you can use them to find slow queries, un-used indexes, or tables which are not getting vacuumed correctly. This talk will cover Postgres tools and tips for the above, including pgstatstatements, useful catalog tables, and recently added Postgres features such as CREATE STATISTICS.
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberCitus Data
You're woken up in the middle of the night to your phone. Your app is down and you're on call to fix it. Eventually you track it down to "something with the db," but what exactly is wrong? And of course, you're sure that nothing changed recently…
Knowing what to fix, and even where to start looking, is a skill that takes a long time to develop. Especially since Postgres normally works very well for months at a time, not letting you get practice!
In this talk, I'll share not only the more common failure cases and how to fix them, but also a general approach to efficiently figuring out what's wrong in the first place.
Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Liv...Citus Data
Watch Sai Srirampur, Solutions Engineer at Citus Data (now part of the Microsoft family), give a live demo of how you can use Postgres and the Citus extension to Postgres to manage real-time analytics workloads.
View if you & your application need:
>> A relational database that scales for customer-facing analytics dashboards, with real-time data ingest and a large volume of queries
>> A way to scale out Postgres horizontally, to address the performance hiccups you’re experiencing as you run into the resource limits of single-node Postgres
>> A way to roll-up and pre-aggregate data to build fast data pipelines and enable sub-second response times.
>> A way to consolidate your database platforms, to avoid having separate stores for your transactional and analytics workloads
Using a 4-node Citus database cluster in the cloud, Sai will show you how Citus shards Postgres to give you lightning fast performance, at scale. Also featuring rollups.
How to write SQL queries | pgDay Paris 2019 | Dimitri FontaineCitus Data
Most of the time we see finished SQL queries, either in code repositories, blog posts of talk slides. This talk focus on the process of how to write an SQL query, from a problem statement expressed in English to code review and long term maintenance of SQL code.
When it all Goes Wrong |Nordic PGDay 2019 | Will LeinweberCitus Data
You're woken up in the middle of the night to your phone. Your app is down and you're on call to fix it. Eventually you track it down to "something with the db," but what exactly is wrong? And of course, you're sure that nothing changed recently…
Knowing what to fix, and even where to start looking, is a skill that takes a long time to develop. Especially since Postgres normally works very well for months at a time, not letting you get practice!
In this talk, I'll share not only the more common failure cases and how to fix them, but also a general approach to efficiently figuring out what's wrong in the first place.
Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire GiordanoCitus Data
I spent the early part of my career working on developer tools, operating systems, high-speed file systems, and scale-out storage. Not databases. Frankly, I always thought that databases were a bit boring. So one year in to my new job at a Postgres company, I continue to be amazed at the enthusiasm of the PostgreSQL developer community and users. I mean, people’s eyes light up when you ask them why they love Postgres. Sure, a lot of us get animated when talking about our newest iPhone, or Ronaldo’s phenomenal free-kick goal in the World Cup, or mint chip gelato from La Strega Nociola—but most platform software simply doesn’t trigger this kind of passion. So why does Postgres? Why is this open source database having such a “moment”? Why now? Well, I’ve been trying to find out, looking at this “Postgres moment” from a few different angles. In this talk I’ll share what I’ve observed to be the top 10 business, technology, and community reasons so many of you have so much affection for PostgreSQL.
Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...Citus Data
There are a number of data architectures you could use when building a multi-tenant app. Some, such as using one database per customer or one schema per customer. These two options scale to an extent when you have say 10s of tenants. However as you start scaling to hundreds and thousands of tenants, you start running into challenges both from performance and maintenance of tenants perspective. You could solve the above problem by adding the notion of tenancy directly into the logic of your SaaS application. How to implement/automate this in Django-ORM is a challenge? We will talk about how to make the django app tenant aware and at a broader level explain how scale out applications that are built on top of Django ORM and follow a multi tenant data model. We'd take postgresql as our database of choice and the logic/implementation can be extended to any other relational databases as well.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
11. Consistent
Dimitri Fontaine (CitusData) Data Modeling, Normalization and Denormalization March 13, 2018
• Data types
• Constraints
check, not null,
pkey, fkey
• Relations
• SQL
• Schema
create table foo
(
id int,
f1 text
);
20. Daily NYSE Group Volume in
NYSE Listed, 2017
2010 1/4/2010 1,425,504,460 4,628,115 $38,495,460,645
2010 1/5/2010 1,754,011,750 5,394,016 $43,932,043,406
2010 1/6/2010 1,655,507,953 5,494,460 $43,816,749,660
2010 1/7/2010 1,797,810,789 5,674,297 $44,104,237,184
create table factbook
(
year int,
date date,
shares text,
trades text,
dollars text
);
copy factbook from 'factbook.csv' with delimiter E't' null ''
21. Daily NYSE Group Volume in
NYSE Listed, 2017
alter table factbook
alter shares
type bigint
using replace(shares, ',', '')::bigint,
alter trades
type bigint
using replace(trades, ',', '')::bigint,
alter dollars
type bigint
using substring(replace(dollars, ',', '') from 2)::numeric;
27. Monthly Report, SQL
set start '2017-02-01'
select date,
to_char(shares, '99G999G999G999') as shares,
to_char(trades, '99G999G999') as trades,
to_char(dollars, 'L99G999G999G999') as dollars
from factbook
where date >= date :'start'
and date < date :'start' + interval '1 month'
order by date;
29. Monthly Report, Python
def fetch_month_data(year, month):
"Fetch a month of data from the database"
date = "%d-%02d-01" % (year, month)
sql = """
select date, shares, trades, dollars
from factbook
where date >= date %s
and date < date %s + interval '1 month'
order by date;
"""
pgconn = psycopg2.connect(CONNSTRING)
curs = pgconn.cursor()
curs.execute(sql, (date, date))
res = {}
for (date, shares, trades, dollars) in curs.fetchall():
res[date] = (shares, trades, dollars)
return res
def list_book_for_month(year, month):
"""List all days for given month, and for each
day list fact book entry.
"""
data = fetch_month_data(year, month)
cal = Calendar()
print("%12s | %12s | %12s | %12s" %
("day", "shares", "trades", "dollars"))
print("%12s-+-%12s-+-%12s-+-%12s" %
("-" * 12, "-" * 12, "-" * 12, "-" * 12))
for day in cal.itermonthdates(year, month):
if day.month != month:
continue
if day in data:
shares, trades, dollars = data[day]
else:
shares, trades, dollars = 0, 0, 0
print("%12s | %12s | %12s | %12s" %
(day, shares, trades, dollars))
34. Monthly Report, Fixed, SQL
select cast(calendar.entry as date) as date,
coalesce(shares, 0) as shares,
coalesce(trades, 0) as trades,
to_char(
coalesce(dollars, 0),
'L99G999G999G999'
) as dollars
from /*
* Generate the target month's calendar then LEFT JOIN
* each day against the factbook dataset, so as to have
* every day in the result set, whether or not we have a
* book entry for the day.
*/
generate_series(date :'start',
date :'start' + interval '1 month'
- interval '1 day',
interval '1 day'
)
as calendar(entry)
left join factbook
on factbook.date = calendar.entry
order by date;
38. Monthly Report, WoW%, SQL
with computed_data as
(
select cast(date as date) as date,
to_char(date, 'Dy') as day,
coalesce(dollars, 0) as dollars,
lag(dollars, 1)
over(
partition by extract('isodow' from date)
order by date
)
as last_week_dollars
from /*
* Generate the month calendar, plus a week
* before so that we have values to compare
* dollars against even for the first week
* of the month.
*/
generate_series(date :'start' - interval '1 week',
date :'start' + interval '1 month'
- interval '1 day',
interval '1 day'
)
as calendar(date)
left join factbook using(date)
)
select date, day,
to_char(
coalesce(dollars, 0),
'L99G999G999G999'
) as dollars,
case when dollars is not null
and dollars <> 0
then round( 100.0
* (dollars - last_week_dollars)
/ dollars
, 2)
end
as "WoW %"
from computed_data
where date >= date :'start'
order by date;
39. Monthly Report, WoW%, SQL
with computed_data as
(
select cast(date as date) as date,
to_char(date, 'Dy') as day,
coalesce(dollars, 0) as dollars,
lag(dollars, 1)
over(
partition by extract('isodow' from date)
order by date
)
as last_week_dollars
from /*
* Generate the month calendar, plus a week
* before so that we have values to compare
* dollars against even for the first week
* of the month.
*/
generate_series(date :'start' - interval '1 week',
date :'start' + interval '1 month'
- interval '1 day',
interval '1 day'
)
as calendar(date)
left join factbook using(date)
)
select date, day,
to_char(
coalesce(dollars, 0),
'L99G999G999G999'
) as dollars,
case when dollars is not null
and dollars <> 0
then round( 100.0
* (dollars - last_week_dollars)
/ dollars
, 2)
end
as "WoW %"
from computed_data
where date >= date :'start'
order by date;
Window Function, SQL’92
42. Thinking in SQL
•Structured Query Language
•Declarative Programming Language
•Relational Model
•Unix: everything is a file
•Java: everything is an object
•Python: packages, modules, classes, methods
•SQL: relations
43. SQL Relations
•SELECT describes the type of the relation
•Named a projection operator
•Defines SQL Query Attribute domains
•FROM introduces base relations
•Relational Operators compute new relations
•INNER JOIN
•OUTER JOIN
•LATERAL JOIN
•set operators: UNION, EXECPT, INTERSECT
44. SQL Relations
with decades as
(
select extract('year' from date_trunc('decade', date)) as decade
from races
group by decade
)
select decade,
rank() over(partition by decade order by wins desc) as rank,
forename, surname, wins
from decades
left join lateral
(
select code, forename, surname, count(*) as wins
from drivers
join results
on results.driverid = drivers.driverid
and results.position = 1
join races using(raceid)
where extract('year' from date_trunc('decade', races.date))
= decades.decade
group by decades.decade, drivers.driverid
order by wins desc
limit 3
)
as winners on true
order by decade asc, wins desc;
45. Top-3 Pilots by decade
decade │ rank │ forename │ surname │ wins
════════╪══════╪═══════════╪════════════╪══════
1950 │ 1 │ Juan │ Fangio │ 24
1950 │ 2 │ Alberto │ Ascari │ 13
1950 │ 3 │ Stirling │ Moss │ 12
1960 │ 1 │ Jim │ Clark │ 25
1960 │ 2 │ Graham │ Hill │ 14
1960 │ 3 │ Jack │ Brabham │ 11
1970 │ 1 │ Niki │ Lauda │ 17
1970 │ 2 │ Jackie │ Stewart │ 16
1970 │ 3 │ Emerson │ Fittipaldi │ 14
1980 │ 1 │ Alain │ Prost │ 39
1980 │ 2 │ Nelson │ Piquet │ 20
1980 │ 2 │ Ayrton │ Senna │ 20
1990 │ 1 │ Michael │ Schumacher │ 35
1990 │ 2 │ Damon │ Hill │ 22
1990 │ 3 │ Ayrton │ Senna │ 21
2000 │ 1 │ Michael │ Schumacher │ 56
2000 │ 2 │ Fernando │ Alonso │ 21
2000 │ 3 │ Kimi │ Räikkönen │ 18
2010 │ 1 │ Lewis │ Hamilton │ 45
2010 │ 2 │ Sebastian │ Vettel │ 40
2010 │ 3 │ Nico │ Rosberg │ 23
(21 rows)
47. SQL & Developer Tooling
with computed_data as
(
select cast(date as date) as date,
to_char(date, 'Dy') as day,
coalesce(dollars, 0) as dollars,
lag(dollars, 1)
over(
partition by extract('isodow' from date)
order by date
)
as last_week_dollars
from /*
* Generate the month calendar, plus a week before
* so that we have values to compare dollars against
* even for the first week of the month.
*/
generate_series(date :'start' - interval '1 week',
date :'start' + interval '1 month'
- interval '1 day',
interval '1 day'
)
as calendar(date)
left join factbook using(date)
)
select date, day,
to_char(
coalesce(dollars, 0),
'L99G999G999G999'
) as dollars,
case when dollars is not null
and dollars <> 0
then round( 100.0
* (dollars - last_week_dollars)
/ dollars
, 2)
end
as "WoW %"
from computed_data
where date >= date :'start'
order by date;
• Code Integration
• SQL Queries in .sql files
• Parameters
• Result Set To Objects
• A Result Set is a Relation
• Testing
• Unit Testing
• Regression Testing
48. Object Relational Mapping
• The R in ORM
stands for
relation
• Every SQL query
result set is a
relation
• Alternatives:
JOOQ, POMM
49. Integration of SQL as code
YeSQL for Clojure
https://github.com/
krisajenkins/yesql
Also exists for:
• Python
• PHP
• C#
• Javascript
• Erlang
• Ruby
50. Python AnoSQL
$ cat queries.sql
-- name: get-all-greetings
-- Get all the greetings in the database
SELECT * FROM greetings;
-- name: $select-users
-- Get all the users from the database,
-- and return it as a dict
SELECT * FROM USERS;
52. RegreSQL
$ regresql test
Connecting to 'postgres:///chinook?sslmode=disable'… ✓
TAP version 13
ok 1 - src/sql/album-by-artist.1.out
ok 2 - src/sql/album-tracks.1.out
ok 3 - src/sql/artist.1.out
ok 4 - src/sql/genre-topn.top-3.out
ok 5 - src/sql/genre-topn.top-1.out
ok 6 - src/sql/genre-tracks.out
57. Geolocation & earthdistance
with geoloc as
(
select location as l
from location
join blocks using(locid)
where iprange
>>=
'212.58.251.195'
)
select name,
pos <@> l miles
from pubnames, geoloc
order by pos <-> l
limit 10;
name │ miles
═════════════════════╪═══════════════════
The Windmill │ 0.238820308117723
County Hall Arms │ 0.343235607674773
St Stephen's Tavern │ 0.355548630092567
The Red Lion │ 0.417746499125936
Zeitgeist │ 0.395340599421532
The Rose │ 0.462805636194762
The Black Dog │ 0.536202634581979
All Bar One │ 0.489581827372222
Slug and Lettuce │ 0.49081531378207
Westminster Arms │ 0.42400619117691
(10 rows)
58. NBA Games Statistics
“An interesting factoid: the team that recorded
the fewest defensive rebounds in a win was the
1995-96 Toronto Raptors, who beat the
Milwaukee Bucks 93-87 on 12/26/1995 despite
recording only 14 defensive rebounds.”
59. with stats(game, team, drb, min) as (
select ts.game, ts.team, drb, min(drb) over ()
from team_stats ts
join winners w on w.id = ts.game
and w.winner = ts.team
)
select game.date::date,
host.name || ' -- ' || host_score as host,
guest.name || ' -- ' || guest_score as guest,
stats.drb as winner_drb
from stats
join game on game.id = stats.game
join team host on host.id = game.host
join team guest on guest.id = game.guest
where drb = min;
NBA Games Statistics
60. -[ RECORD 1 ]----------------------------
date | 1995-12-26
host | Toronto Raptors -- 93
guest | Milwaukee Bucks -- 87
winner_drb | 14
-[ RECORD 2 ]----------------------------
date | 1996-02-02
host | Golden State Warriors -- 114
guest | Toronto Raptors -- 111
winner_drb | 14
-[ RECORD 3 ]----------------------------
date | 1998-03-31
host | Vancouver Grizzlies -- 101
guest | Dallas Mavericks -- 104
winner_drb | 14
-[ RECORD 4 ]----------------------------
date | 2009-01-14
host | New York Knicks -- 128
guest | Washington Wizards -- 122
winner_drb | 14
Time: 126.276 ms
NBA Games Statistics
61. Pure SQL Histograms
with drb_stats as (
select min(drb) as min,
max(drb) as max
from team_stats
),
histogram as (
select width_bucket(drb, min, max, 9) as bucket,
int4range(min(drb), max(drb), '[]') as range,
count(*) as freq
from team_stats, drb_stats
group by bucket
order by bucket
)
select bucket, range, freq,
repeat('■',
( freq::float
/ max(freq) over()
* 30
)::int
) as bar
from histogram;