This document discusses different types of database partitioning including horizontal, vertical, and file group partitioning. It provides examples and definitions of each type. Key benefits highlighted include improved performance through better query plans, reduced backup sizes, and ability to do partial database restores for high availability. The document also discusses considerations for determining good candidates for partitioning and demonstrates partitioning techniques.
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...Michael Rys
When processing TB and PB of data, running your Big Data queries at scale and having them perform at peak is essential. In this session, we show you some state-of-the art tools on how to analyze U-SQL job performances and we discuss in-depth best practices on designing your data layout both for files and tables and writing performing and scalable queries using U-SQL. You will learn how to analyze performance and scale bottlenecks and will learn several tips on how to make your big data processing scripts both faster and scale better.
Data Warehouse Physical Design,Physical Data Model, Tablespaces, Integrity Constraints, ETL (Extract-Transform-Load) ,OLAP Server Architectures, MOLAP vs. ROLAP, Distributed Data Warehouse ,
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
3. Partitioning Defined
Partitioning (noun)
1. the action or state of dividing or being
divided into parts
2. a structure dividing a space into two parts
Source: http://oxforddictionaries.com/definition/english/partition
4. Types of Partitioning
• Horizontal
• Partition Table by Rows
• Vertical
• Partition Table by Columns
• File Group
• Partition Tables by File Group
5. Horizontal Partitioning
Divides the rows into small sets by
boundaries
Year PK Order ID Product ID Qty Cost Line XML
2010 1 1 1 5 1.5 <Line…
2010 2 1 1 7 2 <Line…
2011 1000000 20000 345 1000 5 <Line…
2011 1000001 20000 347 3000 12 <Line…
2012 2000000 40000 705 8000 3 <Line…
2012 2000001 40001 706 3000 7 <Line…
6. Vertical Partitioning
Divides columns from one table into multiple
tables
Year PK Order ID … Year PK Line XML
2010 1 1 … 2010 1 <Line…
2010 2 1 … 2010 2 <Line…
2011 1000000 345 … 2011 1000000 <Line…
2011 1000001 347 … 2011 1000001 <Line…
2012 2000000 705 … 2012 2000000 <Line…
2012 2000001 706 … 2012 2000001 <Line…
7. File Group Partitioning
File group partitioning is separating the storage of tables
and indexes onto separate database files
During a Primary File Group restore the whole database
will be offline
Separate the system & user tables to allow the quickest
restore time
Since the partitioned database is made up of smaller
parts it is easier to manage the storage location of the
files
8. Using Table Partitioning
Supports Horizontal Partitioning
Partition Function defines boundaries based
on a single column
Partition Scheme defines the File Group
Inserts automatically supported
All partitions share the same definition as the
table (indexes, columns, fill factor etc)
9. Table Level Limitations
Only ~200 steps for each statistics across the
whole table
Online Index rebuilds are for the whole table
not individual partitions
Fill Factor
Lock settings (Row, Page, Escalation, etc)
Indexes are defined for the whole table rather
than partitions
10. Using View Partitioning
View Partitioning allows you to overcome
Table Level Constraints
Check Constraints used to define partitions
Combine all tables with a UNION ALL View
Can be used together with Table Partitioning
Trigger required to allow INSERT with Identity
column
Allows more complex partitioning schemes
11. Candidates for Partitioning
Large vs. Small Tables
Replicated vs. Non-Replicated Tables
Normal vs. BLOB columns
Write Heavy Current data vs. Heavily
Read Historic data
Read Write vs. Read Only Tables
12. Partial Database Availability
Also referred to as Piecemeal Restore
Full Recovery Model is Required
Individual File & File Groups Restore
Only the Primary File Group is required to
restore a database
Allows restoring a subset of a correctly
partitioned database for quicker recovery
from a disaster
13. Tipping Point Why is it Table Scanning
As the number of rows in a table increase the
depth of the B+Tree increases
The number of Page Reads for a lookup is
equal to the depth of the B+Tree
The Tipping Point is the point where a Full
Table Scan requires less Page Reads than
the lookups
14. What Does Partition Give Us
Choices
Using different Tiers of Storage
Allows quicker recovery strategies
Allows partial restores of the database
Performance
Allows reduction of overhead of backups
Allows better query plans due to more
accurate statistics on large tables
15. Demos
Moving Tables to File Groups
Partial Restore + Using Partial Restore to
Initialise Replication
Moving Partition Between Tables For Current
vs. Historic
Show how View Partitioning can present a
single view of all tables & still support
partition elimination
Editor's Notes
Today I will be talking about partitioning which tends to be thought about after the lack of it has caused a problem.
SQL Server MCM one of roughly 100 in the worldBI Architect/Consultant with 15 years experience of designing large scale and complex database systems
A slightly different definition of partitioning than the marketing releases from MS
Moving out BLOB columns removes SQL 2008 restrictions on Online Index RebuildsInfrequently access columns add overhead to queries which don’t require them, another good reason to avoid SELECT *
It’s best not to separate Tables and their indexes as both will have to restored to use the tables
Fill Factors waste space in the buffer pool and are only required for the active part of a tableRow Locks might be wasteful on the static historic data but required for active data
Grouping the smaller tables away from the large allows a quicker recovery from data lossInitialise Replication from a backupOnline Index Rebuilds in SQL Server 2008 don’t allow LOBsRead-Only file group is the only guaranteed way to make a table read-only, Column Stores will be Read/Write at some point
A single file from a file group can be restored while the database is online as long as it’s not from the Primary File GroupThe transaction log from the earliest file LSN to the current LSN will need to be restored to return the database into a consistent state
Mention Kimberly Trip’s blog on the Tipping Point
Today we will be covering the choices which should be considered when designing a new database and when deal with an existing database which has got new requirements like a higher uptime or scaling to larger volumes of data