It Depends - Database admin for developers - Rev 20151205Maggie Pint
Â
Do you feel like most interactions with your DBA/IT team result in statement ""It Depends""? Do you wonder what metrics your DBA is evaluating to make assertions about the system? Do you want to know key tips for performance tuning applications? A former DBA turned full-stack web developer will take you through some of the metrics and tools that DBAs use to evaluate performance so that you can more easily communicate with your DBA or troubleshoot your SQL server personally.
It Depends - Database admin for developers - Rev 20151205Maggie Pint
Â
Do you feel like most interactions with your DBA/IT team result in statement ""It Depends""? Do you wonder what metrics your DBA is evaluating to make assertions about the system? Do you want to know key tips for performance tuning applications? A former DBA turned full-stack web developer will take you through some of the metrics and tools that DBAs use to evaluate performance so that you can more easily communicate with your DBA or troubleshoot your SQL server personally.
Brk3288 sql server v.next with support on linux, windows and containers was...Bob Ward
Â
SQL Server is bringing its world-class RDBMS to Linux and Windows with SQL Server v.Next. In this session you will learn what´s next for SQL Server on Linux and how application developers and IT architects can now leverage the enterprise class features of SQL Server in every edition on Linux, Windows and containers.
SQL Server Tuning to Improve Database PerformanceMark Ginnebaugh
Â
SQL Server tuning is a process to eliminate performance bottlenecks and improve application service. This presentation from Confio Software discusses SQL diagramming, wait type data, column selectivity, and other solutions that will help make tuning projects a success, including:
â˘SQL Tuning Methodology
â˘Response Time Tuning Practices
â˘How to use SQL Diagramming techniques to tune SQL statements
â˘How to read executions plans
Enterprise Architect's view of Couchbase 4.0 with N1QLKeshav Murthy
Â
Enterprise architects have to decide on the database platform that will meet various requirements: performance and scalability on one side, ease of data modeling, agile development on the other, elasticity and flexibility to handle change easily, and a database platform that integrates well with tools and within ecosystem. This presentation will highlight the challenges and approaches to solution using Couchbase with N1QL.
Sql server performance tuning and optimizationManish Rawat
Â
Sql server performance tuning and optimization
SQL Server Concepts/Structure
Performance Measuring & Troubleshooting Tools
Locking
Performance Problem : CPU
Performance Problem : Memory
Performance Problem : I/O
Performance Problem : Blocking
Query Tuning
Indexing
SQL Server 2017 Enhancements You Need To KnowQuest
Â
In this session, database experts Pini Dibask and Jason Hall reveal the lesser-known features thatâll help you improve database performance in record time.
Hekaton is the original project name for In-Memory OLTP and just sounds cooler for a title name. Keeping up the tradition of deep technical âInsideâ sessions at PASS, this half-day talk will take you behind the scenes and under the covers on how the In-Memory OLTP functionality works with SQL Server.
We will cover âeverything Hekatonâ, including how it is integrated with the SQL Server Engine Architecture. We will explore how data is stored in memory and on disk, how I/O works, how native complied procedures are built and executed. We will also look at how Hekaton integrates with the rest of the engine, including Backup, Restore, Recovery, High-Availability, Transaction Logging, and Troubleshooting.
Demos are a must for a half-day session like this and what would an inside session be if we didnât bring out the Windows Debugger. As with previous âInsideâŚâ talks Iâve presented at PASS, this session is level 500 and not for the faint of heart. So read through the docs on In-Memory OLTP and bring some extra pain reliever as we move fast and go deep.
This session will appear as two sessions in the program guide but is not a Part I and II. It is one complete session with a small break so you should plan to attend it all to get the maximum benefit.
An Elastic Metadata Store for eBayâs Media PlatformMongoDB
Â
In order to build a robust, multi-tenant, highly available storage services that meet the businessâ SLA your databases has to be sharded. But if your service has to scale continuously through the incremental additions of storage without service interruption or human intervention, basic static sharding is not enough. At eBay, we are building MStore to solve this problem, with MongoDB as the storage engine. In this presentation, we will dive into the key design concepts of this solution.
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Lucas Jellema
Â
This presentation gives an brief overview of the history of relational databases, ACID and SQL and presents some of the key strentgths and potential weaknesses. It introduces the rise of NoSQL - why it arose, what is entails, when to use it. The presentation focuses on MongoDB as prime example of NoSQL document store and it shows how to interact with MongoDB from JavaScript (NodeJS) and Java.
Technical feature review of features introduced by MongoDB 3.4 on graph capabilities, MongoDB UI tool: Compass, improvements on the replication and aggregation framework stages and utils. Operations improvements on Ops Manager and MongoDB Atlas.
Brk3288 sql server v.next with support on linux, windows and containers was...Bob Ward
Â
SQL Server is bringing its world-class RDBMS to Linux and Windows with SQL Server v.Next. In this session you will learn what´s next for SQL Server on Linux and how application developers and IT architects can now leverage the enterprise class features of SQL Server in every edition on Linux, Windows and containers.
SQL Server Tuning to Improve Database PerformanceMark Ginnebaugh
Â
SQL Server tuning is a process to eliminate performance bottlenecks and improve application service. This presentation from Confio Software discusses SQL diagramming, wait type data, column selectivity, and other solutions that will help make tuning projects a success, including:
â˘SQL Tuning Methodology
â˘Response Time Tuning Practices
â˘How to use SQL Diagramming techniques to tune SQL statements
â˘How to read executions plans
Enterprise Architect's view of Couchbase 4.0 with N1QLKeshav Murthy
Â
Enterprise architects have to decide on the database platform that will meet various requirements: performance and scalability on one side, ease of data modeling, agile development on the other, elasticity and flexibility to handle change easily, and a database platform that integrates well with tools and within ecosystem. This presentation will highlight the challenges and approaches to solution using Couchbase with N1QL.
Sql server performance tuning and optimizationManish Rawat
Â
Sql server performance tuning and optimization
SQL Server Concepts/Structure
Performance Measuring & Troubleshooting Tools
Locking
Performance Problem : CPU
Performance Problem : Memory
Performance Problem : I/O
Performance Problem : Blocking
Query Tuning
Indexing
SQL Server 2017 Enhancements You Need To KnowQuest
Â
In this session, database experts Pini Dibask and Jason Hall reveal the lesser-known features thatâll help you improve database performance in record time.
Hekaton is the original project name for In-Memory OLTP and just sounds cooler for a title name. Keeping up the tradition of deep technical âInsideâ sessions at PASS, this half-day talk will take you behind the scenes and under the covers on how the In-Memory OLTP functionality works with SQL Server.
We will cover âeverything Hekatonâ, including how it is integrated with the SQL Server Engine Architecture. We will explore how data is stored in memory and on disk, how I/O works, how native complied procedures are built and executed. We will also look at how Hekaton integrates with the rest of the engine, including Backup, Restore, Recovery, High-Availability, Transaction Logging, and Troubleshooting.
Demos are a must for a half-day session like this and what would an inside session be if we didnât bring out the Windows Debugger. As with previous âInsideâŚâ talks Iâve presented at PASS, this session is level 500 and not for the faint of heart. So read through the docs on In-Memory OLTP and bring some extra pain reliever as we move fast and go deep.
This session will appear as two sessions in the program guide but is not a Part I and II. It is one complete session with a small break so you should plan to attend it all to get the maximum benefit.
An Elastic Metadata Store for eBayâs Media PlatformMongoDB
Â
In order to build a robust, multi-tenant, highly available storage services that meet the businessâ SLA your databases has to be sharded. But if your service has to scale continuously through the incremental additions of storage without service interruption or human intervention, basic static sharding is not enough. At eBay, we are building MStore to solve this problem, with MongoDB as the storage engine. In this presentation, we will dive into the key design concepts of this solution.
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Lucas Jellema
Â
This presentation gives an brief overview of the history of relational databases, ACID and SQL and presents some of the key strentgths and potential weaknesses. It introduces the rise of NoSQL - why it arose, what is entails, when to use it. The presentation focuses on MongoDB as prime example of NoSQL document store and it shows how to interact with MongoDB from JavaScript (NodeJS) and Java.
Technical feature review of features introduced by MongoDB 3.4 on graph capabilities, MongoDB UI tool: Compass, improvements on the replication and aggregation framework stages and utils. Operations improvements on Ops Manager and MongoDB Atlas.
Data Lake Acceleration vs. Data Virtualization - Whatâs the difference?Denodo
Â
Watch full webinar here: https://bit.ly/3hgOSwm
Data Lake technologies have been in constant evolution in recent years, with each iteration primising to fix what previous ones failed to accomplish. Several data lake engines are hitting the market with better ingestion, governance, and acceleration capabilities that aim to create the ultimate data repository. But isn't that the promise of a logical architecture with data virtualization too? So, whatâs the difference between the two technologies? Are they friends or foes? This session will explore the details.
Architectâs Open-Source Guide for a Data Mesh ArchitectureDatabricks
Â
Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh?
In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry.
The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems.
This session is targeted for architects, decision-makers, data-engineers, and system designers.
Data Virtualization Reference Architectures: Correctly Architecting your Solu...Denodo
Â
Correctly Architecting your Solutions for Analytical & Operational Uses reviews the two main types of use cases that can be solved with the Denodo Platform. Both high concurrency scenarios and big reporting use cases are discussed in this presentation in a comparative way, explaining the different approaches that you must take to be successful in any situation.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/wdZgpo.
Transform your DBMS to drive engagement innovation with Big DataAshnikbiz
Â
Erik Baardse and Ajit Gadge from EDB Postgres presented on how to transform your DBMS in order to drive digital business. How Postgres enables you to support a wider range of workloads with your relational database which opens the Big Data doors. They also cover EnterpriseDBâs Strategy around Big Data which focuses on 3 areas and finally last but not the last how to find money in IT with Big Data and digital transformation
Dates and times are one of the most common problems programmers encounter - so why is it that we so often get this space wrong? Let's discuss where our current Date goes bad, better ways to model the date and time space in our code, and how the Temporal proposal making its way through TC39 helps us write correct code for every person in every time zone.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Â
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
Â
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Â
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Â
Are you looking to streamline your workflows and boost your projectsâ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, youâre in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part âEssentials of Automationâ series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Hereâs what youâll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
Weâll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Donât miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
Â
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Â
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Â
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Â
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
Â
As AI technology is pushing into IT I was wondering myself, as an âinfrastructure container kubernetes guyâ, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefitâs both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
5. MongoDB
â˘Dominant player in document databases
â˘Runs on nearly all platforms
â˘Strongly Consistent in default configuration
â˘Indexes are similar to traditional SQL indexes in nature
â˘Stores data in customized Binary JSON (BSON) format that allows typing
â˘No support for cross-collection querying
â˘Client APIâs available in tons of languages
â˘Must use a third party provider like SOLR for advanced search capabilities
6. CouchDB
â˘Stores documents in plain JSON format
â˘Eventually consistent
â˘Indexes are map-reduce and defined in Javascript
â˘Clients in many languages
â˘Runs on Linux, OSX and Windows
â˘CouchDB-Lucene provides a Lucene integration for search
7. RavenDB
â˘Stores documents in plain JSON format
â˘Eventually consistent
â˘Indexes are built on Lucene. Lucene search is native to RavenDB.
â˘Server only runs on Windows
â˘.NET, Java, and HTTP Clients
â˘Limited support for cross-collection querying
8. Other Players
â˘Azure DocumentDB
⢠Very new product from Microsoft
â˘ReactDB
⢠Open source project that integrates push notifications into the database
â˘Cloudant
⢠IBM proprietary implementation of CouchDB
10. How do document databases work?
â˘Stores related data in a single document
â˘Usually uses JSON format for documents
â˘Enables the storage of complex object graphs together, instead of normalizing data out into
tables
â˘Stores documents in collections of the same type
â˘Allows querying within collections
â˘Does not typically allow querying across collections
â˘Offers high availability at the cost of consistency
11. Consideration: Schema Free
PROS
Easy to add properties
Simple migrations
Tolerant of differing data
CONS
Have to account for properties being missing
12. ACID
Atomicity
⌠Each transaction is all or nothing
Consistency
⌠Any transaction brings the database from one valid state to another
Isolation
⌠System ensures that transactions operated concurrently bring the database to the same state as if they
had been operated serially
Durability
⌠Once a transaction is committed, it remains so even in the event of power loss, etc
13. ACID in Document Databases
â˘Traditional transaction support is not available in any document database (except Raven)
â˘Document databases do support something like transactions within the scope of a document
â˘This makes document databases generally inappropriate for a wide variety of applications
⢠Do a google search for FlexCoin
â˘RavenDB is very close to ACID, but the community doesnât agree on whether it is ACID
16. Requirements
â˘An administration area is used to define âSurveysâ.
⢠Surveys have Questions
⢠Questions have answers
â˘Surveys can be administrated in sets called workflows
â˘When a survey changes, this change can only apply to surveys moving forward
⢠Because of this, each user must receive a survey âinstanceâ to track the version of the survey he/she got
17. A Traditional SQL Schema
â˘With various other requirements not described here, this schema came out to 83 tables
â˘For one of our heaviest usage clients, the average user would have 119 answers in the âSaved
Answerâ table
â˘With over 200,000 users after two years of use, the âSaved Answerâ table had 24,014,330 rows
â˘This table was both read and write heavy, so it was extremely difficult to define effective SQL
indexes
â˘The hardware cost for these SQL servers was astronomical
â˘This sucked
18. Designing Documents
â˘An aggregate is a collection of objects that can be treated as one
â˘An aggregate root is the object that contains all other objects inside of it
â˘When designing document schema, find your aggregates and create documents around them
â˘If you have an entity, it should be persisted as itâs own document because you will likely have to
store references to it
19. Survey System Design
â˘A combination SQL and Document DB design was used
â˘Survey Templates (one type of entity) were put into the SQL Database
â˘When a survey was assigned to a user as part of a workflow (another entity, and also an
aggregate), itâs data at that time was put into the document database
â˘The userâs responses were saved as part of the workflow document
â˘Reading a userâs application data became as simple as making one request for her workflow
document
20. Consideration: Models Aggregates Well
PROS
Improves performance by reducing lookups
Allows for easy persistence of object oriented
designs
CONS
none
21. Sharding
â˘Sharding is the practice of distributing data across multiple servers
â˘All major document database providers support sharding natively
â˘Document Databases are ideal for sharding because document data is self contained (less need
to worry about a query having to run on two servers)
â˘Sharding is usually accomplished by selecting a shard key for a collection, and allowing the
collection to be distributed to different nodes based on that key
â˘Tenant Id and geographic regions are typical choices for shard keys
22. Replication
â˘All major document database providers support replication
â˘In most replication setups, a primary node takes all write operations, and a secondary node
asynchronously replicates these write operations
â˘In the event of a failure of the primary, the secondary begins to take write operations
â˘MongoDB can be configured to allow reads from secondaries as a performance optimization,
resulting in eventual instead of strong consistency
24. Survey System: End Result
â˘Each user is associated with about 20 documents
â˘Documents are distributed across multiple databases using sharding
â˘Master/Master replication is used to ensure extremely high availability
â˘There have been no database performance issues in the year and a half the app has been in
production
â˘Because there is no schema migration concern, deploying updates has been drastically
simplified
â˘Hardware cost is reasonable (but not cheap)
25.
26. Indexes
â˘All document databases support some form of indexing to improve query performance
â˘Some document databases do not allow querying without an index
â˘In general, you shouldnât query without an index anyways
31. CRM Requirements
â˘Track customers and basic information about them
â˘Track contacts and basic information about them
â˘Track sales deals and where they are in the pipeline
â˘Track orders generated from sales deals
â˘Track user tasks
32. Customers and Their Deals
â˘Customers and Deals are both entities, which is to say that they have distinct identity
â˘For this reason, Deals and Customer should be two separate collections
â˘There is no native support for cross-collection querying in most Document Databases
⢠The cross-collection querying support in RavenDB can have performance issues
33. Consideration: One document per
interaction
PROS
Improves performance
Encourages modeling aggregates well
CONS
Not actually achievable in most cases
34. Searching Deals by Customer Name
â˘The deal document must contain a denormalized customer object with the customerâs ID and
name
â˘We have a choice to make with this denormalization
⢠Allow the denormalization to just be wrong in the event the customer name is changed
⢠Maintain the denormalization when the customer name is changed
35. Denormalization Considerations
â˘Is stale data acceptable? This is the best option in all cases where it is possible.
â˘If stale data is unacceptable, how many documents are likely to need update when a change is
made? How many collections? How often are changes going to be made?
â˘Using an event bus to move denormalization updates to a background process can be very
beneficial if failure of an update isnât critical for the user to know
36. Consideration: Models Relationships
Poorly
PROS
None
CONS
Stale (out of date) data must be accepted in
the system
Large amounts of boilerplate code must be
written to maintain denormalizations
In certain circumstances a queuing/eventing
system is unavoidable
37. Consideration: No Foreign Key
Constraints
PROS
Donât have to define foreign key constraints
CONS
No built in checks for data consistency
42. ACID
â˘RavenDB has a session that allows multiple documents to be written as a transaction
â˘Keep in mind, reads from indexes are still eventually consistent
â˘http://ayende.com/blog/164066/ravendb-acid-base
43. Eventual Consistency
â˘Issues with eventual consistency can be circumvented by using the wait for non-stale results
functionality
â˘Waiting for non-stale results can result in long wait times
â˘Waiting for non-stale results with a timeout can result in no results
44. Load Document
â˘RavenDB has limited support for cross collection querying in the form of using LoadDocument
â˘This eliminates some of the concerns with the deals by customer name search example
â˘On Ravenâs website they warn that injudicious use of LoadDocument can result in some very
expensive computations
45. Patching
â˘Raven supports partial document updates on a collection of documents using the Patching API
â˘This can be extremely helpful for maintaining denormalizations
â˘Patching is not transactional
47. âŚnerds like us are allowed to be unironically
enthusiastic about stuff⌠Nerds are allowed to
love stuff, like jump-up-and-down-in-the-chair-
canât-control-yourself love it.
-John Green