Spatial support in SQL Server 2008 R2 provides two spatial data types - geometry and geography. It includes a comprehensive set of spatial methods and high performance spatial indexes. The spatial library supports open geospatial standards and can be used to build location-aware applications.
This document discusses several key differences between traditional databases and Hive. Hive uses a schema-on-read model where the schema is not enforced during data loading, making the initial load much faster. However, this impacts query performance since indexing and compression cannot be applied during loading. Pig Latin is a data flow language where operations are executed in a defined sequence, differing from SQL which is declarative. While Hive currently lacks features like transactions and indexing, the developers are working to integrate HBase and improve support for these features.
Spatial databases are designed to store and analyze spatial data more efficiently than traditional databases. Spatial data represents objects in geometric space and includes points, lines, and polygons. Spatial databases use spatial indexes and spatial query languages to optimize storage and retrieval of spatial data types and allow spatial queries and analysis. Common spatial database operations include measurements, functions, and predicates on geometric objects.
This document provides an overview of relational database management systems (RDBMS). It defines RDBMS as a system that structures data into tables with rows and columns, and can relate these tables through common fields. The key aspects covered include relational algebra operations like select, project, join; structured query language (SQL) for manipulating and retrieving data; and the advantages of RDBMS like supporting a tabular data structure, multi-user access, and imposing integrity constraints.
Vector data stores individual map features with high precision and has a linked attribute table for storing metadata. It is well-suited for mapmaking but poorly suited for storing continuously varying surfaces like elevation. Raster data stores information as a grid of cells, each with a single value, making it ideal for representing continuously varying data but with less precision. Some types of analysis are faster with rasters due to more developed analysis tools, while others are faster with vectors.
This document provides an introduction to basics of geographic information systems (GIS) using ESRI ArcGIS Desktop. It contains instructions on key GIS concepts and functions including geo-referencing, digitizing geographical data, and processing digital elevation models. The overall aim is to familiarize users with GIS terminology and functionalities through hands-on exercises in ArcGIS without requiring an expert. Step-by-step guides are provided for geo-referencing a map, digitizing map features, and visualizing digital elevation data.
This document provides an overview of GIS data management and database management systems (DBMS). It discusses how GIS data involves both spatial and attribute data that must be stored and linked. It defines DBMS and describes their key functions like data storage, retrieval, security and integrity. The document outlines the typical components of a DBMS including data definition, storage definition and data manipulation. It also discusses different file structures for organizing GIS data, including simple lists, indexed files and building GIS worlds. Finally, it concludes that a DBMS allows for data backup, recovery and redundancy control for database management.
TYBSC IT PGIS Unit I Chapter II Geographic Information and Spacial DatabaseArti Parab Academics
This document discusses geographic information systems and spatial databases. It covers several key topics:
1) Models and representations of the real world in digital form, including raster and vector data models. Raster models use a grid approach while vector models represent points, lines and polygons.
2) Types of geographic phenomena like fields and objects that can be represented. Fields have values across a continuous space like elevation, while objects are discrete like roads.
3) Computer representations including raster and vector formats. Raster uses a grid of cells while vector uses points, lines and polygons.
4) Topology and spatial relationships between objects like containment, overlap and adjacency.
5) Organizing and managing spatial data in
This document provides an overview of database management systems (DBMS) and database architecture. It discusses what a DBMS is, including that it enables creation, access and modification of databases. It then describes the four main types of DBMS: hierarchical, network, relational and object-oriented. For each type it provides a brief explanation of its structure and functionality. The document concludes with a discussion of the typical functionality of a DBMS and a description of database architecture, including the global conceptual schema, fragmentation and allocation schema, and local schemas.
This document discusses several key differences between traditional databases and Hive. Hive uses a schema-on-read model where the schema is not enforced during data loading, making the initial load much faster. However, this impacts query performance since indexing and compression cannot be applied during loading. Pig Latin is a data flow language where operations are executed in a defined sequence, differing from SQL which is declarative. While Hive currently lacks features like transactions and indexing, the developers are working to integrate HBase and improve support for these features.
Spatial databases are designed to store and analyze spatial data more efficiently than traditional databases. Spatial data represents objects in geometric space and includes points, lines, and polygons. Spatial databases use spatial indexes and spatial query languages to optimize storage and retrieval of spatial data types and allow spatial queries and analysis. Common spatial database operations include measurements, functions, and predicates on geometric objects.
This document provides an overview of relational database management systems (RDBMS). It defines RDBMS as a system that structures data into tables with rows and columns, and can relate these tables through common fields. The key aspects covered include relational algebra operations like select, project, join; structured query language (SQL) for manipulating and retrieving data; and the advantages of RDBMS like supporting a tabular data structure, multi-user access, and imposing integrity constraints.
Vector data stores individual map features with high precision and has a linked attribute table for storing metadata. It is well-suited for mapmaking but poorly suited for storing continuously varying surfaces like elevation. Raster data stores information as a grid of cells, each with a single value, making it ideal for representing continuously varying data but with less precision. Some types of analysis are faster with rasters due to more developed analysis tools, while others are faster with vectors.
This document provides an introduction to basics of geographic information systems (GIS) using ESRI ArcGIS Desktop. It contains instructions on key GIS concepts and functions including geo-referencing, digitizing geographical data, and processing digital elevation models. The overall aim is to familiarize users with GIS terminology and functionalities through hands-on exercises in ArcGIS without requiring an expert. Step-by-step guides are provided for geo-referencing a map, digitizing map features, and visualizing digital elevation data.
This document provides an overview of GIS data management and database management systems (DBMS). It discusses how GIS data involves both spatial and attribute data that must be stored and linked. It defines DBMS and describes their key functions like data storage, retrieval, security and integrity. The document outlines the typical components of a DBMS including data definition, storage definition and data manipulation. It also discusses different file structures for organizing GIS data, including simple lists, indexed files and building GIS worlds. Finally, it concludes that a DBMS allows for data backup, recovery and redundancy control for database management.
TYBSC IT PGIS Unit I Chapter II Geographic Information and Spacial DatabaseArti Parab Academics
This document discusses geographic information systems and spatial databases. It covers several key topics:
1) Models and representations of the real world in digital form, including raster and vector data models. Raster models use a grid approach while vector models represent points, lines and polygons.
2) Types of geographic phenomena like fields and objects that can be represented. Fields have values across a continuous space like elevation, while objects are discrete like roads.
3) Computer representations including raster and vector formats. Raster uses a grid of cells while vector uses points, lines and polygons.
4) Topology and spatial relationships between objects like containment, overlap and adjacency.
5) Organizing and managing spatial data in
This document provides an overview of database management systems (DBMS) and database architecture. It discusses what a DBMS is, including that it enables creation, access and modification of databases. It then describes the four main types of DBMS: hierarchical, network, relational and object-oriented. For each type it provides a brief explanation of its structure and functionality. The document concludes with a discussion of the typical functionality of a DBMS and a description of database architecture, including the global conceptual schema, fragmentation and allocation schema, and local schemas.
The document presents on the implementation of a database management system (DBMS). It defines a DBMS as software that controls the creation and use of a database, allowing different programs to access the same database simultaneously. It describes the key components of a DBMS, including the DBMS engine, data definition subsystem, data manipulation subsystem, and application generation subsystem. The presentation outlines advantages such as a centralized warehouse of information, data security, and consistency. It provides tips for implementing a DBMS, such as identifying necessary data elements, setting data access permissions, testing the system, educating users, and evaluating the system after deployment.
This document discusses shortest path analysis and Dijkstra's algorithm. It defines shortest path analysis as finding the minimum cumulative path between nodes on a network. Dijkstra's algorithm is described as finding the shortest paths from a starting node to all other reachable nodes. An example application calculates the shortest path from node A to G on a sample graph. The document concludes that shortest path analysis can identify key walking routes and inform improvements to pedestrian infrastructure.
1. GIS can be used for data management efficiency through DBMS which allows storage, retrieval, and access of large amounts of spatial data.
2. Military applications include analyzing terrain for combat through GIS and remote sensing to collect spatial data to support effective decision making.
3. Other applications include mapping health facilities and diseases, tracking wildlife populations, disaster management in telecommunications, crime analysis, agriculture and mining resource planning, and property valuation for taxation.
4. Limitations include lack of awareness of GIS potential, effort to digitize analog data, technical capacity to interpret spatial data, and challenges representing 3D and 4D environmental data sets.
This document discusses entity relationship modeling and describes its key components - entities, attributes, and relationships. It defines entities as objects of interest to users, with each entity corresponding to a database table. Attributes are characteristics of entities and include identifiers, single-valued and multi-valued attributes. Relationships connect entities and are defined by connectivity, cardinality, and whether they are identifying or non-identifying. The document provides examples and diagrams to illustrate these entity relationship modeling concepts.
This document outlines the syllabus for a course on Geographic Information Systems (GIS). The course is divided into 5 units that cover fundamentals of GIS, spatial data models, data input and topology, data analysis, and applications of GIS. The objectives are to introduce GIS fundamentals and processes of data management, analysis, and output. Students will learn about spatial data structures, data quality standards, and tools for data input, analysis, and management. The course aims to provide knowledge of GIS concepts and techniques.
TYBSC IT PGIS Unit I Chapter I- Introduction to Geographic Information SystemsArti Parab Academics
A Gentle Introduction to GIS The nature of GIS: Some fundamental observations, Defining GIS, GISystems, GIScience and GIApplications, Spatial data and Geoinformation. The real world and representations of it: Models and modelling, Maps, Databases, Spatial databases and spatial analysis
This document discusses GIS topology, which establishes rules for how geographic features share geometry and spatial relationships. Topology ensures data quality, enhances analysis, and manages coincident geometry. It has three components: connectivity between nodes and arcs, area definition using polygon boundaries, and contiguity to determine adjacent features. Topological rules prevent errors like overlaps, gaps, dangles and ensure proper containment of points and boundaries.
Topics:
1. Introduction to GIS
2. Components of GIS
3. Types of Data
4. Spatial Data
5. Non-Spatial Data
6. GIS Operations
7. Coordinate Systems
8. Datum
9. Map Projections
10. Raster Data Compression Techniques
11. GIS Software
12. Free GIS Data Resources
This document provides summaries of various applications of GIS technology across different domains:
1) GIS is used for urban planning to analyze urban growth and expansion and identify suitable sites for development based on factors like accessibility, land flatness, current usage and water supply.
2) GIS provides mapping functionality and allows non-cartographers to visually represent data on maps. Examples include Google Maps, Bing Maps and Yahoo Maps.
3) GIS helps monitor coal mine safety and identify risks of spontaneous combustion fires.
4) GIS supports business applications like customer tracking, site selection, marketing and optimizing sales territories.
5) GIS is useful for public health applications like evaluating health policies, studying relationships between
Join queries combine data from two or more tables in a database. An inner join returns rows where there is a match between columns in both tables. A left join returns all rows from the left table, along with matched rows from the right table, returning null if there is no match. A right join is similar but returns all rows from the right table. A full join returns all rows and fills in nulls for missing matches between the tables.
Slides from my Introduction to PostGIS workshop at the FOSS4G conference in 2009. The material is available at http://revenant.ca/www/postgis/workshop/
Raster data is represented by a grid of cells, where each cell contains numeric or qualitative values. Raster data comes from sources like images, maps, and satellite imagery. Common analyses of raster data include buffering, reclassification, hillshades, interpolation, and surface calculation. Buffering assigns "in" and "out" values to cells based on their distance from a feature. Reclassification reassigns cell values. Hillshades create shaded relief maps from elevation data. Interpolation estimates values between known data points. Surface calculation performs cell-by-cell mathematical functions on rasters.
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me for other informations and to download
This document provides an introduction to database management systems (DBMS). It discusses key concepts such as database models including hierarchical, network, relational and entity-relationship models. It also covers database planning, design, implementation and maintenance. Specific topics covered include data modeling, database normalization, query languages, transaction management and database administration.
This document discusses the key functions of a geographic information system (GIS). It explains that a GIS allows users to capture, store, query, analyze, display and output geographic data. It describes the vector and raster data models used to store spatial data. The document also outlines the three main views of a GIS - the geovisualization view which includes maps, the geodata view which is the spatial database, and the geoprocessing view which involves tools to transform and derive new information from existing datasets. Finally, it discusses some key concepts for GIS maps including layers, features, attributes, and scale.
This document discusses different types of joins in SQL including inner joins, self joins, outer joins, and cross joins. An inner join combines rows from two tables based on a common column and returns matched rows. A self join performs an inner join on a single table to match rows with itself. Outer joins return all rows from one or both tables whether or not they have a match. A cross join returns the Cartesian product of all rows in two tables.
Descubriendo los datos espaciales en SQL ServerSpanishPASSVC
SQL Server introduce varias mejoras significativas en los tipos de datos espaciales, como el soporte para nuevos subtipos de arcos circulares, métodos nuevos y muchas más mejoras. En esta sesión abordaremos estas nuevas características de los datos geoespaciales desde el punto de vista de un desarrollador de base de datos.
Descubriendo los datos espaciales en sql server 2012John Bulla
Este documento presenta una introducción a los tipos de datos espaciales en SQL Server 2012. Cubre los nuevos tipos de datos geométricos y geográficos, incluidos puntos, líneas, polígonos y colecciones. También describe los métodos para crear y manipular objetos espaciales, así como las nuevas características en SQL Server 2012 como objetos circulares y agregaciones espaciales.
The document presents on the implementation of a database management system (DBMS). It defines a DBMS as software that controls the creation and use of a database, allowing different programs to access the same database simultaneously. It describes the key components of a DBMS, including the DBMS engine, data definition subsystem, data manipulation subsystem, and application generation subsystem. The presentation outlines advantages such as a centralized warehouse of information, data security, and consistency. It provides tips for implementing a DBMS, such as identifying necessary data elements, setting data access permissions, testing the system, educating users, and evaluating the system after deployment.
This document discusses shortest path analysis and Dijkstra's algorithm. It defines shortest path analysis as finding the minimum cumulative path between nodes on a network. Dijkstra's algorithm is described as finding the shortest paths from a starting node to all other reachable nodes. An example application calculates the shortest path from node A to G on a sample graph. The document concludes that shortest path analysis can identify key walking routes and inform improvements to pedestrian infrastructure.
1. GIS can be used for data management efficiency through DBMS which allows storage, retrieval, and access of large amounts of spatial data.
2. Military applications include analyzing terrain for combat through GIS and remote sensing to collect spatial data to support effective decision making.
3. Other applications include mapping health facilities and diseases, tracking wildlife populations, disaster management in telecommunications, crime analysis, agriculture and mining resource planning, and property valuation for taxation.
4. Limitations include lack of awareness of GIS potential, effort to digitize analog data, technical capacity to interpret spatial data, and challenges representing 3D and 4D environmental data sets.
This document discusses entity relationship modeling and describes its key components - entities, attributes, and relationships. It defines entities as objects of interest to users, with each entity corresponding to a database table. Attributes are characteristics of entities and include identifiers, single-valued and multi-valued attributes. Relationships connect entities and are defined by connectivity, cardinality, and whether they are identifying or non-identifying. The document provides examples and diagrams to illustrate these entity relationship modeling concepts.
This document outlines the syllabus for a course on Geographic Information Systems (GIS). The course is divided into 5 units that cover fundamentals of GIS, spatial data models, data input and topology, data analysis, and applications of GIS. The objectives are to introduce GIS fundamentals and processes of data management, analysis, and output. Students will learn about spatial data structures, data quality standards, and tools for data input, analysis, and management. The course aims to provide knowledge of GIS concepts and techniques.
TYBSC IT PGIS Unit I Chapter I- Introduction to Geographic Information SystemsArti Parab Academics
A Gentle Introduction to GIS The nature of GIS: Some fundamental observations, Defining GIS, GISystems, GIScience and GIApplications, Spatial data and Geoinformation. The real world and representations of it: Models and modelling, Maps, Databases, Spatial databases and spatial analysis
This document discusses GIS topology, which establishes rules for how geographic features share geometry and spatial relationships. Topology ensures data quality, enhances analysis, and manages coincident geometry. It has three components: connectivity between nodes and arcs, area definition using polygon boundaries, and contiguity to determine adjacent features. Topological rules prevent errors like overlaps, gaps, dangles and ensure proper containment of points and boundaries.
Topics:
1. Introduction to GIS
2. Components of GIS
3. Types of Data
4. Spatial Data
5. Non-Spatial Data
6. GIS Operations
7. Coordinate Systems
8. Datum
9. Map Projections
10. Raster Data Compression Techniques
11. GIS Software
12. Free GIS Data Resources
This document provides summaries of various applications of GIS technology across different domains:
1) GIS is used for urban planning to analyze urban growth and expansion and identify suitable sites for development based on factors like accessibility, land flatness, current usage and water supply.
2) GIS provides mapping functionality and allows non-cartographers to visually represent data on maps. Examples include Google Maps, Bing Maps and Yahoo Maps.
3) GIS helps monitor coal mine safety and identify risks of spontaneous combustion fires.
4) GIS supports business applications like customer tracking, site selection, marketing and optimizing sales territories.
5) GIS is useful for public health applications like evaluating health policies, studying relationships between
Join queries combine data from two or more tables in a database. An inner join returns rows where there is a match between columns in both tables. A left join returns all rows from the left table, along with matched rows from the right table, returning null if there is no match. A right join is similar but returns all rows from the right table. A full join returns all rows and fills in nulls for missing matches between the tables.
Slides from my Introduction to PostGIS workshop at the FOSS4G conference in 2009. The material is available at http://revenant.ca/www/postgis/workshop/
Raster data is represented by a grid of cells, where each cell contains numeric or qualitative values. Raster data comes from sources like images, maps, and satellite imagery. Common analyses of raster data include buffering, reclassification, hillshades, interpolation, and surface calculation. Buffering assigns "in" and "out" values to cells based on their distance from a feature. Reclassification reassigns cell values. Hillshades create shaded relief maps from elevation data. Interpolation estimates values between known data points. Surface calculation performs cell-by-cell mathematical functions on rasters.
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me for other informations and to download
This document provides an introduction to database management systems (DBMS). It discusses key concepts such as database models including hierarchical, network, relational and entity-relationship models. It also covers database planning, design, implementation and maintenance. Specific topics covered include data modeling, database normalization, query languages, transaction management and database administration.
This document discusses the key functions of a geographic information system (GIS). It explains that a GIS allows users to capture, store, query, analyze, display and output geographic data. It describes the vector and raster data models used to store spatial data. The document also outlines the three main views of a GIS - the geovisualization view which includes maps, the geodata view which is the spatial database, and the geoprocessing view which involves tools to transform and derive new information from existing datasets. Finally, it discusses some key concepts for GIS maps including layers, features, attributes, and scale.
This document discusses different types of joins in SQL including inner joins, self joins, outer joins, and cross joins. An inner join combines rows from two tables based on a common column and returns matched rows. A self join performs an inner join on a single table to match rows with itself. Outer joins return all rows from one or both tables whether or not they have a match. A cross join returns the Cartesian product of all rows in two tables.
Descubriendo los datos espaciales en SQL ServerSpanishPASSVC
SQL Server introduce varias mejoras significativas en los tipos de datos espaciales, como el soporte para nuevos subtipos de arcos circulares, métodos nuevos y muchas más mejoras. En esta sesión abordaremos estas nuevas características de los datos geoespaciales desde el punto de vista de un desarrollador de base de datos.
Descubriendo los datos espaciales en sql server 2012John Bulla
Este documento presenta una introducción a los tipos de datos espaciales en SQL Server 2012. Cubre los nuevos tipos de datos geométricos y geográficos, incluidos puntos, líneas, polígonos y colecciones. También describe los métodos para crear y manipular objetos espaciales, así como las nuevas características en SQL Server 2012 como objetos circulares y agregaciones espaciales.
This document provides an introduction and overview of key concepts related to SQL Server databases including:
- The database engine and its role in storing, processing, and securing data
- System and user databases
- Database objects like tables, views, indexes, stored procedures
- Structured Query Language (SQL) and its sublanguages for data definition, manipulation, and transaction control
- Guidelines for writing SQL statements
- Creating and using databases along with creating tables and defining data types and constraints
XQuery is a language that can query structured or semi-structured XML data. It allows XML data stored in a database to be queried. XQuery is based on XPath but adds support for better iteration, sorting results, and constructing XML.
The document discusses multi-thematic spatial databases for efficiently storing, accessing, processing, and visualizing large volumes of geospatial data from multiple sources and sensors. It describes experience with designing databases to handle terabytes of temporal, multi-sensor data using spatial indexing. The goals are a unified approach for multi-thematic data storage, efficient data handling, and enabling searches across time, space and attributes while incorporating visualizations.
The document discusses cursors in SQL Server 2005, including what they are, their types (forward-only, static, dynamic, keyset driven), and how to work with them using declare, open, fetch, close, and deallocate statements. Cursor operations allow row-by-row processing of result sets for actions like updating or deleting table rows.
This document provides tips for improving front-end website performance. It discusses how DNS lookups, HTTP connections, sequential loading, bloated DOM, bloated CSS, and large payload sizes can negatively impact performance. It recommends strategies like combining CSS and JavaScript files, using CSS sprites for images, lazy-loading images, minimizing selectors in CSS, reducing the total bits transferred, and optimizing media files. Tools like Sass, asset plugins, and minification are suggested to help implement these techniques.
Sql server ___________session3-normailzationEhtisham Ali
Normalization is the process of organizing data in a database to minimize redundancy and dependency. It involves breaking tables into smaller tables and linking them through relationships. The goals are to eliminate storing duplicate data, ensure related data is stored together, and reduce data anomalies. Normalization is achieved through three normal forms - 1NF, 2NF, and 3NF - which introduce rules to simplify attributes, eliminate partial dependencies, and remove transitive dependencies.
Analysis Services uses a multidimensional data model with dimensions, measures, and facts. It provides a logical and physical architecture for building and querying multidimensional databases. The logical architecture includes dimensions, cubes, measures and aggregations. The physical architecture includes server components, storage, and clients that connect via XMLA. Developers can program against Analysis Services using ADOMD.NET, AMO, XMLA, and ASSL.
This document summarizes new features in SQL Server 2008 for .NET developers, including spatial data support, BLOB storage using Filestream, enhancements to T-SQL, new date/time types, improved integration with Visual Studio, and business intelligence tools like SSAS, SSIS, and SSRS. It provides overviews of key concepts like spatial data types, using Filestream for BLOB storage, table-valued parameters, new date/time functionality, MERGE statements, shorthand notation in T-SQL, Entity Framework, SQL CLR, and Reporting Services.
The document provides an introduction to CSS (Cascading Style Sheets), explaining what CSS is, how it works, and some basic syntax and concepts. CSS allows separation of document content from document presentation by defining styles that are applied to HTML elements. Styles can be defined internally, in an external CSS file, or inline. The CSS box model is also explained, with the content, padding, border, and margin areas of elements illustrated. Common CSS properties for text formatting are also listed.
This document provides an introduction to Microsoft SQL Server 2005 and its components. It discusses the history of SQL Server and how it works as a relational database management system. It also summarizes the key components of SQL Server Management Studio, including Object Explorer, Registered Servers, Solution Explorer, and Query Editor. Finally, it provides overviews of SQL Server Agent and how it can automate tasks, as well as how to perform database backup, restore, and import/export functions.
The document provides 5 tips for successfully upgrading SQL Server Integration Services (SSIS) packages to SQL Server 2012:
1. Manually edit package configurations, especially connection strings, after upgrading with the upgrade wizard since configurations are not automatically updated.
2. Use the Project Conversion Wizard to convert packages to the new project deployment model in SQL Server 2012 for improved deployment and management.
3. Update Execute Package tasks to use project references rather than file references for calling child packages within the same project.
4. Parameterize the PackageName property of Execute Package tasks to dynamically configure which child package runs at runtime.
5. Convert package configurations to parameters when possible to take advantage of improved configuration handling in the
The document discusses indexes in SQL Server 2005, including what they are, why they are needed to improve query performance, and the different types (clustered and nonclustered). It also covers how to create indexes using SQL statements, examples of creating indexes on tables, and activities for learners to practice creating, dropping, and rebuilding indexes.
This document provides an introduction and overview of stored procedures and functions in SQL. It discusses transaction management using COMMIT and ROLLBACK statements. It defines stored procedures as precompiled collections of SQL statements that can accept parameters and return values. Stored procedures offer benefits like modular programming and faster execution. The document also introduces user-defined functions and provides examples of creating and executing stored procedures and functions.
Triggers allow SQL code to be automatically executed in response to data changes, such as inserts, updates or deletes. There are two types of triggers: after triggers which execute after the data change, and instead of triggers which execute instead of the triggering statement. Triggers use the inserted and deleted tables to access old and new records. Triggers are created using the CREATE TRIGGER statement and can be altered, dropped or have their definition viewed.
In the presentation we review the Spatial Data in SQL Server.
Best Regards,
Dr. Eduardo Castro Martinez, Microsoft SQL Server MVP
http://ecastrom.blogspot.com
http://tinyurl.com/comunidadwindows
Serverless Cloud Data Lake with Spark for Serving Weather Data
1) The document discusses using a serverless architecture with IBM Cloud services like SQL Query powered by Spark, Cloud Object Storage, and Cloud Functions to build a cost-effective cloud data lake for serving historical weather data on demand.
2) It describes how data skipping techniques and geospatial indexes in SQL Query can accelerate queries by an order of magnitude by pruning irrelevant data.
3) The new serverless solution provides unlimited storage, global coverage, and supports large queries for machine learning and analytics at an order of magnitude lower cost than the previous implementation.
U-SQL - Azure Data Lake Analytics for DevelopersMichael Rys
This document introduces U-SQL, a language for big data analytics on Azure Data Lake Analytics. U-SQL unifies SQL with imperative coding, allowing users to process both structured and unstructured data at scale. It provides benefits of both declarative SQL and custom code through an expression-based programming model. U-SQL queries can span multiple data sources and users can extend its capabilities through C# user-defined functions, aggregates, and custom extractors/outputters. The document demonstrates core U-SQL concepts like queries, joins, window functions, and the metadata model, highlighting how U-SQL brings together SQL and custom code for scalable big data analytics.
Spatial data types allow for location-based queries by representing geometric objects like points, lines, and polygons. SQL Server supports both flat (GEOMETRY) and round earth (GEOGRAPHY) spatial data types that adhere to Open Geospatial Consortium standards. Spatial data can be input and output in different formats and queried using spatial methods and indexed for performance with spatial indexes.
[Research] azure ml anatomy of a machine learning service - Sharat ChikkerurPAPIs.io
In this talk, we describe AzureML: a web service enabling software developers and data scientists to build predictive applications. AzureML provides several unique features. These include (a) Collaboration (b) Versioning (c) Graphical authoring(d) Push button operationalization and (e) Monetization. We outline the design principles, system design and lessons learned in building such a system.
Serverless SQL provides a serverless analytics platform that allows users to analyze data stored in object storage without having to manage infrastructure. Key features include seamless elasticity, pay-per-query consumption, and the ability to analyze data directly in object storage without having to move it. The platform includes serverless storage, data ingest, data transformation, analytics, and automation capabilities. It aims to create a sharing economy for analytics by allowing various users like developers, data engineers, and analysts flexible access to data and analytics.
Non è necessario tirare in ballo l’IoT per immaginare quanto possa essere utile per fare query sui dati mentre questi fluiscono verso il database, e non solamente dopo. Si apre un mondo di possibilità per quanto riguarda alerting & monitoring in tempo reale, che è chiaramente la parte più immediata, ma è anche possibile pensare a cose come real-time dasboarding e soluzioni per aggiustare prezzi ed offerte di prodotti in tempo reale. In questa sessione vedremo come è possibile utilizzare Azure Stream Analytics ed il suo linguaggio SQL-Like per analizzare i dati in streaming, e quindi iniziare a prendere confidenza con questo nuovo approccio ormai sempre pià in voga e sempre più richesto, sia nel mondo dell’IoT che non.
Using Apache Calcite for Enabling SQL and JDBC Access to Apache Geode and Oth...Christian Tzolov
When working with BigData & IoT systems we often feel the need for a Common Query Language. The system specific languages usually require longer adoption time and are harder to integrate within the existing stacks.
To fill this gap some NoSql vendors are building SQL access to their systems. Building SQL engine from scratch is a daunting job and frameworks like Apache Calcite can help you with the heavy lifting. Calcite allow you to integrate SQL parser, cost-based optimizer, and JDBC with your NoSql system.
We will walk through the process of building a SQL access layer for Apache Geode (In-Memory Data Grid). I will share my experience, pitfalls and technical consideration like balancing between the SQL/RDBMS semantics and the design choices and limitations of the data system.
Hopefully this will enable you to add SQL capabilities to your prefered NoSQL data system.
Apache Geode Meetup, Cork, Ireland at CITApache Geode
This document provides an introduction to Apache Geode (incubating), including:
- A brief history of Geode and why it was developed
- An overview of key Geode concepts such as regions, caching, and functions
- Examples of interesting large-scale use cases from companies like Indian Railways
- A demonstration of using Geode with Apache Spark and Spring XD for a stock prediction application
- Information on how to get involved with the Geode open source project community
Fast, distributed NoSQL and relational database at any scale. This contains many features including Partition and Indexes,
Data movement, Change Feed
Integration (Azure Functions and Search), Consistency Models, Replication and Multi-write, etc.,
The document provides information about an upcoming SQL Saturday event on June 1, 2013 focused on SQL Server 2012 Integration Services for beginners. It includes an agenda for the event that covers topics such as an introduction to SSIS, SSIS tools, variables, parameters, expressions, SSIS tasks, containers, and data flows. The speaker is then introduced, which details his experience and qualifications.
Enterprise geodatabase sql access and administrationbrentpierce
The document provides an overview of accessing and administering an enterprise geodatabase through SQL and Python. It discusses how the geodatabase is based on relational database principles with user data stored in tables and system metadata stored in system tables. It describes how spatial types store geometry data and the benefits of using SQL to access and edit geodatabase content. The document also outlines how Python can be used for geodatabase administration tasks like schema creation, maintenance, and publishing tools.
Azure Cosmos DB - The Swiss Army NoSQL Cloud DatabaseBizTalk360
Microsoft Cosmos DB is the Swiss army NoSQL database in the cloud. It is a multi-model, multi-API, globally-distributed, highly-available, and secure No-SQL database in Azure. In this session, we will explore its capabilities and features through several demos.
The document discusses Azure Data Lake and U-SQL. It provides an overview of the Data Lake approach to storing and analyzing data compared to traditional data warehousing. It then describes Azure Data Lake Storage and Azure Data Lake Analytics, which provide scalable data storage and an analytics service built on Apache YARN. U-SQL is introduced as a language that unifies SQL and C# for querying data in Data Lakes and other Azure data sources.
Closer Look at Cloud Centric ArchitecturesTodd Kaplinger
The document discusses the cloud architecture of Presence Insights, a service that provides analytics for physical locations. Some key points:
- Presence Insights migrated from a traditional on-premises JEE architecture to a cloud-native microservices architecture on Bluemix using 29 microservices and 317 Node.js instances.
- The new architecture utilizes various technologies like Node.js, MQLight for messaging, Redis for caching and real-time eventing, and Cloudant for persistence.
- Lessons learned include deciding how to break services into actors, testing complex cloud architectures, optimizing for different scaling needs, and choosing the right data store for read/write patterns.
- The evolution
Apache Geode is an open source in-memory data grid that provides data distribution, replication and high availability. It can be used for caching, messaging and interactive queries. The presentation discusses Geode concepts like cache, region and member. It provides examples of how large companies use Geode for applications requiring real-time response, high concurrency and global data visibility. Geode's performance comes from minimizing data copying and contention through flexible consistency and partitioning. The project is now hosted by Apache and the community is encouraged to get involved through mailing lists, code contributions and example applications.
Este documento describe la evolución de los grandes datos y la analítica, incluyendo el aumento de fuentes de datos, la comprensión de su valor, y la disminución de costos de hardware. También resume los componentes clave de Hadoop como HDFS, MapReduce, Hive y otros para el procesamiento y análisis de grandes cantidades de datos.
Creando tu primer ambiente de AI en Azure ML y SQL ServerEduardo Castro
Este documento proporciona una introducción a cómo crear el primer entorno de inteligencia artificial en Azure. Explica brevemente los beneficios de la inteligencia artificial y el aprendizaje automático para los negocios. Luego describe algunos de los servicios principales de Azure que pueden usarse para analizar datos, desarrollar modelos de aprendizaje automático y implementar soluciones de IA, como Azure Machine Learning, Databricks y HDInsight.
El documento describe las diferentes características y capacidades de seguridad disponibles en Azure SQL Database y Azure SQL Data Warehouse. Incluye gráficos que muestran el número de vulnerabilidades abordadas desde 2010 hasta 2018 y describe opciones como cifrado de datos en tránsito y en reposo, autenticación multifactor, firewalls, detección de amenazas, auditoría y más. El objetivo es ayudar a los clientes a proteger y auditar sus datos de manera segura en la nube.
Este documento describe cómo integrar Azure Synapse con MLflow para habilitar el seguimiento de experimentos de aprendizaje automático y el registro y despliegue de modelos en Azure Machine Learning. Explica cómo configurar los cuadernos de Azure Synapse para usar MLflow conectado a un área de trabajo de Azure Machine Learning, registrar modelos entrenados en Synapse en el registro de modelos de Azure ML y desplegarlos para su uso.
SQL Server can be installed on Windows Server 2022. Eduardo Castro provides a demonstration of how to install SQL Server on the latest Windows server operating system. His demonstration is available at a GitHub link that tracks an issue regarding documentation on installing SQL Server with Windows Server 2022.
El documento describe las nuevas características de SQL Server 2022, incluyendo la integración bidireccional con Azure SQL para replicación de datos, Azure Synapse Link para transferencia automática de cambios a Synapse Analytics, integración con Azure Purview para detección y clasificación de datos, mejoras en rendimiento a través de Query Store y optimización de planes, y mejoras en seguridad, disponibilidad y resolución de conflictos de réplicas.
SQL Server 2022 está habilitado para Azure para recuperación ante desastres, análisis y seguridad. Ofrece nuevas innovaciones como inteligencia de consultas integrada para mejorar el rendimiento, compatibilidad con almacenamiento de objetos y funciones extendidas de T-SQL para nuevos escenarios.
Machine Learning con Azure Managed InstanceEduardo Castro
En esta presentación mostramos las opciones para implementar Machine Learning dentro de Azure, así como las formas de configurar y utilizar Python dentro de Azure Managed Instance
El documento describe las nuevas características de SQL Server 2022, incluyendo la integración bidireccional con Azure SQL para replicación de datos, Azure Synapse Link para transferencia automática de cambios a Synapse Analytics, integración con Azure Purview para detección y clasificación de datos, mejoras en rendimiento a través de Query Store y optimización de planes, nuevas funciones de seguridad como ledger inmutable, y automatización de conflictos de réplicas en entornos de múltiples escrituras.
SQL Server can be installed on Windows Server 2022. Eduardo Castro provides a demonstration of how to install SQL Server on the latest Windows server operating system. His demonstration is available at a GitHub link that tracks an issue regarding documentation on installing SQL Server with Windows Server 2022.
Este documento presenta una introducción a Apache Spark y Azure Databricks. Explica que Spark es un motor de procesamiento de datos a gran escala de código abierto que incluye características como Spark SQL, aprendizaje automático, procesamiento de flujos y grafos. Luego describe cómo Azure Databricks es una plataforma unificada para análisis que utiliza Spark y ofrece mejor rendimiento, procesamiento de grandes volúmenes de datos y arquitectura de clústeres. Finalmente, incluye una demostración de las capacidades de
Este documento proporciona una introducción a los pronósticos con SQL Server 2019, discutiendo métodos como promedios móviles, suavizado exponencial, proyección de tendencias y regresión lineal. También describe cómo SQL Server 2019 permite a los científicos de datos y desarrolladores interactuar directamente con los datos y realizar análisis avanzados dentro de la base de datos, lo que puede aplicarse a soluciones como detección de fraude, pronósticos de ventas y mantenimiento predictivo.
Data warehouse con azure synapse analyticsEduardo Castro
Azure Synapse is the evolution of Azure SQL Data Warehouse, combining big data, data storage and data integration into a single service for end-to-end cloud scale analytics. It provides unlimited analytics with unparalleled speed to gain insights. Azure Synapse brings together enterprise data warehousing and big data analytics to give a unified experience with the advantages of both worlds.
Que hay de nuevo en el Azure Data Lake Storage Gen2Eduardo Castro
Este documento proporciona una actualización sobre las novedades de Azure Data Lake Storage. Incluye mejoras en el rendimiento, escalabilidad de costos, seguridad, soporte para almacenamiento de blobs y sistemas de archivos jerárquicos, y una vista previa de las integraciones con Azure Event Grid y Azure Synapse Analytics.
Azure Synapse Analytics es un servicio de análisis que combina big data, almacenamiento de datos e integración de datos en un solo servicio con escalabilidad en la nube. Ofrece análisis de datos end-to-end con tiempos de respuesta en segundos utilizando SQL, Python, R y otros lenguajes. Incluye características como ingesta de datos, almacenamiento de datos, análisis SQL, machine learning integrado y más.
Este documento presenta los Servicios Cognitivos de Microsoft, que proporcionan APIs de visión, habla, lenguaje y análisis de datos para permitir que las aplicaciones tengan capacidades como reconocimiento facial, detección de emociones, extracción de frases clave y comprensión del lenguaje natural. Los servicios cognitivos se pueden integrar fácilmente en aplicaciones y ayudan a los equipos de datos a resolver problemas en áreas como la atención médica, la seguridad y el comercio minorista.
Script de paso a paso de configuración de Secure EnclavesEduardo Castro
El documento proporciona instrucciones para configurar un equipo HGS como host protegido y otro equipo con SQL Server para usar enclaves seguros con Always Encrypted. Se explica cómo instalar el servicio de protección de host en HGS, configurar el dominio HGS, configurar la atestación de claves y obtener la dirección IP de HGS. Luego, se indica cómo configurar el equipo SQL Server como host protegido, generar y registrar su clave de host, e indicarle dónde debe realizar la atestación. Finalmente, se habilitan los en
Introducción a conceptos de SQL Server Secure EnclavesEduardo Castro
Este documento describe varias técnicas de cifrado de datos, incluido el cifrado de datos en reposo, en uso y en tránsito. Se centra en particular en Always Encrypted, una solución que permite cifrar datos sensibles en las columnas de una base de datos de forma que se mantengan las consultas enriquecidas. Explica cómo los datos cifrados se almacenan de forma segura utilizando claves maestras de columna almacenadas externamente, y cómo las aplicaciones pueden recuperar datos desencriptados de forma segura mediante el uso de encl
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfTechgropse Pvt.Ltd.
In this blog post, we'll delve into the intersection of AI and app development in Saudi Arabia, focusing on the food delivery sector. We'll explore how AI is revolutionizing the way Saudi consumers order food, how restaurants manage their operations, and how delivery partners navigate the bustling streets of cities like Riyadh, Jeddah, and Dammam. Through real-world case studies, we'll showcase how leading Saudi food delivery apps are leveraging AI to redefine convenience, personalization, and efficiency.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Infrastructure Challenges in Scaling RAG with Custom AI modelsZilliz
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
1. SPATIAL SUPPORT IN
SQL SERVER 2008 R2
Ing. Eduardo Castro Martinez
ecastro@simsasys.com
http://tiny.cc/comwindows
http://ecastrom.blogspot.com
2. Presentation Source
• SQL Server 2008 R2 Update for Developers Training Kit
• http://www.microsoft.com/download/en/details.aspx?id=16281
• Building Location-Aware Applications with the SQL Server
Spatial Library
• Ed Katibah, Torsten Grabs and Olivier Meyer SQL Server Microsoft
Corporation
3. Relational and Non-Relational Data
• Relational data uses simple data types
• Each type has a single value
• Generic operations work well with the types
• Relational storage/query may not be optimal for
• Hierarchical data
• Sparse, variable, property bags
• Some types
• benefit by using a custom library
• use extended type system (complex types, inheritance)
• use custom storage and non-SQL APIs
• use non-relational queries and indexing
4. Spatial Data
• Spatial data provides answers to location-based queries
• Which roads intersect the Microsoft campus?
• Does my land claim overlap yours?
• List all of the Italian restaurants within 5 kilometers
• Spatial data is part of almost every database
• If your database includes an address
5. Spatial Data Types
• The Open Geospatial Consortium defines a hierarchy of
spatial data types
• Point
• Linestring
• Polygon
• MultiPoint
• MultiLinestring
• MultiPolygon
• GeomCollection
• Non-instanciable classes based on these
7. SQL Server 2008 Spatial Summary
OVERVIEW FEATURES
• 2 Spatial Data Types (CLR UDT) • 2D Vector Data Support
• Comprehensive set of Spatial Methods • Open Geospatial Consortium Simple
• High Performance Spatial Indexes Features for SQL compatible
• Spatial Library • Supported By Major GIS Vendors
• Sink/Builder APIs ESRI, Intergraph, Autodesk, Pitney Bowes, Safe, etc.
• Management Studio Integration • Standard feature in SQL Server
Express, Workgroup, Web, Standard, Enterprise and
Developer
• Support for very large spatial objects
DETAILS
• Geography data type for geodetic Data
• Geometry data type for planar Data
• Standard spatial methods
STIntersects, STBuffer, STLength, STArea, etc.
• Standard spatial format support
WKT, WKB and GML
• Multiple spatial indexes per column
• Create new CLR-based spatial methods
with the Builder API
• Redistributable Spatial Library
SQLSysClrTypes
8. SQL Server Spatial Library Resources
SQL SERVER SPATIAL LIBRARY
Microsoft SQL Server System CLR Types
The SQL Server System CLR Types package contains the components
implementing the geometry, geography, and hierarchy id types in SQL Server
2008 R2. This component can be installed separately from the server to allow
client applications to use these types outside of the server.
X86 Package(SQLSysClrTypes_x86.msi) – 3,342 KB
X64 Package (SQLSysClrTypes._x64msi) – 3,459 KB
IA64 Package(SQLSysClrTypes_ia64.msi) – 5,352 KB
Search for: Microsoft SQL Server 2008 Feature Pack, October 2008
---
CODEPLEX SQL Server Spatial Tools
Code Samples Utilizing the SQL Server Spatial Library
SQL Server Spatial Tools – including source code for tools
Search for: Codeplex SQL Server Spatial Tools
9. SQL Server 2008 and Spatial Data
• SQL Server supports two spatial data types
• GEOMETRY - flat earth model
• GEOGRAPHY - round earth model
• Both types support all of the instanciable OGC types
• InstanceOf method can distinguish between them
• Supports two dimension data
• X and Y or Lat and Long members
• Z member - elevation (user-defined semantics)
• M member - measure (user-defined semantics)
10. GEOGRAPHY Requirements
• GEOGRAPHY type has additional requirements
• Coordinate order is
• Longitude/Latitude for WKT, WKB
• Latitude/Longitude for GML
• Exterior polygon rings must have their describing
coordinates in counter-clockwise order (left-hand rule)
with interior rings (holes) in clockwise-order (right-hand
rule)
• A single GEOGRAPHY object cannot span more than a
logical hemisphere
12. Properties and Methods
• The spatial data types are exposed as SQLCLR UDTs
• Use '.' syntax for properties
• Use '.' syntax for instance methods
• Use '::' syntax for static methods
• Methods and Properties are case-sensitive
• Each type uses a set of properties and methods that
correspond to OGC functionality
• With Extensions
• Geometry implements all OGC properties and methods
• Geography implements most OGC properties and methods
• 2-D vector only implemented
13. Input
• Spatial data is stored in a proprietary binary format
• Instance of the type can be NULL
• Can be input as
• Well Known binary - ST[Type]FromWKB
• Well Known text - ST[Type]FromText
• Geography Markup Language (GML) - GeomFromGml
• Can also use SQLCLR functions
• Parse
• Point - extension function
• Input from SQLCLR Type - SqlGeometry, SqlGeography
• Spatial builder API –
Populate, IGeometrySink/IGeographySink
14. Output
• Spatial Data Can Be Output As
• Well Known binary - STAsBinary
• Well Known text - STAsText
• GML - AsGml
• Text with Z and M values - AsTextZM
• SQLCLR standard method
• ToString - returns Well Known text
• As SQLCLR object - SqlGeometry, SqlGeography
• Other useful formats are GeoRSS, KML
• Not Directly Supported
15. SRID
• Each instance of a spatial type must have an SRID
• Spatial Reference Identifier
• SRID specifies the specification used to compute it
• SRID 4326 - GPS, default for GEOGRAPHY
• SRID 4269 - usually used by ESRI
• SRID 0 - no special reference, default for GEOMETRY
• Methods that use multiple spatial types (e.g., STDistance)
must have types with matching SRID
• Else method returns NULL
• Geography instance must reference one of these SRID
stored in sys.spatial_reference_systems
17. Sample Query
SELECT *
Which roads intersect Microsoft’s main
FROM roads campus?
WHERE roads.geom.STIntersects(@ms)=1
18. Extension Methods
• SQL Server 2008 extends OGC methods
• MakeValid - Converts to OGC valid instance
• BufferWithTolerence - similar to STBuffer, allows approximation and
variation
• Reduce - Simplify a complex geography or geometry
• NumRings, RingN - polygons with multiple rings
• GML support
• Z and M properties and AsTextZM method
• Filter - provides a quick intersection set but with false positives
• EnvelopeCenter,EnvelopeAngle for Geography types
19. Spatial Indexes
• SQL Server Spatial Indexes Based on B-Trees
• Uses tessellation to tile 2D to linear
• Divides space into grid of cells(uses Hilbert algorithm)
• Meant as a first level of row elimination
• Can produce false positives
• Never false negatives
• You specify
• Bounding box of top level grid - GEOMETRY index only
• Cells per object - number of cells recorded for matching
• Grids
• Four Grid Levels
• Three Grid Densities Per Level - Low, Medium, High
22. What is CEP?
Complex Event Processing (CEP) is the continuous and
incremental processing of event streams from multiple
sources based on declarative query and pattern specifications
with near-zero latency.
Database Applications Event-driven Applications
Query Ad-hoc queries or Continuous standing
Paradigm requests queries
Latency Seconds, hours, days Milliseconds or less
Data Rate Hundreds of events/sec Tens of thousands of
events/sec or more
Event
request
output
input stream
response stream
23. Shuttle Tracker
519,000+ data points, covering 1 day of operation
24. Review
• Spatial data provides answers to location-based queries
• SQL Server supports two spatial data types
• GEOMETRY - flat earth model
• GEOGRAPHY - round earth model
• Spatial data has
• Useful properties and functions
• Library of spatial functions
• Three standard input and output formats
• Spatial indexes
25. Resources
• SQL Server Spatial Data Technology Center
http://www.microsoft.com/sql/2008/technologies/spatial.mspx
• Whitepaper: Delivering Location Intelligence with Spatial Data
http://www.microsoft.com/sql/techinfo/whitepapers/spatialdata.mspx
• MSDN Webcast: Building Spatial Applications with SQL Server
2008, Event ID: 1032353123
• Whitepaper: What's New for XML in SQL Server 2008
http://www.microsoft.com/sql/techinfo/whitepapers/sql_2008_xml.mspx
• Whitepaper: Managing Unstructured Data with SQL Server
2008
http://www.microsoft.com/sql/techinfo/whitepapers/sql_2008_unstructure
d.mspx
Simple types only store one value, and work with generic methods like adding, subtracting, assignment. Relational storage isn’t optimal for storing and querying hierarchies, location based data, or coping with sparse data where the data required per row differs.Some data types benefit from having complex logic for parsing and processing, and the benefit from other programming features such as inheritance. What's more the storage requirements and indexing requirements require special processing as the storage is not byte ordered.
Reference: http://www.sqlskills.com/BLOGS/BOBB/post/Spatial-Data-a-niche-or-a-tool-for-the-masses.aspxSpatial data allows you to answer questions in 2 dimensions, often related to location based information. Almost every database contains location information, i.e.. Addresses, however they don’t know where those addresses are. They don’t know that 4th street is next to 3rd street and that 1st avenue intersects both of them.
The spatial implementation in SQL Server 2008 is based on the OGC standards.There is a hierarchy of Spatial data instances starting with a point then a line and finally a polygon. This relates to their dimensionality, a point has 0 dimensions, a line has 1 and a polygon has 2. You also have MULTI instances which contain more than one of the other types and then the collection type which can contain any of the other types
This is the hierarchy.Not the terms curve and surface, this implies 3 dimensions which SQL Server spatial data doesn’t support, it only supports 2 dimensions. More on that later.
Unlike other providers SQL splits planar and non planar data into two separate types. Geometry and geography.Non planar data is all about projections. If you think about maps of the world they have to use some form of projection to convert the elliptical surface of the earth into a flat map. In other providers they store the data in a planar projection and then perform the projection to workout the geographic calculations.We mentioned earlier that the spatial data is only in 2 dimensions. Whilst you can hold two other values Z and M for points they are not used in any of the spatial methods.
The geometry type covers a planar surface which is infinite your x and y can go on for ever. With the Geography type that's not the case. Latitude goes from -90 degrees to 90degrees. Longitude goes from 0 to 360.The order of how these will be parsed in the WKB (Well-Known Binary) and WKT (Well-Known Text) formats will be switched before RTM to fit in with the other products on the market.A geography shape must be contained with a single hemisphere. This doesn’t have to the north or south hemisphere but half of the earth essentially the maximum angle mad at the centre by lines to the edges cannot be greater than 180.Because the earth is a finite shape, you have to define your points in a certain order to ensure that you include the area within your shape and not the area outside of the shape. If you find you shapes are violating the hemisphere rule you might have your points going in the wrong order.
This demo will show how to use the SQL Server spatial data types with data from the Mondial database, which contains latitude and longitude fields for three of the tables: City, Mountain, and Island. We will use this information to construct a new column of type Geography, and then use the spatial methods of the Geography data type to perform SQL spatial queries on this data.The material for this demo can be found at the Source folder of the Spatial demo. Make sure to read the demo document to get started with this demo.
Reference: http://www.sqlskills.com/blogs/bobb/2007/06/24/SQLServerSystemDataTypesImplementedInNET.aspx Some new SQL Server 2008 data types are implemented/exposed as .NET UDTs. These include the spatial data types Geometry and Geography, and also HierarchyID, mentioned later in this slideset.The Date/Time series of data type ARE NOT implemented as UDT, and therefore are referred to as "new T-SQL data types".Spatial data types are implemented as CLR User defined types. YOU DO NOT HAVE TO ENABLE CLR ON THE SERVER TO USE SPATIAL DATA TYPES.You get some benefits because of this. You can call methods on the types directly. You can also call static methods on the base type by using ::. It is the later that can be used to create instances of the types.Note: You are calling methods on the types directly and so the methods and properties are case sensitive.The OGC spec defines a set of methods that should be supported, the Geometry type supports all of them. The Geography type doesn’t. It will support more by RTM but not all of them.
Spatial data is created from a defined format, that is either a text base format, a binary format or an XML format.WKT is the text representationWKB is the binary formGML is the XML structure.Only WKT support passing in Z and M values.WKT is more readable but as its text it doesn’t perform as well as the binary format. It is stored in the database in a proprietary binary format. If you have that format you can also create an instance from that, if you are passing data from a client application using this format is the best to use. You create an instance by assigning a string to a type, using parse, or one of the STFROM... Methods.Much of the data in the industry is in shape file format. This is a format define by ESRI and for which you will need an application to convert to a format usable by SQL ServerFor more information on the spatial builder API and an example, see http://blogs.msdn.com/isaac/archive/2008/05/30/our-upcoming-builder-api.aspx
You can output any of the formats that are used to create an instance.Other formats used by applications such as Google maps an virtual earth are not supported directly. You have to create these yourself more on that later.
SRID defines the unitof measure datum (shape of the ellipsoid) and projection. Each piece of geography data requires a spatial reference.You cannot combine data of different SRIDs, as this is like comparing apples and pears.You need to make sure you use the correct reference to ensure you calculations are correct. Lengths and areas will differ under different spatial references when dealing with geography data.There is no mechanism for translating between spatial references in SQL Server. You will need a 3rd party application to achieve that.
One of the killer features of spatial types is the methods they bring. You have methods that provide information about shapes.Methods that show the relationship between shapesMethods used to combine and manipulate shapes. If you’ve done set theory and venn diagrams these methods will make sense.Finally methods that provide information about the structure of the shape. Because of the limitations of SQLCLR these are not collections put rather methods that take an index to return the relevant value.
SQL Server extends the spatial types beyond what is in the spec. The following are the methods and there are more coming. EnveloperCentre and EnvelopeAngle provide the information used to determine the bounds of the geography type and whether it fits in a hemisphere.
Index is an internal table. One row represents a hierarchy of intersections of cells with in the gridEach cell that a shape intersects is recorded in the index.4 levels of grid used to provide greater accuracy. Each cell in grid is split into x more cells in the next level grid.Number of cells in each grid is defined when you create the index.
Top three show the level 1, Level 2 and then Level 3 and 4 intersections.If we stored all of these intersections we would have 85 matches. What we do find is that some of the cells are complete matches these don’t need to be broken down to the lower level cells. Like in Figure 4.However we still have a large number of matches, depending on the cells per object setting on the index the tessellation process will stop once it hits the limit as we have in Figure 5.This shows how imprecise the index is. It is only meant as a filter to avoid doing a very expensive calculation on all the data.
This demo is aimed to show the spatial capabilities of SQL Server 2008 integrated with ASP.NET. The demo relies on SQL Server to store and perform calculations on the new Geography data types; and on Virtual Earth to consume the spatial data and render it on a map.The material for this demo can be found at the Source folder of the Spatial demo. Make sure to read the demo document to get started with this demo.