19 reasons to join first international Oracle conference in Poland
•
1 like•665 views
POUG Chapter IV is a first international Oracle community conference in Poland and fourth meeting od Pint with Oracle User Group. During the 2-days-long event in Hotel Lord, Warsaw, you will be able to listen 20 experts from all over the world. Besides the possibility to gain database knowledge, participants will have a chance to discuss and integrate during the after party with open bar. Detailes and tickets: www.poug.org
Report
Share
Report
Share
Download to read offline
Recommended
An Engineering Approach to Database Evaluations
This talk will go over a methodical approach for making a decision, dig into interesting tradeoffs, and give tips about what things to look for under the hood and how to evaluate the tech behind the database.
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Python Notebooks are great for communicating data analysis & research but how do you port these data visualizations between the many available platforms (Jupyter, Databricks, Zeppelin, Colab,…). Also learn about how to scale up your visualizations using Spark
Real World NoSQL (by Chris Yuen)
The Hong Kong Big Data community had a guest speaker at our Tuesday, 18 February meeting. Chris Yuen from Demyst Data discussed his experience with three NoSQL solutions: Cassandra, MongoDB, and HBase. For more information see http://www.infoincog.com/hong-kong-big-data-meeting-tuesday-18-february/.
Delivering Insights from 20M+ Smart Homes with 500M+ Devices
We started out processing big data using AWS S3, EMR clusters, and Athena to serve Analytics data extracts to Tableau BI.
However as our data and teams sizes increased, Avro schemas from source data evolved, and we attempted to serve analytics data through Web apps, we hit a number of limitations in the AWS EMR, Glue/Athena approach.
This is a story of how we scaled out our data processing and boosted team productivity to meet our current demand for insights from 20M+ Smart Homes and 500M+ devices across the globe, from numerous internal business teams and our 150+ CSP partners.
We will describe lessons learnt and best practices established as we enabled our teams with DataBricks autoscaling Job clusters and Notebooks and migrated our Avro/Parquet data to use MetaStore, SQL Endpoints and SQLA Console, while charting the path to the Delta lake…
Azure database as a service options
The document summarizes information about a conference on relational database options in Azure. It provides an agenda that includes introductions to Azure SQL Database, SQL Data Warehouse, Azure CosmosDB, and other database platforms on Azure. It also discusses infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS) models and options for migrating databases to Azure. The presentation aims to help attendees learn about relational database options in Microsoft's cloud computing platform.
Getting Started with Databricks SQL Analytics
It has long been said that business intelligence needs a relational warehouse, but that view is changing. With the Lakehouse architecture being shouted from the rooftops, Databricks have released SQL Analytics, an alternative workspace for SQL-savvy users to interact with an analytics-tuned cluster. But how does it work? Where do you start? What does a typical Data Analyst’s user journey look like with the tool?
This session will introduce the new workspace and walk through the various key features – how you set up a SQL Endpoint, the query workspace, creating rich dashboards and connecting up BI tools such as Microsoft Power BI.
If you’re truly trying to create a Lakehouse experience that satisfies your SQL-loving Data Analysts, this is a tool you’ll need to be familiar with and include in your design patterns, and this session will set you on the right path.
Scylla Summit 2018: Scylla Feature Talks - Gains by Using Scylla-Specific Dri...
Scylla offers full compatibility with Cassandra, meaning any Cassandra application works unmodified with Scylla using the standard drivers available to the community. While this will still be the case, there are many gains to be had by using Scylla-specific drivers. Those drivers will work with both Cassandra and Scylla but upon detecting scylla-specific features they will able to unleash many performance optimizations. In this talk we will ll cover which scenarios will benefit from the Scylla drivers and what’s coming next.
ETL Made Easy with Azure Data Factory and Azure Databricks
This document summarizes Mark Kromer's presentation on using Azure Data Factory and Azure Databricks for ETL. It discusses using ADF for nightly data loads, slowly changing dimensions, and loading star schemas into data warehouses. It also covers using ADF for data science scenarios with data lakes. The presentation describes ADF mapping data flows for code-free data transformations at scale in the cloud without needing expertise in Spark, Scala, Python or Java. It highlights how mapping data flows allow users to focus on business logic and data transformations through an expression language and provides debugging and monitoring of data flows.
Recommended
An Engineering Approach to Database Evaluations
This talk will go over a methodical approach for making a decision, dig into interesting tradeoffs, and give tips about what things to look for under the hood and how to evaluate the tech behind the database.
Portable Scalable Data Visualization Techniques for Apache Spark and Python N...
Python Notebooks are great for communicating data analysis & research but how do you port these data visualizations between the many available platforms (Jupyter, Databricks, Zeppelin, Colab,…). Also learn about how to scale up your visualizations using Spark
Real World NoSQL (by Chris Yuen)
The Hong Kong Big Data community had a guest speaker at our Tuesday, 18 February meeting. Chris Yuen from Demyst Data discussed his experience with three NoSQL solutions: Cassandra, MongoDB, and HBase. For more information see http://www.infoincog.com/hong-kong-big-data-meeting-tuesday-18-february/.
Delivering Insights from 20M+ Smart Homes with 500M+ Devices
We started out processing big data using AWS S3, EMR clusters, and Athena to serve Analytics data extracts to Tableau BI.
However as our data and teams sizes increased, Avro schemas from source data evolved, and we attempted to serve analytics data through Web apps, we hit a number of limitations in the AWS EMR, Glue/Athena approach.
This is a story of how we scaled out our data processing and boosted team productivity to meet our current demand for insights from 20M+ Smart Homes and 500M+ devices across the globe, from numerous internal business teams and our 150+ CSP partners.
We will describe lessons learnt and best practices established as we enabled our teams with DataBricks autoscaling Job clusters and Notebooks and migrated our Avro/Parquet data to use MetaStore, SQL Endpoints and SQLA Console, while charting the path to the Delta lake…
Azure database as a service options
The document summarizes information about a conference on relational database options in Azure. It provides an agenda that includes introductions to Azure SQL Database, SQL Data Warehouse, Azure CosmosDB, and other database platforms on Azure. It also discusses infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS) models and options for migrating databases to Azure. The presentation aims to help attendees learn about relational database options in Microsoft's cloud computing platform.
Getting Started with Databricks SQL Analytics
It has long been said that business intelligence needs a relational warehouse, but that view is changing. With the Lakehouse architecture being shouted from the rooftops, Databricks have released SQL Analytics, an alternative workspace for SQL-savvy users to interact with an analytics-tuned cluster. But how does it work? Where do you start? What does a typical Data Analyst’s user journey look like with the tool?
This session will introduce the new workspace and walk through the various key features – how you set up a SQL Endpoint, the query workspace, creating rich dashboards and connecting up BI tools such as Microsoft Power BI.
If you’re truly trying to create a Lakehouse experience that satisfies your SQL-loving Data Analysts, this is a tool you’ll need to be familiar with and include in your design patterns, and this session will set you on the right path.
Scylla Summit 2018: Scylla Feature Talks - Gains by Using Scylla-Specific Dri...
Scylla offers full compatibility with Cassandra, meaning any Cassandra application works unmodified with Scylla using the standard drivers available to the community. While this will still be the case, there are many gains to be had by using Scylla-specific drivers. Those drivers will work with both Cassandra and Scylla but upon detecting scylla-specific features they will able to unleash many performance optimizations. In this talk we will ll cover which scenarios will benefit from the Scylla drivers and what’s coming next.
ETL Made Easy with Azure Data Factory and Azure Databricks
This document summarizes Mark Kromer's presentation on using Azure Data Factory and Azure Databricks for ETL. It discusses using ADF for nightly data loads, slowly changing dimensions, and loading star schemas into data warehouses. It also covers using ADF for data science scenarios with data lakes. The presentation describes ADF mapping data flows for code-free data transformations at scale in the cloud without needing expertise in Spark, Scala, Python or Java. It highlights how mapping data flows allow users to focus on business logic and data transformations through an expression language and provides debugging and monitoring of data flows.
Eugene Polonichko "Architecture of modern data warehouse"
The document discusses the architecture of a modern data warehouse using Microsoft technologies. It describes traditional data warehousing approaches and outlines ten characteristics of a modern data warehouse. It then details Microsoft's approach using Azure Data Factory to ingest diverse data types into Azure Blob Storage, Azure Databricks for analytics and data transformation, and Azure SQL Data Warehouse for combined structured data. It also discusses technologies for storage, visualization, and links for further information.
Ssi Tech Arch Overview 03072009
Synergy Systems provides technical architecture support for their clients including application architecture using Apache, JBoss, and PostgreSQL. Their networking uses Linux routers and F5 load balancers. A data center review showed aging hardware that is over 5 years old. Business is growing rapidly and forecast to increase transactions from 75,000 to 500,000 per month. Contingency options discussed include a traditional contingency site, regional processing centers, and virtual machinery.
Data virtualization using polybase
This document provides an overview of using Polybase for data virtualization in SQL Server. It discusses installing and configuring Polybase, connecting external data sources like Azure Blob Storage and SQL Server, using Polybase DMVs for monitoring and troubleshooting, and techniques for optimizing performance like predicate pushdown and creating statistics on external tables. The presentation aims to explain how Polybase can be leveraged to virtually access and query external data using T-SQL without needing to know the physical data locations or move the data.
Introduction SQL Analytics on Lakehouse Architecture
This document provides an introduction and overview of SQL Analytics on Lakehouse Architecture. It discusses the instructor Doug Bateman's background and experience. The course goals are outlined as describing key features of a data Lakehouse, explaining how Delta Lake enables a Lakehouse architecture, and defining features of the Databricks SQL Analytics user interface. The course agenda is then presented, covering topics on Lakehouse Architecture, Delta Lake, and a Databricks SQL Analytics demo. Background is also provided on Lakehouse architecture, how it combines the benefits of data warehouses and data lakes, and its key features.
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN✔ Eric David Benari, PMP
This document summarizes features in MariaDB 10.1 and 10.2. It discusses how MariaDB was created to ensure a free and community developed version of MySQL always exists. It provides overviews of the MariaDB foundation and goals. It also summarizes many new features and improvements in areas like performance, replication, security and more. Intro to databricks delta lake
Delta Lake is an open-source storage layer that brings ACID
transactions to Apache Spark™ and big data workloads.
high performance databases
Mehdi Varse presented on high performance databases. The presentation outlined performance metrics to monitor in enterprise applications including business transactions, query performance, user and query conflicts, capacity, configuration, and NoSQL databases. It also discussed database tuning, in-memory databases, parallel and distributed database systems, and high-performance database requirements. Examples of databases used at Facebook like MySQL, Memcached, Haystack, and Cassandra were reviewed.
MemSQL 201: Advanced Tips and Tricks Webcast
This document summarizes a webinar on advanced tips and tricks for MemSQL. It discusses the differences between rowstore and columnstore storage models and when each is best used. It also covers data ingestion using MemSQL Pipelines for real-time loading, data sharding and query tuning techniques like using reference tables. Additionally, it discusses monitoring memory usage, workload management using management views, and query optimization tools like analyzing and optimizing tables.
L’approche Big Data en finance de marché 1/2
Quartet FS provides the in-memory analytics platform ActivePivot to enable operational decision making for financial institutions working with large, time-sensitive data. ActivePivot allows customers to embed high-performance analytics into processes to take action in real-time. It can perform complex calculations on large data sets at high speeds with flexibility. Quartet FS has over 50 customers and offices worldwide, and ActivePivot provides an open framework to aggregate and calculate data from multiple sources in real-time to power interactive analysis and alerts.
Accelerate Your ML Pipeline with AutoML and MLflow
Building ML models is a time consuming endeavor that requires a thorough understanding of feature engineering, selecting useful features, choosing an appropriate algorithm, and performing hyper-parameter tuning. Extensive experimentation is required to arrive at a robust and performant model. Additionally, keeping track of the models that have been developed and deployed may be complex. Solving these challenges is key for successfully implementing end-to-end ML pipelines at scale.
In this talk, we will present a seamless integration of automated machine learning within a Databricks notebook, thus providing a truly unified analytics lifecycle for data scientists and business users with improved speed and efficiency. Specifically, we will show an app that generates and executes a Databricks notebook to train an ML model with H2O’s Driverless AI automatically. The resulting model will be automatically tracked and managed with MLflow. Furthermore, we will show several deployment options to score new data on a Databricks cluster or with an external REST server, all within the app.
The Last Mile: Challenges and Opportunities in Data Tools (Strata 2014)
The document discusses challenges and opportunities in data tools. It summarizes various programming libraries and tools used for data analysis, manipulation, and visualization including pandas, dplyr, Spark, and analytic databases. It also discusses trends toward SQL-on-Hadoop, improved data preparation tools, and standardized machine learning toolkits. The author advocates for more integrated environments that abstract away execution details and enhance collaboration capabilities.
Spark introduction and architecture
Introduction to Apache Spark, understanding of the architecture, resilient distributed datasets and working.
Simplify Data Conversion from Spark to TensorFlow and PyTorch
In this talk, I would like to introduce an open-source tool built by our team that simplifies the data conversion from Apache Spark to deep learning frameworks.
Imagine you have a large dataset, say 20 GBs, and you want to use it to train a TensorFlow model. Before feeding the data to the model, you need to clean and preprocess your data using Spark. Now you have your dataset in a Spark DataFrame. When it comes to the training part, you may have the problem: How can I convert my Spark DataFrame to some format recognized by my TensorFlow model?
The existing data conversion process can be tedious. For example, to convert an Apache Spark DataFrame to a TensorFlow Dataset file format, you need to either save the Apache Spark DataFrame on a distributed filesystem in parquet format and load the converted data with third-party tools such as Petastorm, or save it directly in TFRecord files with spark-tensorflow-connector and load it back using TFRecordDataset. Both approaches take more than 20 lines of code to manage the intermediate data files, rely on different parsing syntax, and require extra attention for handling vector columns in the Spark DataFrames. In short, all these engineering frictions greatly reduced the data scientists’ productivity.
The Databricks Machine Learning team contributed a new Spark Dataset Converter API to Petastorm to simplify these tedious data conversion process steps. With the new API, it takes a few lines of code to convert a Spark DataFrame to a TensorFlow Dataset or a PyTorch DataLoader with default parameters.
In the talk, I will use an example to show how to use the Spark Dataset Converter to train a Tensorflow model and how simple it is to go from single-node training to distributed training on Databricks.
Azure SQL Data Warehouse for beginners
Slides from QSSUG Aug 2017 by David Alzamendi:
When on-premise, Data Warehouses are not the only option, many questions arise surrounding Azure SQL Data Warehouse.
In this session, David will cover the fundamentals of using Azure SQL Data Warehouse from a beginner's perspective. He'll discuss the benefits, demystify the pricing measurements and explain the difference between Azure SQL Database and Big Data.
By the end of this session, you will know how to deploy this service in just a few minutes using some of the latest techniques like extracting data from Azure data lakes and accessing Azure blob storage through PolyBase.
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 2)
Im zweiten Teil unserer Microsoft Big Data Session geht es darum, wie Big Data Informationen über "klassisches" SQL zugänglich gemacht werden können und wie sich mit der neuen PolyBase-Engine unstrukturierte Hadoop-Daten mit relationalen Data Warehouse-Daten einfach verknüpfen lassen.
In der Hadoop-Welt wird der SQL-Zugriff über die Komponente Hive ermöglicht.
Über den Microsoft Hive ODBC-Konnektor können die üblichen BI-Tools, wie PowerPivot, diesen Zugriff direkt nutzen.
Die PolyBase-Engine schließlich wird ein Bestandteil des SQL Server 2012 Parallel Data Warehouse werden und erlaubt einem transparenten SQL-Zugriff, egal, wo sich die Daten befinden.
SQL To NoSQL - Top 6 Questions Before Making The Move
Learn what you need to consider when moving from the world of relational databases to a NoSQL document store.
Hear from Developer Advocate Glynn Bird as he explains the key differences between relational databases and JSON document stores like Cloudant, as well as how to dodge the pitfalls of migrating from a relational database to NoSQL.
Qubole - Big data in cloud
Big Data Challenges and How to Overcome Them with Qubole - a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds. Storing, accessing, and analyzing large amounts of data from diverse sources and making it easily accessible to deliver actionable insights for users can be challenging for data driven organizations. The solution for customers is to optimize scaling and create a unified interface to simplify analysis. Qubole helps customers simplify their big data analytics with speed and scalability, while providing data analysts and scientists self-service access in Cloud. The platform is fully elastic and automatically scales or contracts clusters based on workload. We will try to overview main features, advantages and drawback of this platform.
Build Your Own Data Beast : Greenplum + Dell
Looking for a software and hardware stack that can get you results with big data now? Greenplum Database plus Dell hardware will help you construct a data science monster.
Processing Large Datasets for ADAS Applications using Apache Spark
Semantic segmentation is the classification of every pixel in an image/video. The segmentation partitions a digital image into multiple objects to simplify/change the representation of the image into something that is more meaningful and easier to analyze [1][2]. The technique has a wide variety of applications ranging from perception in autonomous driving scenarios to cancer cell segmentation for medical diagnosis.
Exponential growth in the datasets that require such segmentation is driven by improvements in the accuracy and quality of the sensors generating the data extending to 3D point cloud data. This growth is further compounded by exponential advances in cloud technologies enabling the storage and compute available for such applications. The need for semantically segmented datasets is a key requirement to improve the accuracy of inference engines that are built upon them.
Streamlining the accuracy and efficiency of these systems directly affects the value of the business outcome for organizations that are developing such functionalities as a part of their AI strategy.
This presentation details workflows for labeling, preprocessing, modeling, and evaluating performance/accuracy. Scientists and engineers leverage domain-specific features/tools that support the entire workflow from labeling the ground truth, handling data from a wide variety of sources/formats, developing models and finally deploying these models. Users can scale their deployments optimally on GPU-based cloud infrastructure to build accelerated training and inference pipelines while working with big datasets. These environments are optimized for engineers to develop such functionality with ease and then scale against large datasets with Spark-based clusters on the cloud.
Scaling Data and ML with Apache Spark and Feast
Gojek, Indonesia’s first billion-dollar startup, has seen an explosive growth in both users and data over the past three years.
22059 slides
The document discusses how IntelliReal built a low-cost virtualized computing grid using a master Oracle database and a partitioned MySQL grid to scale their real estate property data. It describes their initial architecture of partitioning the 400GB of data from Oracle across 800 MySQL databases, which later expanded to over 10,000 partitions. It also outlines the lessons learned around managing their own servers and the networking bandwidth required. Finally, it discusses how this approach helped IntelliReal scale quickly and displace competitors in the market.
MT42 The impact of high performance Oracle workloads on the evolution of the ...
Increased data, along with innovations in application development have led to increasing I/O demands, which are not being met by existing architectures. Find out how high performance applications, particularly analytics applications running on a variety of files systems, are being constrained by storage performance and how Dell EMC's broad portfolio of storage infrastructure can meet their extreme performance demands.
Discover how Dell EMC's revolutionary performance can help you streamline and improve the performance of your entire Oracle environment. Performance and cost comparisons will show you how Dell EMC's performance is not just for extreme workloads but can also help you achieve massive consolidation, more simplified data architectures, increased data agility and reduced management overhead.
"
More Related Content
What's hot
Eugene Polonichko "Architecture of modern data warehouse"
The document discusses the architecture of a modern data warehouse using Microsoft technologies. It describes traditional data warehousing approaches and outlines ten characteristics of a modern data warehouse. It then details Microsoft's approach using Azure Data Factory to ingest diverse data types into Azure Blob Storage, Azure Databricks for analytics and data transformation, and Azure SQL Data Warehouse for combined structured data. It also discusses technologies for storage, visualization, and links for further information.
Ssi Tech Arch Overview 03072009
Synergy Systems provides technical architecture support for their clients including application architecture using Apache, JBoss, and PostgreSQL. Their networking uses Linux routers and F5 load balancers. A data center review showed aging hardware that is over 5 years old. Business is growing rapidly and forecast to increase transactions from 75,000 to 500,000 per month. Contingency options discussed include a traditional contingency site, regional processing centers, and virtual machinery.
Data virtualization using polybase
This document provides an overview of using Polybase for data virtualization in SQL Server. It discusses installing and configuring Polybase, connecting external data sources like Azure Blob Storage and SQL Server, using Polybase DMVs for monitoring and troubleshooting, and techniques for optimizing performance like predicate pushdown and creating statistics on external tables. The presentation aims to explain how Polybase can be leveraged to virtually access and query external data using T-SQL without needing to know the physical data locations or move the data.
Introduction SQL Analytics on Lakehouse Architecture
This document provides an introduction and overview of SQL Analytics on Lakehouse Architecture. It discusses the instructor Doug Bateman's background and experience. The course goals are outlined as describing key features of a data Lakehouse, explaining how Delta Lake enables a Lakehouse architecture, and defining features of the Databricks SQL Analytics user interface. The course agenda is then presented, covering topics on Lakehouse Architecture, Delta Lake, and a Databricks SQL Analytics demo. Background is also provided on Lakehouse architecture, how it combines the benefits of data warehouses and data lakes, and its key features.
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN✔ Eric David Benari, PMP
This document summarizes features in MariaDB 10.1 and 10.2. It discusses how MariaDB was created to ensure a free and community developed version of MySQL always exists. It provides overviews of the MariaDB foundation and goals. It also summarizes many new features and improvements in areas like performance, replication, security and more. Intro to databricks delta lake
Delta Lake is an open-source storage layer that brings ACID
transactions to Apache Spark™ and big data workloads.
high performance databases
Mehdi Varse presented on high performance databases. The presentation outlined performance metrics to monitor in enterprise applications including business transactions, query performance, user and query conflicts, capacity, configuration, and NoSQL databases. It also discussed database tuning, in-memory databases, parallel and distributed database systems, and high-performance database requirements. Examples of databases used at Facebook like MySQL, Memcached, Haystack, and Cassandra were reviewed.
MemSQL 201: Advanced Tips and Tricks Webcast
This document summarizes a webinar on advanced tips and tricks for MemSQL. It discusses the differences between rowstore and columnstore storage models and when each is best used. It also covers data ingestion using MemSQL Pipelines for real-time loading, data sharding and query tuning techniques like using reference tables. Additionally, it discusses monitoring memory usage, workload management using management views, and query optimization tools like analyzing and optimizing tables.
L’approche Big Data en finance de marché 1/2
Quartet FS provides the in-memory analytics platform ActivePivot to enable operational decision making for financial institutions working with large, time-sensitive data. ActivePivot allows customers to embed high-performance analytics into processes to take action in real-time. It can perform complex calculations on large data sets at high speeds with flexibility. Quartet FS has over 50 customers and offices worldwide, and ActivePivot provides an open framework to aggregate and calculate data from multiple sources in real-time to power interactive analysis and alerts.
Accelerate Your ML Pipeline with AutoML and MLflow
Building ML models is a time consuming endeavor that requires a thorough understanding of feature engineering, selecting useful features, choosing an appropriate algorithm, and performing hyper-parameter tuning. Extensive experimentation is required to arrive at a robust and performant model. Additionally, keeping track of the models that have been developed and deployed may be complex. Solving these challenges is key for successfully implementing end-to-end ML pipelines at scale.
In this talk, we will present a seamless integration of automated machine learning within a Databricks notebook, thus providing a truly unified analytics lifecycle for data scientists and business users with improved speed and efficiency. Specifically, we will show an app that generates and executes a Databricks notebook to train an ML model with H2O’s Driverless AI automatically. The resulting model will be automatically tracked and managed with MLflow. Furthermore, we will show several deployment options to score new data on a Databricks cluster or with an external REST server, all within the app.
The Last Mile: Challenges and Opportunities in Data Tools (Strata 2014)
The document discusses challenges and opportunities in data tools. It summarizes various programming libraries and tools used for data analysis, manipulation, and visualization including pandas, dplyr, Spark, and analytic databases. It also discusses trends toward SQL-on-Hadoop, improved data preparation tools, and standardized machine learning toolkits. The author advocates for more integrated environments that abstract away execution details and enhance collaboration capabilities.
Spark introduction and architecture
Introduction to Apache Spark, understanding of the architecture, resilient distributed datasets and working.
Simplify Data Conversion from Spark to TensorFlow and PyTorch
In this talk, I would like to introduce an open-source tool built by our team that simplifies the data conversion from Apache Spark to deep learning frameworks.
Imagine you have a large dataset, say 20 GBs, and you want to use it to train a TensorFlow model. Before feeding the data to the model, you need to clean and preprocess your data using Spark. Now you have your dataset in a Spark DataFrame. When it comes to the training part, you may have the problem: How can I convert my Spark DataFrame to some format recognized by my TensorFlow model?
The existing data conversion process can be tedious. For example, to convert an Apache Spark DataFrame to a TensorFlow Dataset file format, you need to either save the Apache Spark DataFrame on a distributed filesystem in parquet format and load the converted data with third-party tools such as Petastorm, or save it directly in TFRecord files with spark-tensorflow-connector and load it back using TFRecordDataset. Both approaches take more than 20 lines of code to manage the intermediate data files, rely on different parsing syntax, and require extra attention for handling vector columns in the Spark DataFrames. In short, all these engineering frictions greatly reduced the data scientists’ productivity.
The Databricks Machine Learning team contributed a new Spark Dataset Converter API to Petastorm to simplify these tedious data conversion process steps. With the new API, it takes a few lines of code to convert a Spark DataFrame to a TensorFlow Dataset or a PyTorch DataLoader with default parameters.
In the talk, I will use an example to show how to use the Spark Dataset Converter to train a Tensorflow model and how simple it is to go from single-node training to distributed training on Databricks.
Azure SQL Data Warehouse for beginners
Slides from QSSUG Aug 2017 by David Alzamendi:
When on-premise, Data Warehouses are not the only option, many questions arise surrounding Azure SQL Data Warehouse.
In this session, David will cover the fundamentals of using Azure SQL Data Warehouse from a beginner's perspective. He'll discuss the benefits, demystify the pricing measurements and explain the difference between Azure SQL Database and Big Data.
By the end of this session, you will know how to deploy this service in just a few minutes using some of the latest techniques like extracting data from Azure data lakes and accessing Azure blob storage through PolyBase.
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 2)
Im zweiten Teil unserer Microsoft Big Data Session geht es darum, wie Big Data Informationen über "klassisches" SQL zugänglich gemacht werden können und wie sich mit der neuen PolyBase-Engine unstrukturierte Hadoop-Daten mit relationalen Data Warehouse-Daten einfach verknüpfen lassen.
In der Hadoop-Welt wird der SQL-Zugriff über die Komponente Hive ermöglicht.
Über den Microsoft Hive ODBC-Konnektor können die üblichen BI-Tools, wie PowerPivot, diesen Zugriff direkt nutzen.
Die PolyBase-Engine schließlich wird ein Bestandteil des SQL Server 2012 Parallel Data Warehouse werden und erlaubt einem transparenten SQL-Zugriff, egal, wo sich die Daten befinden.
SQL To NoSQL - Top 6 Questions Before Making The Move
Learn what you need to consider when moving from the world of relational databases to a NoSQL document store.
Hear from Developer Advocate Glynn Bird as he explains the key differences between relational databases and JSON document stores like Cloudant, as well as how to dodge the pitfalls of migrating from a relational database to NoSQL.
Qubole - Big data in cloud
Big Data Challenges and How to Overcome Them with Qubole - a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds. Storing, accessing, and analyzing large amounts of data from diverse sources and making it easily accessible to deliver actionable insights for users can be challenging for data driven organizations. The solution for customers is to optimize scaling and create a unified interface to simplify analysis. Qubole helps customers simplify their big data analytics with speed and scalability, while providing data analysts and scientists self-service access in Cloud. The platform is fully elastic and automatically scales or contracts clusters based on workload. We will try to overview main features, advantages and drawback of this platform.
Build Your Own Data Beast : Greenplum + Dell
Looking for a software and hardware stack that can get you results with big data now? Greenplum Database plus Dell hardware will help you construct a data science monster.
Processing Large Datasets for ADAS Applications using Apache Spark
Semantic segmentation is the classification of every pixel in an image/video. The segmentation partitions a digital image into multiple objects to simplify/change the representation of the image into something that is more meaningful and easier to analyze [1][2]. The technique has a wide variety of applications ranging from perception in autonomous driving scenarios to cancer cell segmentation for medical diagnosis.
Exponential growth in the datasets that require such segmentation is driven by improvements in the accuracy and quality of the sensors generating the data extending to 3D point cloud data. This growth is further compounded by exponential advances in cloud technologies enabling the storage and compute available for such applications. The need for semantically segmented datasets is a key requirement to improve the accuracy of inference engines that are built upon them.
Streamlining the accuracy and efficiency of these systems directly affects the value of the business outcome for organizations that are developing such functionalities as a part of their AI strategy.
This presentation details workflows for labeling, preprocessing, modeling, and evaluating performance/accuracy. Scientists and engineers leverage domain-specific features/tools that support the entire workflow from labeling the ground truth, handling data from a wide variety of sources/formats, developing models and finally deploying these models. Users can scale their deployments optimally on GPU-based cloud infrastructure to build accelerated training and inference pipelines while working with big datasets. These environments are optimized for engineers to develop such functionality with ease and then scale against large datasets with Spark-based clusters on the cloud.
Scaling Data and ML with Apache Spark and Feast
Gojek, Indonesia’s first billion-dollar startup, has seen an explosive growth in both users and data over the past three years.
What's hot (20)
Eugene Polonichko "Architecture of modern data warehouse"
Eugene Polonichko "Architecture of modern data warehouse"
Introduction SQL Analytics on Lakehouse Architecture
Introduction SQL Analytics on Lakehouse Architecture
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN
MariaDB 10.2 & MariaDB 10.1 by Michael Monty Widenius at Database Camp 2016 @ UN
Accelerate Your ML Pipeline with AutoML and MLflow
Accelerate Your ML Pipeline with AutoML and MLflow
The Last Mile: Challenges and Opportunities in Data Tools (Strata 2014)
The Last Mile: Challenges and Opportunities in Data Tools (Strata 2014)
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 2)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 2)
SQL To NoSQL - Top 6 Questions Before Making The Move
SQL To NoSQL - Top 6 Questions Before Making The Move
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Similar to 19 reasons to join first international Oracle conference in Poland
22059 slides
The document discusses how IntelliReal built a low-cost virtualized computing grid using a master Oracle database and a partitioned MySQL grid to scale their real estate property data. It describes their initial architecture of partitioning the 400GB of data from Oracle across 800 MySQL databases, which later expanded to over 10,000 partitions. It also outlines the lessons learned around managing their own servers and the networking bandwidth required. Finally, it discusses how this approach helped IntelliReal scale quickly and displace competitors in the market.
MT42 The impact of high performance Oracle workloads on the evolution of the ...
Increased data, along with innovations in application development have led to increasing I/O demands, which are not being met by existing architectures. Find out how high performance applications, particularly analytics applications running on a variety of files systems, are being constrained by storage performance and how Dell EMC's broad portfolio of storage infrastructure can meet their extreme performance demands.
Discover how Dell EMC's revolutionary performance can help you streamline and improve the performance of your entire Oracle environment. Performance and cost comparisons will show you how Dell EMC's performance is not just for extreme workloads but can also help you achieve massive consolidation, more simplified data architectures, increased data agility and reduced management overhead.
"
Daniel Ridder ABAP Core Data Services No Pain, No Gain
Hitchhiker's Guide to ABAP CDS since NetWeaver 7.40
SAP Inside Track 2017, Frankfurt Daniel Ridder @nielseabap
Oracle 12c
1. Oracle Database 12c includes new features such as pluggable databases for consolidation, increased size limits for data types, and inline PL/SQL functions.
2. Other features include improved redaction policies, the ability to migrate table partitions online without disruption, and indexing on single columns.
3. Performance is also enhanced through in-memory options, adaptive query optimization, and temporary undo settings.
Apache Kylin: OLAP Engine on Hadoop - Tech Deep Dive
Kylin is an open source Distributed Analytics Engine from eBay Inc. that provides SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets.
If you want to do multi-dimension analysis on large data sets (billion+ rows) with low query latency (sub-seconds), Kylin is a good option. Kylin also provides seamless integration with existing BI tools (e.g Tableau).
SQLCAT: Tier-1 BI in the World of Big Data
This document summarizes a presentation on tier-1 business intelligence (BI) in the world of big data. The presentation will cover Microsoft's BI capabilities at large scales, big data workloads from Yahoo and investment banks, Hadoop and the MapReduce framework, and extracting data out of big data systems into BI tools. It also shares a case study on Yahoo's advertising analytics platform that processes billions of rows daily from terabytes of data.
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
The effective use of big data is the key to gaining a competitive advantage and outperforming the competition. This change demands that companies consume and blend enormous amount of data created from divergent and inherently mismatched sources, which represents a paradigm shift to the traditional data warehouse.
Companies need to modernize their data warehouse, augmenting it with a platform that allows storage, processing, exploration and analysis of large and diverse datasets without limiting the ability to deliver the data access, and flexibility responding to the needs of the business. That’s where Oracle Cloud and Qubole work together delivering a new breed of data platform —capable of storing and processing the overwhelming amount of data that on-premises big data deployments cannot handle.
Watch this on-demand webinar to understand:
- Why deploying big data on-premises is expensive, complex to maintain and limits your ability to scale across new use cases and data sources
- How Oracle Bare Metal Cloud's predictable and fast performance compute and network services deliver the foundation of a cost-effective, high-performance big data platform
- How Qubole leverages Oracle Bare Metal Cloud to provide a turnkey big data service that optimizes cost, performance, and scale, enabling self-service data exploration.
Qubole delivers a cloud-based, turnkey, self-service big data service that removes the complexity and reduces the cost of doing big data. It leverages Oracle Bare Metal Cloud’s next generation of scalable, inexpensive and performant compute, network and storage public cloud infrastructure to provide a solution that accelerates time to market and reduces the risk of your big data initiatives.
Inside Oracle Exalytics and Oracle TimesTen for Exalytics - Hotsos 2012
Mark Rittman presented on Oracle Exalytics and Oracle TimesTen for Exalytics at the Hotsos Symposium 2012. He discussed (1) what Exalytics is as an in-memory appliance for Oracle Business Intelligence that combines specialized hardware and optimized software, (2) how it addresses performance issues for analytics workloads by caching data and aggregates in memory, and (3) its architecture which includes optimized versions of OBIEE and Essbase running on TimesTen for fast in-memory analytics.
2019 - GUOB Tech Day / Groundbreakers LAD Tour - Database Migration Methods t...
2019 - GUOB Tech Day / Groundbreakers LAD Tour - Database Migration Methods t...Marcus Vinicius Miguel Pedro
Presentation delivered at GUOB Tech Day 2019 (Groundbreakers LAD Tour) about the Database Migration Methods to the Cloud (in English)
Database as a Service on the Oracle Database Appliance Platform
Speaker: Marc Fielding, Co-speaker: Maris Elsins.
Oracle Database Appliance provides a robust, highly-available, cost-effective, and surprisingly scalable platform for database as a service environment. By leveraging Oracle Enterprise Manager's self-service features, databases can be provisioned on a self-service basis to a cluster of Oracle Database Appliance machines. Discover how multiple ODA devices can be managed together to provide both high availability and incremental, cost-effective scalability. Hear real-world lessons learned from successful database consolidation implementations.
Understanding Query Plans and Spark UIs
"The common use cases of Spark SQL include ad hoc analysis, logical warehouse, query federation, and ETL processing. Spark SQL also powers the other Spark libraries, including structured streaming for stream processing, MLlib for machine learning, and GraphFrame for graph-parallel computation. For boosting the speed of your Spark applications, you can perform the optimization efforts on the queries prior employing to the production systems. Spark query plans and Spark UIs provide you insight on the performance of your queries. This talk discloses how to read and tune the query plans for enhanced performance. It will also cover the major related features in the recent and upcoming releases of Apache Spark.
"
SharePoint Connections Conference Amsterdam - Pitfalls and success factors of...
Valkuilen en successen binnen een implementatie van Office 365, SharePoint, Exchange en Lync Online in de Cloud. 4000 seats binnen Zorg - Health. Intramurele en extramurele medewerkers hebben samen een digitale werkomgeving. Het Nieuwe Werken voor een thuiszorginstelling.
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013
Michael Corey, Ntirety, Inc
Jeff Szastak, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Bb performance-engineering-spotlight
Database administrators are responsible for maintaining high performance of databases like Oracle, SQL Server, and Red Hat that support enterprise applications like Blackboard. Quest Software's Spotlight tool helps administrators monitor and diagnose performance issues across various database platforms in real-time and historically. It provides visualizations of process flows, metrics, and alerts to identify bottlenecks and root causes of poor performance. Spotlight supports various database environments including Oracle, Oracle RAC, and SQL Server databases.
Databases in the Cloud - DevDay Austin 2017 Day 2
Databases in the Cloud discusses AWS database services for moving workloads to the cloud. It describes Amazon Relational Database Service (RDS) which provides several fully managed relational database options including MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, and Amazon Aurora. It also discusses non-relational database services like DynamoDB, ElastiCache, and Redshift for analytics workloads. The document provides guidance on choosing between SQL and NoSQL databases and discusses benefits of managed database services over hosting databases on-premises or in EC2 instances.
PASS Summit 2020
Migrating Oracle workloads to Azure requires understanding the workload and hardware requirements. It is important to analyze the workload using the Automatic Workload Repository (AWR) report to accurately size infrastructure needs. The right virtual machine series and storage options must be selected to meet the identified input/output and capacity needs. Rather than moving existing hardware, the focus should be migrating the Oracle workload to take advantage of cloud capabilities while ensuring performance and high availability.
Steps towards business intelligence
“A broad category of applications and technologies for gathering, storing, analyzing, sharing and providing access to data to help enterprise users make better business decisions” -Gartner
Meetup Oracle Database: 3 Analizar, Aconsejar, Automatizar… las nuevas funcio...
Meetup Oracle Database: 3 Analizar, Aconsejar, Automatizar… las nuevas funcio...avanttic Consultoría Tecnológica
Las nuevas arquitecturas, servicios y micro-servicios web, aplicaciones y apps, Bots, IoT, AI, etc., que demandan las organizaciones, necesitan cada vez más del talento y experiencia de los Administradores de Bases de Datos para dar consejos, sugerencias y respuestas que aporten un valor diferencial a los grupos de desarrollo y usuarios de negocio.
Te mostramos las claves del nuevo rol del DBA, que complementa la “A” de Administrar con: Analizar, Aconsejar, Automatizar y crear Arquitecturas eficientes y Autónomas para la gestión Avanzada de datos, colaborando con los desarrolladores y usuarios desde un conocimiento profundo de las base de datos.Geek Sync | Locating and Resolving Common Database Performance Issues in Micr...
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/8wTs50A5bgc
No database professional can afford to avoid performance issues, no matter how big or small. What best practices are you following?
Join IDERA and Devin Gallagher on Thursday, November 2 at 4 PM CT as he walks through locating and resolving common database performance issues. Devin will explore hot objects, poor performing SQL, missing or bad indexes and more. Catch an inside look into DB Optimizer with a technical specialist and learn how it successfully automates SQL tuning and profiling, and locates and resolves many of your common and not-so-common performance issues. Devin will also look at some performance drill down areas within Aqua Data Studio.
Attending PASS Summit? This will be a special live Geek Sync you will not want to miss in Booth 112.
Database 12c is ready for you... Are you ready for 12c?
This presentation is an overview of things all IT management team needs to consider before upgrading to Oracle Database 12c and were presented in a webinar: bit.ly/1yzSdsd
Oracle ACE Director Dan Morgan was a 12c beta tester and is intimately familiar with changes in architecture and how they will impact existing infrastructure and planning and budgeting considerations.
In 2015, most organizations will begin to migrate to the latest version of Oracle Database 12c, but few are aware of the challenges to management, planning and budgeting that need to be addressed before executing such a project. Dan Morgan presents what needs to be considered and shares a few tips about 12c new features.
If you have any questions regarding your future Oracle migrations and upgrades, feel free to visit Performance Tuning Corporation at www.perftuning.com
Similar to 19 reasons to join first international Oracle conference in Poland (20)
MT42 The impact of high performance Oracle workloads on the evolution of the ...
MT42 The impact of high performance Oracle workloads on the evolution of the ...
Daniel Ridder ABAP Core Data Services No Pain, No Gain
Daniel Ridder ABAP Core Data Services No Pain, No Gain
Apache Kylin: OLAP Engine on Hadoop - Tech Deep Dive
Apache Kylin: OLAP Engine on Hadoop - Tech Deep Dive
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Inside Oracle Exalytics and Oracle TimesTen for Exalytics - Hotsos 2012
Inside Oracle Exalytics and Oracle TimesTen for Exalytics - Hotsos 2012
2019 - GUOB Tech Day / Groundbreakers LAD Tour - Database Migration Methods t...
2019 - GUOB Tech Day / Groundbreakers LAD Tour - Database Migration Methods t...
Database as a Service on the Oracle Database Appliance Platform
Database as a Service on the Oracle Database Appliance Platform
SharePoint Connections Conference Amsterdam - Pitfalls and success factors of...
SharePoint Connections Conference Amsterdam - Pitfalls and success factors of...
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right
Meetup Oracle Database: 3 Analizar, Aconsejar, Automatizar… las nuevas funcio...
Meetup Oracle Database: 3 Analizar, Aconsejar, Automatizar… las nuevas funcio...
Geek Sync | Locating and Resolving Common Database Performance Issues in Micr...
Geek Sync | Locating and Resolving Common Database Performance Issues in Micr...
Database 12c is ready for you... Are you ready for 12c?
Database 12c is ready for you... Are you ready for 12c?
Recently uploaded
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Tomaz Bratanic
Graph ML and GenAI Expert - Neo4j
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Main news related to the CCS TSI 2023 (2023/1695)
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
"Choosing proper type of scaling", Olena Syrota
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
June Patch Tuesday
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
Mutation Testing for Task-Oriented Chatbots
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Pitangent Analytics & Technology Solutions Pvt. Ltd
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications."Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Recently uploaded (20)
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
19 reasons to join first international Oracle conference in Poland
- 3. #DBADev, bridging the gap between development and operation Table
- 4. Why has my plan changed.
- 5. DBA, Heal Thyself: Five Diseases of IT Organizations and How to Cure Them Analyze This! Practical Examples of Oracle Analytical Functions
- 7. Living in a Virtualised World: Oracle VM on Exadata
- 8. • •
- 10. Back2Basics - 50 Shades of #Fail: OBI Worst Practices in Real Life
- 11. Living in a Virtualised World: Oracle VM on Exadata
- 12. #DBADev, bridging the gap between development and operation Table
- 13. • Opening the black box called "Cost Based Optimizer" • Query Transformations
- 14. Combining Cloud Based Data with on-prem Exalytics using In-Memory Technologies
- 15. Introducation to graph databases
- 16. Function Madness: how a clever Architecure becomes a problem Function Result Cache - Fixed..
- 17. • The Forgotten Performance Tuning Trick - Clustering your Data • Tips on Bulk Data Processing with SQL and PL/SQL
- 18. • Upgrade Oracle databases to 12cR2: Upgrade paths and RAC and Grid infrastructure upgrade • Leveraging Oracle ASM Cluster File System for Private Cloud storage