BigData Meets the Federal Data Center
•
4 likes•1,757 views
BigData Meets the Federal Data Center - an overview of nosql solutions to data challenges (e.g. Hadoop, Hbase, Mongodb, cassandra, redis etc). Also includes a vignette on Google Prediction API.
Report
Share
Report
Share
![whoami ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Intro to Machine Learning with H2O and AWS
Navdeep Gill @ Galvanize Seattle- May 2016
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The Rise of the DataOps - Dataiku - J On the Beach 2016
Many organisations are creating groups dedicated to data. These groups have many names : Data Team, Data Labs, Analytics Teams….
But whatever the name, the success of those teams depends a lot on the quality of the data infrastructure and their ability to actually deploy data science applications in production.
In that regards a new role of “DataOps” is emerging. Similar, to Dev Ops for (Web) Dev, the Data Ops is a merge between a data engineer and a platform administrator. Well versed in cluster administration and optimisation, a data ops would have also a perspective on the quality of data quality and the relevance of predictive models.
Do you want to be a Data Ops ? We’ll discuss its role and challenges during this talk
IOGDC - McKeel presentation on mashups and OpenEI
A mashup combines data or functionality from two or more sources to create new services. It implies easy and fast integration using open APIs. A brief history of mashups showed examples from 1854 of combining multiple data sources on multiple axes to a 2009 mashup that compiled global server records. Popular mashups today include Gapminder which visualizes 438 data sources. Future mashups may use visual programming languages to allow non-programmers to create DIY mashups. Standard data formats, open licensing, and reusable software are speeding adoption of mashups while intellectual property, security, data quality, and finding data remain challenges holding mashups back.
Aginity Big Data Research Lab V3
This document summarizes the goals and findings of Aginity's "Big Data" research lab. The lab aimed to build a 10 terabyte massively parallel processing always-on data warehouse using $10,000 in commodity hardware and $15,000 per terabyte for software. They were able to construct a 9-node server farm with 10 terabytes of storage for $5,682.10, demonstrating the power and cost-effectiveness of software-only "Big Data" systems. The lab tested various databases and analytics capabilities like complex queries, indexing, and in-database analytics on large datasets.
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
This document discusses big data technologies, jobs, and opportunities in Hong Kong. It defines big data and its key components such as volume, velocity, variety and veracity. It outlines the major technologies involved like servers, storage, databases, visualization tools, and platforms like Hadoop. It discusses why big data is now possible due to cheap storage, abundant computing power, and data accessibility. It also examines career opportunities for data scientists and developers in both Hong Kong and the US and provides advice on getting involved in the big data community in Hong Kong.
Visual Search engine with MXNet Gluon and HNSW
In this talk I introduce a project I worked on, creating a Visual Search engine for 1M Amazon product using MXNet Gluon and the K-Nearest Neighbor search library HNSW.
For implementation details, check this repository: https://github.com/ThomasDelteil/VisualSearch_MXNet
Video available here:
https://www.youtube.com/watch?v=9a8MAtfFVwI
Demo website available here:
https://thomasdelteil.github.io/VisualSearch_MXNet/
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Andrew Thompson gave this talk at the December 2014 meeting of DVRPC's IREG group, which was billed as a special full-day "Open Source Geo Day" event.
Democratize ai with google cloud
Today we’re seeing revolutionary changes in hardware and software that are democratizing machine learning (ML) and making it accessible to any developer or data scientist. Whether you’re new to ML or you’re already an expert, Google Cloud has a variety of tools to help you. Learn the options available and how they support the full machine learning lifecycle for both realtime and batch data.
Recommended
Intro to Machine Learning with H2O and AWS
Navdeep Gill @ Galvanize Seattle- May 2016
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The Rise of the DataOps - Dataiku - J On the Beach 2016
Many organisations are creating groups dedicated to data. These groups have many names : Data Team, Data Labs, Analytics Teams….
But whatever the name, the success of those teams depends a lot on the quality of the data infrastructure and their ability to actually deploy data science applications in production.
In that regards a new role of “DataOps” is emerging. Similar, to Dev Ops for (Web) Dev, the Data Ops is a merge between a data engineer and a platform administrator. Well versed in cluster administration and optimisation, a data ops would have also a perspective on the quality of data quality and the relevance of predictive models.
Do you want to be a Data Ops ? We’ll discuss its role and challenges during this talk
IOGDC - McKeel presentation on mashups and OpenEI
A mashup combines data or functionality from two or more sources to create new services. It implies easy and fast integration using open APIs. A brief history of mashups showed examples from 1854 of combining multiple data sources on multiple axes to a 2009 mashup that compiled global server records. Popular mashups today include Gapminder which visualizes 438 data sources. Future mashups may use visual programming languages to allow non-programmers to create DIY mashups. Standard data formats, open licensing, and reusable software are speeding adoption of mashups while intellectual property, security, data quality, and finding data remain challenges holding mashups back.
Aginity Big Data Research Lab V3
This document summarizes the goals and findings of Aginity's "Big Data" research lab. The lab aimed to build a 10 terabyte massively parallel processing always-on data warehouse using $10,000 in commodity hardware and $15,000 per terabyte for software. They were able to construct a 9-node server farm with 10 terabytes of storage for $5,682.10, demonstrating the power and cost-effectiveness of software-only "Big Data" systems. The lab tested various databases and analytics capabilities like complex queries, indexing, and in-database analytics on large datasets.
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
This document discusses big data technologies, jobs, and opportunities in Hong Kong. It defines big data and its key components such as volume, velocity, variety and veracity. It outlines the major technologies involved like servers, storage, databases, visualization tools, and platforms like Hadoop. It discusses why big data is now possible due to cheap storage, abundant computing power, and data accessibility. It also examines career opportunities for data scientists and developers in both Hong Kong and the US and provides advice on getting involved in the big data community in Hong Kong.
Visual Search engine with MXNet Gluon and HNSW
In this talk I introduce a project I worked on, creating a Visual Search engine for 1M Amazon product using MXNet Gluon and the K-Nearest Neighbor search library HNSW.
For implementation details, check this repository: https://github.com/ThomasDelteil/VisualSearch_MXNet
Video available here:
https://www.youtube.com/watch?v=9a8MAtfFVwI
Demo website available here:
https://thomasdelteil.github.io/VisualSearch_MXNet/
Your New Partners: Understanding Civic Hackathons, Why You Should be Involved...
Andrew Thompson gave this talk at the December 2014 meeting of DVRPC's IREG group, which was billed as a special full-day "Open Source Geo Day" event.
Democratize ai with google cloud
Today we’re seeing revolutionary changes in hardware and software that are democratizing machine learning (ML) and making it accessible to any developer or data scientist. Whether you’re new to ML or you’re already an expert, Google Cloud has a variety of tools to help you. Learn the options available and how they support the full machine learning lifecycle for both realtime and batch data.
Talk in Google fest 2013
This document discusses how BigQuery, Google Spreadsheets, and Google App Script can be combined to create powerful analytic dashboards. Specifically, it shows how BigQuery allows for ad-hoc querying of large datasets, App Script enables connecting Google services and running scripts, and Spreadsheets provides a familiar interface. Together these tools allow for processing big data, performing queries, displaying results and charts in a spreadsheet, and exporting to Excel all within a web-based dashboard. An example dashboard is demonstrated that queries a 68 million row US birth data set from BigQuery and displays results and charts in a Google Spreadsheet using this combined approach.
Data Engineering Efficiency @ Netflix - Strata 2017
Slides from Strata 2017 talk, "Data Engineering Efficiency @ Netflix."
Michelle Ufford explains how Netflix’s data engineering and analytics team is using data to find common patterns among the chaos that enable the company to automate repetitive and time-consuming tasks and discover ways to improve data quality, reduce costs, and quickly identify and respond to issues. Michelle provides a quick overview of Netflix’s analytics environment before diving into some of the major challenges facing the company’s data engineers. Along the way, Michelle shares how Netflix is building more intelligent data platform services and tools to improve data quality, automate data maintenance, alert on job optimization opportunities, and more.
How to get your engineers to care about the AWS Bill
Your engineers need to be able to start their own machines, set up their own services, and be independent in the cloud. That is nice, but that also means relinquishing control of costs to some extent. To do that and not go bankrupt, you need to get your engineers to care about costs. This is how.
Is big data dead?
Hadoop enabled big data but the technology stack has become more complicated over time. Big data continues to drive artificial intelligence development. While big data is not dead, the paradigm is shifting again as technologies like IoT and edge computing generate enormous new data sources. This will require new approaches and platforms to integrate device data, perform analytics at the edge, and deploy trained machine learning models across different environments while ensuring data privacy and security.
Data Visualization for the Web - How to Get Started
Bethesda Data Science Meetup February 2019
Chris Conlan and Paulo Martinez give a brief overview of the software ecosystem for web-based data viz, then dive into their own portfolios (not in slides).
Report: EDA of TV shows & movies available on Netflix
This document describes an exploratory data analysis project analyzing TV shows and movies available on Netflix. The analysis uses a dataset from Kaggle containing 7,788 rows of data about Netflix content. Visualizations were created using Plotly to show trends in movies versus TV shows by year and month. Additional insights include the top countries of origin, most frequent casts, and genres for movies and TV shows. The analysis provides a overview of Netflix content and how it varies by factors like country, genre, and duration.
Top 5 Deep Learning and AI Stories - November 3, 2017
The document discusses insights into deep learning and artificial intelligence. It provides the top 5 headlines: 1) Pentagon official discusses how AI and machine learning will revolutionize the US intelligence community. 2) Startup is working on an AI system to detect lung cancer earlier from chest X-rays to save lives. 3) NVIDIA's GPU Cloud gives developers access to optimized deep learning tools in the cloud. 4) Non-profit AI4ALL partners with NVIDIA to increase students' access to AI resources and careers. 5) NVIDIA expands its Deep Learning Institute to address the growing need for AI experts.
Massive data twitter and semantic analysis in Reador.NET project
The document discusses the Reador.net project, which aims to create a new semantic news aggregating tool. It notes that there is more and more data being produced that needs to be organized. The tool seeks to add meaning to data by representing it semantically and connecting different sources. It currently extracts over 2 million news articles daily from various providers and stores over 20GB of data.
The IoT Transformation and What it Means to You - Nir Dobovizky
IoT is not about controlling random devices from your phone - IoT devices can revolutionize how businesses collect, process, and act upon data. In this talk, we will cover why IoT is as important as the hype says and what it means for your business
Big Data
Understanding Big Data summarizes big data and popular big data technologies. It discusses how big data is generated from various sources and is too large to be processed by traditional databases. Popular technologies like Hadoop, HDFS, MapReduce, Hive, Pig, HBase, and Mahout are able to collect, store, process, and analyze big data. Companies are using big data to gain insights from customer data, optimize operations, prevent fraud, and make recommendations.
How I built a ml human hybrid workflow using computer vision - Amir Shitrit
While not new at all, Machine Learning has been on the rise of the past years, both because of the ubiquity of data and because of the increase in adoption of Cloud Computing. In recent years, however, ML has become more prevalent than ever - mainly due to its ease of use and its accessibility to non-mathematicians.
In some cases, ML can do things that would’ve been extremely difficult, if not impossible, for us to achieve in the past. In other cases, however, ML is here to assist us, rather than replace us, by relieving us of our most boring and repetitive tasks, and this often has to do with the low accuracy in which ML models operate.
In this talk we are going to build business workflows using the joint effort of humans and software to automate those boring tasks, while compensating for the inaccuracy of ML with human intervention.
Big Data LDN 2017: Deep Learning Demystified
Date: 16th November 2017
Location: Keynote Theatre
Time: 15:10 - 15:40
Speaker: Adam Grzywaczewski
Organisation: NVIDIA
Switching From Web Development to Data Science
This DataCamp infographic describes how you can make the switch between using Python for web development and using it for data science.
Do you want to learn Python for Data Science? Consider www.datacamp.com!
Biq query devfest2017_slides
This document contains an agenda for a presentation on BigQuery. The agenda includes sections on why big data is important, getting started with BigQuery including enabling APIs and creating projects, loading sample data and running queries, and integrating BigQuery with other tools through its API. It also provides examples of different types of queries that can be run on BigQuery including simple aggregates, complex processing, nested selects, and joins.
The lessons learned from handling billions of events
The document discusses lessons learned from handling billions of events in a digital transformation journey. It describes how things were initially thought to look with AI handling everything, but in reality more was needed. Five key lessons are outlined: 1) AI is just the peak of the iceberg and more infrastructure is required; 2) both data scientists and data engineers are needed; 3) people, not technology, are the real limit; 4) beware of costs from bleeding-edge technology; and 5) whether to embed data scientists within products or have them in a centralized platform. The presentation concludes by outlining several areas the company is working on, including data quality tools, metrics research, and AI techniques like image classification and topic modeling.
Manuel Martinez - Structured Data @ Search London SEO Meetup
What is structured data? What is schema.org? How do we use it and why should we bother? These are the questioned addressed during my talk at Search London, 19 Feb 2018.
Making Sense of Graph Databases
The document discusses graph databases and their advantages over traditional databases for modeling connected data. It provides an overview of graph databases and what they are used for. Key points include:
- Graph databases simplify and speed up access to connected data by using nodes, edges, and properties to represent relationships. This is challenging for other database types.
- Graph databases are gaining popularity faster than any other database category due to their ability to rapidly access complex networks of connected data.
- Graph databases support use cases involving social networks, recommendations, fraud detection, and more where relationships are important.
- When evaluating graph databases, considerations include performance, scalability, support for real-time access, and lowering the total cost of
Data Engineering and the Data Science Lifecycle
Everyone wants to be a data scientist. Data modeling is the hottest thing since Tickle Me Elmo. But data scientists don’t work alone. They rely on data engineers to help with data acquisition and data shaping before their model can be developed. They rely on data engineers to deploy their model into production. Once the model is in production, the data engineer’s job isn’t done. The model must be monitored to make sure that it retains its predictive power. And when the model slips, the data engineer and the data scientist need to work together to correct it through retraining or remodeling.
Strata Conference NYC 2013
The document provides an overview of presentations and discussions from the Strata Conference + Hadoop World 2013. Key themes included:
- The importance of understanding business needs and asking the right questions of data
- Choosing the right tools for each problem, not just popular ones, from the big data ecosystem
- Moving beyond just managing large volumes of data to delivering actionable insights
- The value of prototyping and experimentation in building support for new techniques
GDG Varna - Hadoop
The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing. Learn the fundamental principles behind it, and how you can use its power to make sense of your Big Data.
Anticipatory Intelligence
This document discusses four enablers of anticipatory intelligence: new trust models, publicly available information ("big data" OSINT), proxy variable selection, and radical quantification. It provides examples of how aggregating and analyzing large amounts of publicly available data through techniques like counting, correlation analysis, and geographic analysis can provide insights and even anticipate future events. Proper quantification and visualization of proxy variables can unlock insights from aggregated human behaviors and trends.
Taming Social Media with MongoDB
This document summarizes using MongoDB to collect and analyze tweets from social media. It describes setting up MongoDB collections to store tweets collected from Twitter's API, querying the tweets by location, user, and other fields, and building an interface with maps and visualization tools to interact with the tweets. Key steps included collecting tweets from Australia, augmenting the data, indexing for efficient querying, finding the most active users, and lessons learned around data structure and API limitations.
More Related Content
What's hot
Talk in Google fest 2013
This document discusses how BigQuery, Google Spreadsheets, and Google App Script can be combined to create powerful analytic dashboards. Specifically, it shows how BigQuery allows for ad-hoc querying of large datasets, App Script enables connecting Google services and running scripts, and Spreadsheets provides a familiar interface. Together these tools allow for processing big data, performing queries, displaying results and charts in a spreadsheet, and exporting to Excel all within a web-based dashboard. An example dashboard is demonstrated that queries a 68 million row US birth data set from BigQuery and displays results and charts in a Google Spreadsheet using this combined approach.
Data Engineering Efficiency @ Netflix - Strata 2017
Slides from Strata 2017 talk, "Data Engineering Efficiency @ Netflix."
Michelle Ufford explains how Netflix’s data engineering and analytics team is using data to find common patterns among the chaos that enable the company to automate repetitive and time-consuming tasks and discover ways to improve data quality, reduce costs, and quickly identify and respond to issues. Michelle provides a quick overview of Netflix’s analytics environment before diving into some of the major challenges facing the company’s data engineers. Along the way, Michelle shares how Netflix is building more intelligent data platform services and tools to improve data quality, automate data maintenance, alert on job optimization opportunities, and more.
How to get your engineers to care about the AWS Bill
Your engineers need to be able to start their own machines, set up their own services, and be independent in the cloud. That is nice, but that also means relinquishing control of costs to some extent. To do that and not go bankrupt, you need to get your engineers to care about costs. This is how.
Is big data dead?
Hadoop enabled big data but the technology stack has become more complicated over time. Big data continues to drive artificial intelligence development. While big data is not dead, the paradigm is shifting again as technologies like IoT and edge computing generate enormous new data sources. This will require new approaches and platforms to integrate device data, perform analytics at the edge, and deploy trained machine learning models across different environments while ensuring data privacy and security.
Data Visualization for the Web - How to Get Started
Bethesda Data Science Meetup February 2019
Chris Conlan and Paulo Martinez give a brief overview of the software ecosystem for web-based data viz, then dive into their own portfolios (not in slides).
Report: EDA of TV shows & movies available on Netflix
This document describes an exploratory data analysis project analyzing TV shows and movies available on Netflix. The analysis uses a dataset from Kaggle containing 7,788 rows of data about Netflix content. Visualizations were created using Plotly to show trends in movies versus TV shows by year and month. Additional insights include the top countries of origin, most frequent casts, and genres for movies and TV shows. The analysis provides a overview of Netflix content and how it varies by factors like country, genre, and duration.
Top 5 Deep Learning and AI Stories - November 3, 2017
The document discusses insights into deep learning and artificial intelligence. It provides the top 5 headlines: 1) Pentagon official discusses how AI and machine learning will revolutionize the US intelligence community. 2) Startup is working on an AI system to detect lung cancer earlier from chest X-rays to save lives. 3) NVIDIA's GPU Cloud gives developers access to optimized deep learning tools in the cloud. 4) Non-profit AI4ALL partners with NVIDIA to increase students' access to AI resources and careers. 5) NVIDIA expands its Deep Learning Institute to address the growing need for AI experts.
Massive data twitter and semantic analysis in Reador.NET project
The document discusses the Reador.net project, which aims to create a new semantic news aggregating tool. It notes that there is more and more data being produced that needs to be organized. The tool seeks to add meaning to data by representing it semantically and connecting different sources. It currently extracts over 2 million news articles daily from various providers and stores over 20GB of data.
The IoT Transformation and What it Means to You - Nir Dobovizky
IoT is not about controlling random devices from your phone - IoT devices can revolutionize how businesses collect, process, and act upon data. In this talk, we will cover why IoT is as important as the hype says and what it means for your business
Big Data
Understanding Big Data summarizes big data and popular big data technologies. It discusses how big data is generated from various sources and is too large to be processed by traditional databases. Popular technologies like Hadoop, HDFS, MapReduce, Hive, Pig, HBase, and Mahout are able to collect, store, process, and analyze big data. Companies are using big data to gain insights from customer data, optimize operations, prevent fraud, and make recommendations.
How I built a ml human hybrid workflow using computer vision - Amir Shitrit
While not new at all, Machine Learning has been on the rise of the past years, both because of the ubiquity of data and because of the increase in adoption of Cloud Computing. In recent years, however, ML has become more prevalent than ever - mainly due to its ease of use and its accessibility to non-mathematicians.
In some cases, ML can do things that would’ve been extremely difficult, if not impossible, for us to achieve in the past. In other cases, however, ML is here to assist us, rather than replace us, by relieving us of our most boring and repetitive tasks, and this often has to do with the low accuracy in which ML models operate.
In this talk we are going to build business workflows using the joint effort of humans and software to automate those boring tasks, while compensating for the inaccuracy of ML with human intervention.
Big Data LDN 2017: Deep Learning Demystified
Date: 16th November 2017
Location: Keynote Theatre
Time: 15:10 - 15:40
Speaker: Adam Grzywaczewski
Organisation: NVIDIA
Switching From Web Development to Data Science
This DataCamp infographic describes how you can make the switch between using Python for web development and using it for data science.
Do you want to learn Python for Data Science? Consider www.datacamp.com!
Biq query devfest2017_slides
This document contains an agenda for a presentation on BigQuery. The agenda includes sections on why big data is important, getting started with BigQuery including enabling APIs and creating projects, loading sample data and running queries, and integrating BigQuery with other tools through its API. It also provides examples of different types of queries that can be run on BigQuery including simple aggregates, complex processing, nested selects, and joins.
The lessons learned from handling billions of events
The document discusses lessons learned from handling billions of events in a digital transformation journey. It describes how things were initially thought to look with AI handling everything, but in reality more was needed. Five key lessons are outlined: 1) AI is just the peak of the iceberg and more infrastructure is required; 2) both data scientists and data engineers are needed; 3) people, not technology, are the real limit; 4) beware of costs from bleeding-edge technology; and 5) whether to embed data scientists within products or have them in a centralized platform. The presentation concludes by outlining several areas the company is working on, including data quality tools, metrics research, and AI techniques like image classification and topic modeling.
Manuel Martinez - Structured Data @ Search London SEO Meetup
What is structured data? What is schema.org? How do we use it and why should we bother? These are the questioned addressed during my talk at Search London, 19 Feb 2018.
Making Sense of Graph Databases
The document discusses graph databases and their advantages over traditional databases for modeling connected data. It provides an overview of graph databases and what they are used for. Key points include:
- Graph databases simplify and speed up access to connected data by using nodes, edges, and properties to represent relationships. This is challenging for other database types.
- Graph databases are gaining popularity faster than any other database category due to their ability to rapidly access complex networks of connected data.
- Graph databases support use cases involving social networks, recommendations, fraud detection, and more where relationships are important.
- When evaluating graph databases, considerations include performance, scalability, support for real-time access, and lowering the total cost of
Data Engineering and the Data Science Lifecycle
Everyone wants to be a data scientist. Data modeling is the hottest thing since Tickle Me Elmo. But data scientists don’t work alone. They rely on data engineers to help with data acquisition and data shaping before their model can be developed. They rely on data engineers to deploy their model into production. Once the model is in production, the data engineer’s job isn’t done. The model must be monitored to make sure that it retains its predictive power. And when the model slips, the data engineer and the data scientist need to work together to correct it through retraining or remodeling.
Strata Conference NYC 2013
The document provides an overview of presentations and discussions from the Strata Conference + Hadoop World 2013. Key themes included:
- The importance of understanding business needs and asking the right questions of data
- Choosing the right tools for each problem, not just popular ones, from the big data ecosystem
- Moving beyond just managing large volumes of data to delivering actionable insights
- The value of prototyping and experimentation in building support for new techniques
GDG Varna - Hadoop
The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing. Learn the fundamental principles behind it, and how you can use its power to make sense of your Big Data.
What's hot (20)
Data Engineering Efficiency @ Netflix - Strata 2017
Data Engineering Efficiency @ Netflix - Strata 2017
How to get your engineers to care about the AWS Bill
How to get your engineers to care about the AWS Bill
Data Visualization for the Web - How to Get Started
Data Visualization for the Web - How to Get Started
Report: EDA of TV shows & movies available on Netflix
Report: EDA of TV shows & movies available on Netflix
Top 5 Deep Learning and AI Stories - November 3, 2017
Top 5 Deep Learning and AI Stories - November 3, 2017
Massive data twitter and semantic analysis in Reador.NET project
Massive data twitter and semantic analysis in Reador.NET project
The IoT Transformation and What it Means to You - Nir Dobovizky
The IoT Transformation and What it Means to You - Nir Dobovizky
How I built a ml human hybrid workflow using computer vision - Amir Shitrit
How I built a ml human hybrid workflow using computer vision - Amir Shitrit
The lessons learned from handling billions of events
The lessons learned from handling billions of events
Manuel Martinez - Structured Data @ Search London SEO Meetup
Manuel Martinez - Structured Data @ Search London SEO Meetup
Viewers also liked
Anticipatory Intelligence
This document discusses four enablers of anticipatory intelligence: new trust models, publicly available information ("big data" OSINT), proxy variable selection, and radical quantification. It provides examples of how aggregating and analyzing large amounts of publicly available data through techniques like counting, correlation analysis, and geographic analysis can provide insights and even anticipate future events. Proper quantification and visualization of proxy variables can unlock insights from aggregated human behaviors and trends.
Taming Social Media with MongoDB
This document summarizes using MongoDB to collect and analyze tweets from social media. It describes setting up MongoDB collections to store tweets collected from Twitter's API, querying the tweets by location, user, and other fields, and building an interface with maps and visualization tools to interact with the tweets. Key steps included collecting tweets from Australia, augmenting the data, indexing for efficient querying, finding the most active users, and lessons learned around data structure and API limitations.
2012 humangeo smn
The document discusses the basics of social media through three core concepts: search, social, and crowd. It defines key terms, explores how people use social media to search for information, connect with others virtually, and collaborate through crowdsourcing. Examples are provided of how these concepts apply to social media platforms and applications like photo sharing sites, Ushahidi, and multichannel intelligence tools.
Heatmaps are the Heat
The document provides an overview of heatmaps and how they can be used to summarize large amounts of spatial data. It discusses how heatmaps emerged as a solution to analyze big data, lists different types of heatmaps that can be created (turnkey vs custom), and provides two recipes for building heatmaps using Python libraries and geo-tagged Twitter data. The document is presented as a menu by an organization called HumanGeo that provides spatial data analysis services.
Crowds, Computers, and Coordinates
The document discusses how crowdsourcing, computing power, and geolocation data will change how information is assessed by 2050. It notes that the global population and number of internet users will continue growing significantly in the coming decades. At the same time, computing power and data storage will vastly increase, and billions of internet-connected devices will generate continuous streams of geolocation data. This will enable new models for assessing information that leverage crowds, computers, and coordinates to provide persistent situational awareness, improved algorithms for determining truth, and personal location advisors.
Taming Social Media
This document discusses building scalable social media analytics tools. It introduces work with HumanGeo to develop visualization and analysis tools for social media data. It describes the wealth of information available on social media platforms and their widespread adoption. It then contrasts vertical and horizontal scaling, explaining advantages of each approach. The document outlines collecting tweets from Australia and storing them in MongoDB using Python scripts. It also presents a visualization tool built with Bottle, MongoDB, jQuery, Leaflet, and Dust that demonstrates analyzing geotagged tweets.
Big Data and the Social Sciences
Big Data and the Social Sciences
Ex-Google engineer Abe Usher presents a talk about Big Data technology and methods applicable to social science.
Participants will learn techniques that are used by Google engineers to collect, clean, analyze, and visualize Big Data.
Additionally Mr. Usher will provide URLs to sample data, open source applications, and code to those interested in applying these Big Data methods themselves.
Advanced Web-Based Geospatial Visualization using Leaflet
In Intel Analytic DC's first meetup, Danny Holloway presented a tool that allows users to find and map the latest tweets in Australia using Leaflet, TileMill, MongoDB, and other technologies. Given the audience interest in geospatial technologies/analytics and web-based mapping, this presentation introduces and provides examples using the HumanGeo Data Visualization Framework, a soon to be released open source JavaScript framework based on CloudMade's Leaflet web-mapping framework. The goal of the Data Visualization Framework is to provide a core set of capabilities for visualizing data using Leaflet while simplifying common tasks and reducing the amount of code that developers need to write in order to create compelling geospatial visualizations.
NoSQL Type, Bigdata, and Analytics
This document discusses NoSQL databases and how they relate to big data. It provides examples of column-oriented NoSQL databases like Cassandra, document-oriented databases like MongoDB, and key-value stores like Dynamo. It also briefly summarizes characteristics of different database categories and how big data problems can be differentiated based on the five V's: volume, velocity, variety, value and variability.
The Future Of Big Data
A high level overview of common Cassandra use cases, adoption reasons, BigData trends, DataStax Enterprise and the future of BigData given at the 7th Advanced Computing Conference in Seoul, South Korea
Daily Newsletter: 10th March, 2011
The document summarizes Indian and global market performance for March 9, 2011. Domestically, the key indices ended flat amid choppiness, with the BSE Sensex gaining 30 points to close at 18,470 and NSE Nifty gaining 10 points to close at 5,531. Globally, US stocks fell slightly on concerns over violence in Libya. In corporate news, Reliance Communications signed a loan agreement with China Development Bank and Ashok Leyland expanded dealerships in Punjab and Himachal Pradesh.
Video Marketing Project for ESL students
This document outlines an assignment for students to create a 3-5 minute marketing video for an English language program. Students will be placed into groups and choose a topic related to studying at the school to feature in their video. The video must include titles, at least one video clip, and a scripted voiceover. Students will work on storyboarding, collecting materials, and editing their video over the course of several class periods. The final video is due November 2nd.
Как UX-специалист делился своими инструментами с agile-командами
Прошло 2 года. Семен повзрослел и возмужал (в профессиональном и жизненном плане). За это время он успел поработать с несколькими agile-командами и насмотреться разного скрама и срама, набить очередных шишек при внедрении процесса проектирования в гибкие процессы разработки. Но во всех случаях он видел, что некоторые инструменты проектировщика могут пригодиться и другим участникам процесса, командам, которые не имеют проектировщиков интерфейса у себя в штате. Ведь эти инструменты просты в понимании и не требуют много времени на проработку.
И Семен решил попробовать Сначала на своей команде, а потом и на кошках, т.е. на знакомых командах.
Каких тем коснулся Семён, куда и какие инструменты UX-специалиста он попытался внедрить:
- как ещё (кроме привычных инструментов) можно собирать и фиксировать требования касаемо планируемых фич;
- как можно проапгрейдить user story в сторону большей эмпатии пользователям и какие инструменты в этом могут помочь;
- как можно с большей эффективностью разбивать крупные user story на более мелкие (опять же, с большей эмпатией);
- как фиксировать общий опыт взаимодействия пользователя, чтобы в следующей итерации не наломать дров при реализации новых фич. Ведь всегда сложно держать в голове всю картину взаимодействия человека с продуктом. А когда ты добавляешь все новые и новые фичи, часто вместо помощи вставляются палки в колёса;
как можно использовать любимый многими impact map для проработки целей пользователя;
- как можно проверить необходимость фич (а точнее, ожидаемую удовлетворенность от наличия/отсутствия) перед тем, как их поместить в бэклог.
3 d pie chart circular puzzle with hole in center process 9 stages style 2 po...
The document describes a 9-stage 3D circular puzzle process. It includes 9 text boxes arranged in a circular formation labeled TEXT 1 through TEXT 9. Below each text box are instructions to add text and download an awesome diagram. The overall purpose is to create a circular puzzle slide in PowerPoint with editable images and text boxes.
Responsive design and Drupal, case Costume.fi
This document summarizes the design and development process of the Costume.fi mobile-first digital fashion magazine brand. It describes the concept of creating a democratic platform where young women can participate in creating and sharing content. It outlines the mobile-first responsive design approach using Drupal themes and modules to achieve flexibility across devices. Some issues encountered included banner images not scaling properly and compromising functionality for cost reasons. The results were successful, exceeding the launch goals of 50,000-60,000 weekly unique visitors by reaching over 63,000 in the first few months.
La potestad tributaria
El documento define la potestad tributaria como la facultad del Estado de crear tributos de forma unilateral y exigir su pago a las personas sometidas a su jurisdicción. Explica que la potestad tributaria es abstracta, permanente, irrenunciable e indelegrable. Además, señala que la potestad tributaria de los estados y municipios tiene límites establecidos en la Constitución y que éstos pueden determinar tributos de forma autónoma dentro de sus competencias. Finalmente, provee la definición de poder tributario como la facultad
Google App Engine
The document discusses the benefits of meditation for reducing stress and anxiety. Regular meditation practice can help calm the mind and body by lowering heart rate and blood pressure. Studies have shown that meditating for just 10-20 minutes per day can have significant positive impacts on both mental and physical health over time.
007 model
The 007 model helps you track and measure value from strategic to backend services. This walkthrough gives you the basics of the model to get started.
Viewers also liked (20)
Advanced Web-Based Geospatial Visualization using Leaflet
Advanced Web-Based Geospatial Visualization using Leaflet
Как UX-специалист делился своими инструментами с agile-командами
Как UX-специалист делился своими инструментами с agile-командами
3 d pie chart circular puzzle with hole in center process 9 stages style 2 po...
3 d pie chart circular puzzle with hole in center process 9 stages style 2 po...
Similar to BigData Meets the Federal Data Center
How Cloud is Affecting Data Scientists
This document discusses how data science models have transitioned to the cloud to take advantage of greater computing resources. It notes that data science models are resource-intensive and traditionally required powerful local machines. The cloud allows data scientists to run models on cloud infrastructure for lower costs than high-end laptops and with access to many GPUs. Several major cloud platforms - Azure, AWS, and Google Cloud - are discussed and compared in terms of their machine learning offerings. The document also introduces Microsoft's Team Data Science Process, which aims to help data science teams collaborate more effectively on projects in the cloud.
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
This document discusses Google Cloud Platform and its data and analytics capabilities. It begins by explaining the evolution of cloud computing models from virtualized data centers to true on-demand cloud services. It then highlights some of Google Cloud Platform's key differentiators like true cloud economics, future-proof infrastructure, access to innovation, and Google-grade security. The document provides overviews of Google Cloud Platform's storage, database, big data, and machine learning offerings and common use cases for each. It also showcases some of Google's innovations in data analytics and machine learning technologies.
Hot Technologies of 2013: Investigative Analytics
Dr. Robin Bloor and Philip Howard with Infobright
Live Webcast on May 29, 2013
Getting to the bottom of serious situations quickly can separate success from failure in the information economy. Whether you're dealing with customer attrition or dropped phone calls, lost sales or failed machinery, the ability to perform effective root cause analysis can offer tremendous value, especially within critical time windows. This is the domain of investigative analytics – using insights gleaned from complex data sets to identify behavioral patterns, then building predictive models that send the business on a better course.
Register for this episode of Hot Technologies to hear veteran Analysts Dr. Robin Bloor of The Bloor Group and Philip Howard of Bloor Research, as they articulate their vision of what you need to utilize investigative analytics. They'll be briefed by Don DeLoach, who will discuss the Infobright solution's ability to analyze large amounts of data quickly and flexibly, thus enabling the kind of root cause analysis that can solve business issues as they arise. Infobright will focus on how their technology is designed to help businesses harness and gain insight from their machine-generated data, which is increasingly generated by instrumentation, aka the Internet of Things.
How Google Does Big Data - DevNexus 2014
This document provides an overview and agenda for a presentation on how Google handles big data. The presentation covers Google Cloud Platform and how it can be used to run Hadoop clusters on Google Compute Engine and leverage BigQuery for analytics. It also discusses how Google processes big data internally using technologies like MapReduce, BigTable and Dremel and how these concepts apply to customer use cases.
HadoopWorkshopJuly2014
This document provides an overview of a Hadoop session that will cover:
1. An introduction to big data including the history and evolution of Hadoop and how it addresses challenges with traditional databases.
2. The Hadoop architecture and ecosystem including components like HDFS, MapReduce, HBase and how they address issues with scalability, flexibility and cost compared to traditional databases.
3. Hands-on analysis of a soccer dataset using Hadoop to perform tasks like data classification, prediction and player analysis.
Big Data LDN 2018: HOW AUTOMATION CAN ACCELERATE THE DELIVERY OF MACHINE LEAR...
Date: 14th November 2018
Location: AI Lab Theatre
Time: 14:30 - 15:00
Speaker: Andre Balleyguier
Organisation: DataRobot
Structured Data & Schema.org - SMX Milan 2014
My presentation from SMX Milan 2014 covering the history of structured data, usage statistics, schema.org implementation guidelines as well as tools.
Taming Big Data With Modern Software Architecture
The document discusses how modern software architectures can help tame big data. It introduces the speakers and provides an overview of WidasConcepts. The agenda includes a discussion of how big data can help businesses, an example of big data applied in the CarbookPlus platform, and new software architectures for big data. Real-time systems and architectures like lambda architecture are presented as ways to process big data at high velocity and volume. The conclusion emphasizes that big data improves business efficiency but requires tailored implementations and new skills.
Cloud as a Data Platform
My slides on how to use cloud as a data platform at BigDataWeek 2013 Romania
http://www.eurocloud.ro/en/events/all-there-is-to-know-about-big-data/#.UXZFaUDvlVI
RightScale Roadtrip Boston: Accelerate to Cloud
The Accelerate to Cloud keynote will help you understand the current state of cloud adoption, identify the business value for your organization, and provide you a framework to plot your course to cloud adoption.
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...Google Cloud Platform - Japan
2015 年 4 月 9 日（木）に開催された Google Developers Summit Tokyo 2015 から、Developer Advocate の 佐藤 一憲による Google Cloud Platform セッション。What is the future of data strategy?
Watch here: https://bit.ly/3i2iJbu
You will often hear that "data is the new gold". In this context, data management is one of the areas that has received more attention by the software community in recent years. From Artificial Intelligence and Machine Learning to new ways to store and process data, the landscape for data management is in constant evolution. From the privileged perspective of an enterprise middleware platform, we at Denodo have the advantage of seeing many of these changes happen.
Join us for an exciting session that will cover:
- The most interesting trends in data management.
- Our predictions on how those trends will change the data management world.
- How these trends are shaping the future of data virtualization and our own software.
IoT NY - Google Cloud Services for IoT
This document provides an overview of Google Cloud Platform services for IoT and big data analytics, including fully managed ingestion, processing, and analysis of IoT data. It introduces Google Cloud Pub/Sub for messaging, Cloud Dataflow for stream and batch data processing, and BigQuery for petabyte-scale data warehousing and analysis. The presentation includes demos of building an event streaming pipeline using these services to ingest data from Pub/Sub, process it in Dataflow, and analyze results in BigQuery.
Customer Presentation - IBM Cloud Pak for Data Overview (Level 100).PPTX
This document provides instructions for using a presentation deck on Cloud Pak for Data. It instructs the user to:
1. Delete the first slide before using the deck.
2. Customize the presentation for the intended audience as the deck covers various topics and using all slides may not fit a single meeting.
3. The deck contains 6 embedded video records for a demo that takes 15-25 minutes to present. Guidance on pitching the demo is available.
The appendix contains slides on Cloud Pak for Data licensing and IBM's overall strategy.
Course 8 : How to start your big data project by Eric Rodriguez
For more info about our Big Data courses, check out our website ➡️ https://www.betacowork.com/big-data/
---------
"Data is the new oil" - Many companies and professionals do not know how to use their data or are not aware of the added value they could gain from it.
It is in response to these problems that the project “Brussels: The Beating Heart of Big Data” was born.
This project, financed by the Region of Brussels Capital and organised by Betacowork, offers 3 training cycles of 10 courses on big data, at both beginner and advanced levels. These 3 cycles will be followed by a Hackathon weekend.
No prerequisites are required to start these courses. The aim of these courses is to familiarize participants with the principles of Big Data.
------
For more info about our Big Data courses, check out our website ➡️ https://www.betacowork.com/big-data/
Improve your Tech Quotient
This document discusses various topics related to website development and optimization. It covers front-end performance techniques like using content delivery networks and gzipping components. It also discusses tools for front-end performance analysis. Other topics covered include tag management systems, version control systems like Git and SVN, responsive vs adaptive design, and content management systems. The document provides information on technologies and best practices for building high performing websites.
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
Title
DataOps, the secret weapon for delivering AI, data science, and business intelligence value at speed.
Synopsis
● According to recent research, just 7.3% of organisations say the state of their data and analytics is excellent, and only 22% of companies are currently seeing a significant return from data science expenditure.
● Poor returns on data & analytics investment are often the result of applying 20th-century thinking to 21st-century challenges and opportunities.
● Modern data science and analytics require secure, efficient processes to turn raw data from multiple sources and in numerous formats into useful inputs to a data product.
● Developing, orchestrating and iterating modern data pipelines is an extremely complex process requiring multiple technologies and skills.
● Other domains have to successfully overcome the challenge of delivering high-quality products at speed in complex environments. DataOps applies proven agile principles, lean thinking and DevOps practices to the development of data products.
● A DataOps approach aligns data producers, analytical data consumers, processes and technology with the rest of the organisation and its goals.
Building your data driven business with Reactive Marketing Technology
The document discusses data-driven business and reactive marketing technology. It begins with key questions about data-driven business, the benefits of analytics, and introduces the "9D" model for big data business. Tools for building reactive marketing technology are presented, including Apache Storm, Apache Kafka, Apache Spark, and the Hadoop ecosystem. A case study demonstrates how to build a digital marketing software using open source big data tools. The philosophy and a lightweight lambda architecture for building a reactive system is described.
DataOps - The Foundation for Your Agile Data Architecture
Achieving agility in data and analytics is hard. It’s no secret that most data organizations struggle to deliver the on-demand data products that their business customers demand. Recently, there has been much hype around new design patterns that promise to deliver this much sought-after agility.
In this webinar, Chris Bergh, CEO and Head Chef of DataKitchen will cut through the noise and describe several elegant and effective data architecture design patterns that deliver low errors, rapid development, and high levels of collaboration. He’ll cover:
• DataOps, Data Mesh, Functional Design, and Hub & Spoke design patterns;
• Where Data Fabric fits into your architecture;
• How different patterns can work together to maximize agility; and
• How a DataOps platform serves as the foundational superstructure for your agile architecture.
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Building a data lake involves more than installing Hadoop or putting data into AWS. The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This tutorial covers design assumptions, design principles, and how to approach the architecture and planning for multi-use data infrastructure in IT.
Long:
The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This session will discuss hidden design assumptions, review design principles to apply when building multi-use data infrastructure, and provide a reference architecture to use as you work to unify your analytics infrastructure.
The focus in our market has been on acquiring technology, and that ignores the more important part: the larger IT landscape within which this technology lives and the data architecture that lies at its core. If one expects longevity from a platform then it should be a designed rather than accidental architecture.
Architecture is more than just software. It starts from use and includes the data, technology, methods of building and maintaining, and organization of people. What are the design principles that lead to good design and a functional data architecture? What are the assumptions that limit older approaches? How can one integrate with, migrate from or modernize an existing data environment? How will this affect an organization's data management practices? This tutorial will help you answer these questions.
Topics covered:
* A brief history of data infrastructure and past design assumptions
* Categories of data and data use in organizations
* Data architecture
* Functional architecture
* Technology planning assumptions and guidance
Similar to BigData Meets the Federal Data Center (20)
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
Google Cloud Platform & rockPlace Big Data Event-Mar.31.2016
Big Data LDN 2018: HOW AUTOMATION CAN ACCELERATE THE DELIVERY OF MACHINE LEAR...
Big Data LDN 2018: HOW AUTOMATION CAN ACCELERATE THE DELIVERY OF MACHINE LEAR...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Googl...
Customer Presentation - IBM Cloud Pak for Data Overview (Level 100).PPTX
Customer Presentation - IBM Cloud Pak for Data Overview (Level 100).PPTX
Course 8 : How to start your big data project by Eric Rodriguez
Course 8 : How to start your big data project by Eric Rodriguez
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
DataOps - Big Data and AI World London - March 2020 - Harvinder Atwal
Building your data driven business with Reactive Marketing Technology
Building your data driven business with Reactive Marketing Technology
DataOps - The Foundation for Your Agile Data Architecture
DataOps - The Foundation for Your Agile Data Architecture
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Architecting a Data Platform For Enterprise Use (Strata NY 2018)
Recently uploaded
June Patch Tuesday
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Astute Business Solutions | Oracle Cloud Partner |
Your goto partner for Oracle Cloud, PeopleSoft, E-Business Suite, and Ellucian Banner. We are a firm specialized in managed services and consulting.
Essentials of Automations: Exploring Attributes & Automation Parameters
Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they?
Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality.
You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Northern Engraving | Nameplate Manufacturing Process - 2024
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Leveraging the Graph for Clinical Trials and Standards
Katja Glaß
OpenStudyBuilder Community Manager - Katja Glaß Consulting
Marius Conjeaud
Principal Consultant - Neo4j
Choosing The Best AWS Service For Your Website + API.pptx
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Taking AI to the Next Level in Manufacturing.pdf
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
Main news related to the CCS TSI 2023 (2023/1695)
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
GNSS spoofing via SDR (Criptored Talks 2024)
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Principle of conventional tomography-Bibash Shahi ppt..pptx
before the computed tomography, it had been widely used.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Recently uploaded (20)
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
BigData Meets the Federal Data Center
- 1. BigData Meets the Federal Data Center:Practical Solutions for Wicked Problems Abe Usher, CCHP, CISSP – Chief Technology Officer abe.usher@thehumangeo.com
- 4. Former officer USA, USAF
- 5. Cloudera Certified Hadoop ProfessionalFor the past three years, I’ve been focused on BigData analytics for special operations
- 7. Challenges
- 9. Solutions
- 10. Action plan for decision makers
- 11. Homework for data engineersFun stuff
- 12. What is Cloud? Cloud Computing defined: “Delivery of computing as a service”
- 13. Big Trends Michael Driscoll http://radar.oreilly.com/2011/08/building-data-startups.html
- 14. More Trends BigData is now cool (not just geeky) There is an explosion of open source technology for BigData Available cloud technologies are significantly changing our society
- 15. Common Challenges Federal organizations face declining IT budgets Legacy systems not engineered for BigData Creating value from data is hard
- 17. Where are we? Where do we want to go? Data processing “State of the Art” A Better (elusive) Future “We have great intentions, but It is a big mess.” “Don’t look behind the curtain.” “The right tool for the right problem.” “Outsource/eliminate things outside of core competencies.”
- 18. Pattern 1: Outsource Infrastructure & Apps “The Enterprise” “The Cloud” Just-in-time Servers Email & Calendar Travel Coordination
- 19. Pattern 2: Consolidate Data and Analyze It “Future Enterprise” “The Enterprise Today” Redis MongoDB 1. Incrementally adopt BigData tools as you evolve your Enterprise 2. Maintain parallel capabilities if necessary Hadoop
- 20. Vignette 1 Tame massive streaming data in 5 minutes or less.
- 22. cURL utility
- 23. MongoDB
- 26. Recipe1: MongoDB tames Twitter Why MongoDB: Incredibly easy to setup Fast data inserts (> 20,000 per second or 1,728,000,000 per day) Horizontal scaling as data grows Pluggable compression with Snappy http://bit.ly/ggIWWN Get the code! http://bit.ly/humangeo_twitterpipe
- 27. Vignette 2 Ask Google to solve your problems
- 30. cURL utility
- 31. Raw data in multiple languages
- 33. Recipe2: Results
- 35. Spam detection
- 36. Recommendation system (e.g. Netflix, Amazon)
- 38. Document / email classification
- 41. Predict driver behavior and optimize vehicle control systems**Between the Google Prediction API and our own research, we are discovering ways to make information work for the driver and help deliver optimal vehicle performance. –Ryan McGee, Technical Expert, Ford Research and Innovation *http://code.google.com/apis/predict/ ** http://www.google.com/enterprise/cloud/index.html
- 42. Action Plan for Decision Makers Experiment with Cloud-sourcing: http://bit.ly/cuRUCr Inventory your data and your systems Join a Meetup to get informed http://bit.ly/neNCRq Take a risk*
- 43. Homework for Data Engineers Understand Google MapReduce: http://bit.ly/GZBw Experiment with NoSQL: http://bit.ly/VCpR5 Ask Google to Predict the future: http://bit.ly/dCiOoc Take the cloud for a test drive: http://bit.ly/9c9IYy Try something, fail fast
- 44. On-line Twitter: @abeusher E-mail: abe.usher@thehumangeo.com Web: http://thehumangeo.com Facebook: http://on.fb.me/nZS87d This presentation: http://bit.ly/humangeo_cloud2011
- 45. BACKUP
- 47. Monitoring trends and opportunities
- 48. Augmenting and enriching data with influence and sentiment indicatorsTier One special operations intelligence and technology experts with Google experience and agility