Big data tutorial_part4
•
0 likes•279 views
This document discusses big data and techniques for working with large datasets. It introduces key concepts like what big data is, why it is important, and when big data problems arise. It then outlines some common techniques, tools, and applications for big data, including MapReduce, NoSQL databases, and analytics platforms. Specific examples are provided of using big data for personalized recommendations, network monitoring, and analyzing social networks.
Report
Share
Report
Share
Download to read offline
Recommended
Mining Big Data in Real Time
Big Data is a new term used to identify datasets that we can not manage with current methodologies or data mining software tools due to their large size and complexity. Big Data mining is the capability of extracting useful information from these large datasets or streams of data. New mining techniques are necessary due to the volume, variability, and velocity, of such data.
Data mining on big data
This document discusses data mining on big data. It begins with an introduction to big data and data mining. It then discusses the characteristics of big data, known as the HASE theorem, which are that big data is huge in volume, heterogeneous and diverse, from autonomous sources, and has complex and evolving relationships. It presents a conceptual framework for big data processing with three tiers: tier I focuses on low-level data access and computing using techniques like MapReduce; tier II concentrates on semantics, knowledge and privacy; and tier III addresses data mining algorithms. The document concludes that high performance computing platforms are needed to fully leverage big data.
Data mining with big data
This document discusses data mining with big data. It defines data mining as the process of discovering patterns in large data sets and big data as collections of data that are too large to process using traditional software tools. The document notes that 2.5 quintillion bytes of data are created daily and that 90% of data was produced in the past two years. It provides examples of big data like presidential debates and photos. It also discusses challenges of mining big data due to its huge volume and complex, evolving relationships between data points.
Big data mining
This document discusses big data mining. It defines big data as large volumes of structured and unstructured data that are difficult to process using traditional methods due to their size. It describes the characteristics of big data including volume, variety, velocity, variability, and complexity. It also discusses challenges of big data such as data location, volume, hardware resources, and privacy. Popular tools for big data mining include Hadoop, Apache S4, Storm, Apache Mahout, and MOA. Hadoop is an open source software framework that allows distributed processing of large datasets across clusters of computers. Common algorithms for big data mining operate at the model and knowledge levels to discover patterns and correlations across distributed data sources.
Bigdatacooltools
The document discusses tools for analyzing dark data and dark matter, including DeepDive and Apache Spark. DeepDive is highlighted as a system that helps extract value from dark data by creating structured data from unstructured sources and integrating it into existing databases. It allows for sophisticated relationships and inferences about entities. Apache Spark is also summarized as providing high-level abstractions for stream processing, graph analytics, and machine learning on big data.
Big Data - Big Insights - Waze @Google
Waze @Google is a Big Data company.
We use data and complex analytics to gain insights and make decisions on a daily basis.
This presentations includes teasers and ideas for you based on real use cases from Waze
Data minig with Big data analysis
This document discusses data mining with big data. It begins with an agenda that covers problem definition, objectives, literature review, algorithms, existing systems, advantages, disadvantages, big data characteristics, challenges, tools, and applications. It then goes on to define the problem, objectives, provide a literature review summarizing several papers, and describe the architecture, algorithms, existing systems, HACE theorem that models big data characteristics, advantages of the proposed system, challenges, and characteristics of big data. It concludes that formalizing big data analysis processes will be important as data volumes continue increasing.
How to build and run a big data platform in the 21st century
The document provides an overview of big data platform architectures that have been built by various companies and organizations. It discusses self-built platforms from companies like Airbnb, Netflix, Facebook, Slack, and Uber. It also covers cloud-built platforms on IBM Cloud, Microsoft Azure, Google Cloud, and Amazon AWS. Consulting-built platforms from Cloudera and ThoughtWorks are presented. Finally, it introduces the NIST Big Data Reference Architecture as a standard reference model and discusses generic batch vs streaming architectures like Lambda and Kappa.
Recommended
Mining Big Data in Real Time
Big Data is a new term used to identify datasets that we can not manage with current methodologies or data mining software tools due to their large size and complexity. Big Data mining is the capability of extracting useful information from these large datasets or streams of data. New mining techniques are necessary due to the volume, variability, and velocity, of such data.
Data mining on big data
This document discusses data mining on big data. It begins with an introduction to big data and data mining. It then discusses the characteristics of big data, known as the HASE theorem, which are that big data is huge in volume, heterogeneous and diverse, from autonomous sources, and has complex and evolving relationships. It presents a conceptual framework for big data processing with three tiers: tier I focuses on low-level data access and computing using techniques like MapReduce; tier II concentrates on semantics, knowledge and privacy; and tier III addresses data mining algorithms. The document concludes that high performance computing platforms are needed to fully leverage big data.
Data mining with big data
This document discusses data mining with big data. It defines data mining as the process of discovering patterns in large data sets and big data as collections of data that are too large to process using traditional software tools. The document notes that 2.5 quintillion bytes of data are created daily and that 90% of data was produced in the past two years. It provides examples of big data like presidential debates and photos. It also discusses challenges of mining big data due to its huge volume and complex, evolving relationships between data points.
Big data mining
This document discusses big data mining. It defines big data as large volumes of structured and unstructured data that are difficult to process using traditional methods due to their size. It describes the characteristics of big data including volume, variety, velocity, variability, and complexity. It also discusses challenges of big data such as data location, volume, hardware resources, and privacy. Popular tools for big data mining include Hadoop, Apache S4, Storm, Apache Mahout, and MOA. Hadoop is an open source software framework that allows distributed processing of large datasets across clusters of computers. Common algorithms for big data mining operate at the model and knowledge levels to discover patterns and correlations across distributed data sources.
Bigdatacooltools
The document discusses tools for analyzing dark data and dark matter, including DeepDive and Apache Spark. DeepDive is highlighted as a system that helps extract value from dark data by creating structured data from unstructured sources and integrating it into existing databases. It allows for sophisticated relationships and inferences about entities. Apache Spark is also summarized as providing high-level abstractions for stream processing, graph analytics, and machine learning on big data.
Big Data - Big Insights - Waze @Google
Waze @Google is a Big Data company.
We use data and complex analytics to gain insights and make decisions on a daily basis.
This presentations includes teasers and ideas for you based on real use cases from Waze
Data minig with Big data analysis
This document discusses data mining with big data. It begins with an agenda that covers problem definition, objectives, literature review, algorithms, existing systems, advantages, disadvantages, big data characteristics, challenges, tools, and applications. It then goes on to define the problem, objectives, provide a literature review summarizing several papers, and describe the architecture, algorithms, existing systems, HACE theorem that models big data characteristics, advantages of the proposed system, challenges, and characteristics of big data. It concludes that formalizing big data analysis processes will be important as data volumes continue increasing.
How to build and run a big data platform in the 21st century
The document provides an overview of big data platform architectures that have been built by various companies and organizations. It discusses self-built platforms from companies like Airbnb, Netflix, Facebook, Slack, and Uber. It also covers cloud-built platforms on IBM Cloud, Microsoft Azure, Google Cloud, and Amazon AWS. Consulting-built platforms from Cloudera and ThoughtWorks are presented. Finally, it introduces the NIST Big Data Reference Architecture as a standard reference model and discusses generic batch vs streaming architectures like Lambda and Kappa.
Spark Social Media
This document discusses leveraging social big data and the evolution from existing rigid operations to predictive analytics using social media. It begins with an overview of handouts and reference materials on big data, Hadoop, Spark, and data science projects. It then discusses areas for conversation around social content, structure and analytics, data science primers and resources, and data science innovation. It presents a roadmap showing the evolution from rigid and siloed operations to being more flexible, connected, adaptive and predictive using social media. Finally, it discusses types of intentionality and how social CRM can integrate social data.
Datapreneurs
This document discusses several topics related to data and data-driven businesses. It begins by outlining trends in big data and machine learning. It then discusses how to build data-centric businesses by identifying data opportunities and sources, understanding the data lifecycle, and extracting value from data. Examples are provided of Netflix as a data-driven company. The future of professions in a data-driven world is also examined, as well as talent scarcity issues and the need for data-savvy managers. The document provides an overview of many relevant topics at the intersection of data and business.
Big Data
Big data refers to large volumes of high velocity, variety and veracity information that require advanced methods to enable analysis and insights. It is data produced from many sources, including social media interactions, website activity, photos, videos and more. Companies and governments collect and analyze big data to improve services, facilitate operations, seek new business models and boost effectiveness. However, big data also raises privacy and surveillance concerns when collected by governments. The future will see big data continue growing rapidly in size and analytical capabilities, while also facing increasing debates around security, privacy and the environmental impacts of data processing.
Systemof insight
The document discusses data science innovation and the future of professions in light of new technologies. It describes how accounting work may be automated or replaced by computer-assisted techniques and predictive analytics software. This would allow accountants to shift from reactive to proactive work by leveraging accounting data and insights to predict client scenarios and advise clients. Key areas discussed include systems of insight using big data, machine engineering to create applications from insights, and the role of data science.
Geospatial Intelligence Middle East 2013_Big Data_Steven Ramage
Some initial considerations and discussion points around geospatial big data. Location adds context and relevance. Need to consider a number of V factors including Value.
A brief history of "big data"
The history of Big Data from the beginning to Spark. An introduction to the big data software tools for developers and data analysts.
The Web3 Data Economy: Ocean Protocol
1. The document discusses the need for a "Web3 Data Economy" using blockchain and tokens to open up data access and incentivize data sharing, similar to how cryptocurrencies have opened up and incentivized money transfers.
2. It proposes that Ocean Protocol can serve as the foundational platform and token for this new data economy, fulfilling similar roles that Ethereum plays in the cryptocurrency economy like being the unit of exchange and hosting applications.
3. The goal is to connect data owners with AI companies and researchers to provide more open access to data and tools while protecting privacy and properly incentivizing all parties, helping to shift power away from current large data platforms.
Big data
Nandan Shah's presentation covered key topics related to big data including definitions, important technologies like Hadoop and MapReduce, benefits such as enabling real-time analysis and faster decision making, risks around data security and privacy, and applications across various domains such as banking, retail, and healthcare. The presentation also provided interesting facts about the size of data holdings and growth of the big data industry.
Spark
Unleashing Data Science Innovations: Sparking Big Data
This document discusses data science innovations using big data. It covers topics like statistics versus data mining versus data science, the big data challenge of moving beyond transactions to relationships, different data types, Hadoop and Spark, data science discoveries and workflows, new sources of data from social media and IoT, and examples of data science innovations using Apache Spark.
Big-Data Computing on the Cloud
Andrea Pietracaprina - In this talk, we will overview some popular computing frameworks (e.g., MapReduce, Spark) which are widely used to unleash the computational potential of the cloud for big-data applications. For concreteness, we will describe efficient implementations of some key tools used in data analysis (e.g, clustering, diversity maximization).
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...Connected Data World
As one of the largest financial institutions worldwide, JP Morgan is reliant on data to drive its day-to-day operations, against an ever evolving regulatory regime. Our global data landscape possesses particular challenges of effectively maintaining data governance and metadata management.
The Data strategy at JP Morgan intends to:
a) generate business value
b) adhere to regulatory & compliance requirements
c) reduce barriers to access
d) democratize access to data
In this talk, we show how JP Morgan leverages semantic technologies to drive the implementation of our data strategy. We demonstrate how we exploit knowledge graph capabilities to answer:
1) What Data do I need?
2) What Data do we have?
3) Where does my Data come from?
4) Where should my Data come from?
5) What Data should be shared most?Big Data and Classification
The document discusses how to gain understanding from big data through effective data governance and classification. It argues that proper categorization of data using controlled vocabularies, taxonomies, and ontologies improves search, analytics and other uses of big data. A framework is presented outlining the key components of a data governance lifecycle for big data, including content creation, mining and classification, management of vocabularies/taxonomies/ontologies, and use of the structured data for search, transactions and analytics. Effective use of this framework can help organizations apply meaning and understanding to their big data.
Graph Database in Graph Intelligence
Introduces general graph database application, and the architecture of a scalable realtime graph database built on top of HBase.
Web search-metrics-tutorial-www2010-section-1of7-introduction
This document provides an introduction to a tutorial on web search engine metrics for measuring user satisfaction. It discusses the need for metrics to measure and improve search engines. It outlines the typical search engine pipeline and how metrics can evaluate different parts of the pipeline from a user and system perspective. The document then covers various considerations for collecting and analyzing metrics, such as sampling methods, metric dimensions, and challenges. It concludes by listing some key open problems in metrics and providing references for further reading.
The future of mobile and big data
Mobile and big data change the world. Goodarz Mahbobi demonstrated the effects of it at spirit conference 2014.
Tackling climate change through agricultural supply chain transparency | Javi...
Tackling climate change through agricultural supply chain transparency | Javi...Connected Data World
This document discusses using supply chain transparency to tackle sustainability challenges. It notes rising global consumption is driving widespread environmental impacts. However, there is also cause for hope with unprecedented commitments to more sustainable supply chains from many countries, companies, and organizations. Current capabilities may still be insufficient to provide the transparency needed. The Trase platform aims to map forest-risk commodity supply chains, link actors to sustainability issues in production landscapes, and monitor progress through open-access online information. The goal is for Trase to become the leading supply chain information system for traders, governments, and investors seeking more sustainable production and consumption of tropical commodities by 2020.Challenging Problems for Scalable Mining of Heterogeneous Social and Informat...
In today’s interconnected real world, social and informational entities are interconnected, forming gigantic, interconnected, integrated social and information networks. By structuring these data objects into multiple types, such networks become semi-structured heterogeneous social and information networks. Most real world applications that handle big data, including interconnected social media and social networks, medical information systems, online e-commerce systems, or database systems, can be structured into typed, heterogeneous social and information networks. For example, in a medical care network, objects of multiple types, such as patients, doctors, diseases, medication, and links such as visits, diagnosis, and treatments are intertwined together, providing rich information and forming heterogeneous information networks. Effective analysis of large-scale heterogeneous social and information networks poses an interesting but critical challenge.
In this talk, we present a set of data mining scenarios in heterogeneous social and information networks and show that mining typed, heterogeneous networks is a new and promising research frontier in data mining research. However, such mining may raise some serious challenging problems on scalability computation. We identify a set of problems on scalable computation and calls for serious studies on such problems. This includes how to efficiently computation for (1) meta path-based similarity search, (2) rank-based clustering, (3) rank-based classification, (4) meta path-based link/relationship prediction, and (5) topical hierarchies from heterogeneous information networks. We introduce some recent efforts, discuss the trade-offs between query-independent pre-computation vs. query-dependent online computation, and point out some promising research directions.
Graph Realities
Graph applications were once considered “exotic” and expensive. Until recently, few software engineers had much experience putting graphs to work. However, the use cases are now becoming more commonplace.
This talk explores a practical use case, one which addresses key issues of data governance and reproducible research, and depends on sophisticated use of graph technology.
Consider: some academic disciplines such as astronomy enjoy a wealth of data — mostly open data. Popular machine learning algorithms, open source Python libraries, and distributed systems all owe much to those disciplines and their history of big data.
Other disciplines require strong guarantees for privacy and security. Datasets used in social science research involve confidential details about human subjects: medical histories, wages, home addresses for family members, police records, etc.
Those cannot be shared openly, which impedes researchers from learning about related work by others. Reproducibility of research and the pace of science in general are limited. Nonetheless, social science research is vital for civil governance, especially for evidence-based policymaking (US federal law since 2018).
Even when data may be too sensitive to share openly, often the metadata can be shared. Constructing knowledge graphs of metadata about datasets — along with metadata about authors, their published research, methods used, data providers, data stewards, and so on — that provides effective means to tackle hard problems in data governance.
Knowledge graph work supports use cases such as entity linking, discovery and recommendations, axioms to infer about compliance, etc. This talk reviews the Rich Context AI competition and the related ADRF framework used now by more than 15 federal agencies in the US.
We’ll explore knowledge graph use cases, use of open standards and open source, and how this enhances reproducible research. Social science research for the public sector has much in common with data use in industry.
Issues of privacy, security, and compliance overlap, pointing toward what will be required of banks, media channels, etc., and what technologies apply. We’ll look at comparable work emerging in other parts of industry: open source projects, open standards emerging, and in particular a new set of features in Project Jupyter that support knowledge graphs about data governance.
Software is eating the world data is digesting it DF2013
The document discusses how software is increasingly being used to collect and analyze data, with data becoming more valuable than the software itself. It outlines the four stages of software production and how companies have transitioned from selling software to using it to collect customer data and generate insights. New technologies like Hadoop and NoSQL databases are enabling companies to more easily capture, store and analyze large volumes of user data. Data is the new differentiator as companies look to engage customers and gain insights from their behaviors online.
Neo4j GraphTalks Munich - Einführung
This document provides an agenda and information for a Neo4j GraphTalks event on DSGVO & Compliance. The agenda includes three presentations: an introduction to graph databases and Neo4j, a talk on using graphs for GDPR metadata management and data governance, and a session on successful Neo4j project implementation. The document also includes background information on Neo4j and graph databases.
Big Data: Kunden auf der Spur
Vortrag vor dem Markting Club Dortmund am 31.3.2014
Keine Streuverluste mehr dank zielgenauer, bedarfsgerechter Kundeninformation: Big Data scheint möglich zu machen, wovon Marketer träumen.
Ob das wirklich so ist, was die schnelle Analyse massenhafter Daten tatsächlich kann und welchen Stellenwert die Kreativität künftiger Kampagnen noch einnehmen wird, nimmt Meinert Jacobson unter die Lupe. Meinert Jacobson ist Geschäftsführer der Gesellschaft für Marketing, Analysen und Consulting (MarAnCon).
More Related Content
What's hot
Spark Social Media
This document discusses leveraging social big data and the evolution from existing rigid operations to predictive analytics using social media. It begins with an overview of handouts and reference materials on big data, Hadoop, Spark, and data science projects. It then discusses areas for conversation around social content, structure and analytics, data science primers and resources, and data science innovation. It presents a roadmap showing the evolution from rigid and siloed operations to being more flexible, connected, adaptive and predictive using social media. Finally, it discusses types of intentionality and how social CRM can integrate social data.
Datapreneurs
This document discusses several topics related to data and data-driven businesses. It begins by outlining trends in big data and machine learning. It then discusses how to build data-centric businesses by identifying data opportunities and sources, understanding the data lifecycle, and extracting value from data. Examples are provided of Netflix as a data-driven company. The future of professions in a data-driven world is also examined, as well as talent scarcity issues and the need for data-savvy managers. The document provides an overview of many relevant topics at the intersection of data and business.
Big Data
Big data refers to large volumes of high velocity, variety and veracity information that require advanced methods to enable analysis and insights. It is data produced from many sources, including social media interactions, website activity, photos, videos and more. Companies and governments collect and analyze big data to improve services, facilitate operations, seek new business models and boost effectiveness. However, big data also raises privacy and surveillance concerns when collected by governments. The future will see big data continue growing rapidly in size and analytical capabilities, while also facing increasing debates around security, privacy and the environmental impacts of data processing.
Systemof insight
The document discusses data science innovation and the future of professions in light of new technologies. It describes how accounting work may be automated or replaced by computer-assisted techniques and predictive analytics software. This would allow accountants to shift from reactive to proactive work by leveraging accounting data and insights to predict client scenarios and advise clients. Key areas discussed include systems of insight using big data, machine engineering to create applications from insights, and the role of data science.
Geospatial Intelligence Middle East 2013_Big Data_Steven Ramage
Some initial considerations and discussion points around geospatial big data. Location adds context and relevance. Need to consider a number of V factors including Value.
A brief history of "big data"
The history of Big Data from the beginning to Spark. An introduction to the big data software tools for developers and data analysts.
The Web3 Data Economy: Ocean Protocol
1. The document discusses the need for a "Web3 Data Economy" using blockchain and tokens to open up data access and incentivize data sharing, similar to how cryptocurrencies have opened up and incentivized money transfers.
2. It proposes that Ocean Protocol can serve as the foundational platform and token for this new data economy, fulfilling similar roles that Ethereum plays in the cryptocurrency economy like being the unit of exchange and hosting applications.
3. The goal is to connect data owners with AI companies and researchers to provide more open access to data and tools while protecting privacy and properly incentivizing all parties, helping to shift power away from current large data platforms.
Big data
Nandan Shah's presentation covered key topics related to big data including definitions, important technologies like Hadoop and MapReduce, benefits such as enabling real-time analysis and faster decision making, risks around data security and privacy, and applications across various domains such as banking, retail, and healthcare. The presentation also provided interesting facts about the size of data holdings and growth of the big data industry.
Spark
Unleashing Data Science Innovations: Sparking Big Data
This document discusses data science innovations using big data. It covers topics like statistics versus data mining versus data science, the big data challenge of moving beyond transactions to relationships, different data types, Hadoop and Spark, data science discoveries and workflows, new sources of data from social media and IoT, and examples of data science innovations using Apache Spark.
Big-Data Computing on the Cloud
Andrea Pietracaprina - In this talk, we will overview some popular computing frameworks (e.g., MapReduce, Spark) which are widely used to unleash the computational potential of the cloud for big-data applications. For concreteness, we will describe efficient implementations of some key tools used in data analysis (e.g, clustering, diversity maximization).
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...Connected Data World
As one of the largest financial institutions worldwide, JP Morgan is reliant on data to drive its day-to-day operations, against an ever evolving regulatory regime. Our global data landscape possesses particular challenges of effectively maintaining data governance and metadata management.
The Data strategy at JP Morgan intends to:
a) generate business value
b) adhere to regulatory & compliance requirements
c) reduce barriers to access
d) democratize access to data
In this talk, we show how JP Morgan leverages semantic technologies to drive the implementation of our data strategy. We demonstrate how we exploit knowledge graph capabilities to answer:
1) What Data do I need?
2) What Data do we have?
3) Where does my Data come from?
4) Where should my Data come from?
5) What Data should be shared most?Big Data and Classification
The document discusses how to gain understanding from big data through effective data governance and classification. It argues that proper categorization of data using controlled vocabularies, taxonomies, and ontologies improves search, analytics and other uses of big data. A framework is presented outlining the key components of a data governance lifecycle for big data, including content creation, mining and classification, management of vocabularies/taxonomies/ontologies, and use of the structured data for search, transactions and analytics. Effective use of this framework can help organizations apply meaning and understanding to their big data.
Graph Database in Graph Intelligence
Introduces general graph database application, and the architecture of a scalable realtime graph database built on top of HBase.
Web search-metrics-tutorial-www2010-section-1of7-introduction
This document provides an introduction to a tutorial on web search engine metrics for measuring user satisfaction. It discusses the need for metrics to measure and improve search engines. It outlines the typical search engine pipeline and how metrics can evaluate different parts of the pipeline from a user and system perspective. The document then covers various considerations for collecting and analyzing metrics, such as sampling methods, metric dimensions, and challenges. It concludes by listing some key open problems in metrics and providing references for further reading.
The future of mobile and big data
Mobile and big data change the world. Goodarz Mahbobi demonstrated the effects of it at spirit conference 2014.
Tackling climate change through agricultural supply chain transparency | Javi...
Tackling climate change through agricultural supply chain transparency | Javi...Connected Data World
This document discusses using supply chain transparency to tackle sustainability challenges. It notes rising global consumption is driving widespread environmental impacts. However, there is also cause for hope with unprecedented commitments to more sustainable supply chains from many countries, companies, and organizations. Current capabilities may still be insufficient to provide the transparency needed. The Trase platform aims to map forest-risk commodity supply chains, link actors to sustainability issues in production landscapes, and monitor progress through open-access online information. The goal is for Trase to become the leading supply chain information system for traders, governments, and investors seeking more sustainable production and consumption of tropical commodities by 2020.Challenging Problems for Scalable Mining of Heterogeneous Social and Informat...
In today’s interconnected real world, social and informational entities are interconnected, forming gigantic, interconnected, integrated social and information networks. By structuring these data objects into multiple types, such networks become semi-structured heterogeneous social and information networks. Most real world applications that handle big data, including interconnected social media and social networks, medical information systems, online e-commerce systems, or database systems, can be structured into typed, heterogeneous social and information networks. For example, in a medical care network, objects of multiple types, such as patients, doctors, diseases, medication, and links such as visits, diagnosis, and treatments are intertwined together, providing rich information and forming heterogeneous information networks. Effective analysis of large-scale heterogeneous social and information networks poses an interesting but critical challenge.
In this talk, we present a set of data mining scenarios in heterogeneous social and information networks and show that mining typed, heterogeneous networks is a new and promising research frontier in data mining research. However, such mining may raise some serious challenging problems on scalability computation. We identify a set of problems on scalable computation and calls for serious studies on such problems. This includes how to efficiently computation for (1) meta path-based similarity search, (2) rank-based clustering, (3) rank-based classification, (4) meta path-based link/relationship prediction, and (5) topical hierarchies from heterogeneous information networks. We introduce some recent efforts, discuss the trade-offs between query-independent pre-computation vs. query-dependent online computation, and point out some promising research directions.
Graph Realities
Graph applications were once considered “exotic” and expensive. Until recently, few software engineers had much experience putting graphs to work. However, the use cases are now becoming more commonplace.
This talk explores a practical use case, one which addresses key issues of data governance and reproducible research, and depends on sophisticated use of graph technology.
Consider: some academic disciplines such as astronomy enjoy a wealth of data — mostly open data. Popular machine learning algorithms, open source Python libraries, and distributed systems all owe much to those disciplines and their history of big data.
Other disciplines require strong guarantees for privacy and security. Datasets used in social science research involve confidential details about human subjects: medical histories, wages, home addresses for family members, police records, etc.
Those cannot be shared openly, which impedes researchers from learning about related work by others. Reproducibility of research and the pace of science in general are limited. Nonetheless, social science research is vital for civil governance, especially for evidence-based policymaking (US federal law since 2018).
Even when data may be too sensitive to share openly, often the metadata can be shared. Constructing knowledge graphs of metadata about datasets — along with metadata about authors, their published research, methods used, data providers, data stewards, and so on — that provides effective means to tackle hard problems in data governance.
Knowledge graph work supports use cases such as entity linking, discovery and recommendations, axioms to infer about compliance, etc. This talk reviews the Rich Context AI competition and the related ADRF framework used now by more than 15 federal agencies in the US.
We’ll explore knowledge graph use cases, use of open standards and open source, and how this enhances reproducible research. Social science research for the public sector has much in common with data use in industry.
Issues of privacy, security, and compliance overlap, pointing toward what will be required of banks, media channels, etc., and what technologies apply. We’ll look at comparable work emerging in other parts of industry: open source projects, open standards emerging, and in particular a new set of features in Project Jupyter that support knowledge graphs about data governance.
Software is eating the world data is digesting it DF2013
The document discusses how software is increasingly being used to collect and analyze data, with data becoming more valuable than the software itself. It outlines the four stages of software production and how companies have transitioned from selling software to using it to collect customer data and generate insights. New technologies like Hadoop and NoSQL databases are enabling companies to more easily capture, store and analyze large volumes of user data. Data is the new differentiator as companies look to engage customers and gain insights from their behaviors online.
Neo4j GraphTalks Munich - Einführung
This document provides an agenda and information for a Neo4j GraphTalks event on DSGVO & Compliance. The agenda includes three presentations: an introduction to graph databases and Neo4j, a talk on using graphs for GDPR metadata management and data governance, and a session on successful Neo4j project implementation. The document also includes background information on Neo4j and graph databases.
What's hot (20)
Geospatial Intelligence Middle East 2013_Big Data_Steven Ramage
Geospatial Intelligence Middle East 2013_Big Data_Steven Ramage
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...
Enterprise Data Governance: Leveraging Knowledge Graph & AI in support of a d...
Web search-metrics-tutorial-www2010-section-1of7-introduction
Web search-metrics-tutorial-www2010-section-1of7-introduction
Tackling climate change through agricultural supply chain transparency | Javi...
Tackling climate change through agricultural supply chain transparency | Javi...
Challenging Problems for Scalable Mining of Heterogeneous Social and Informat...
Challenging Problems for Scalable Mining of Heterogeneous Social and Informat...
Software is eating the world data is digesting it DF2013
Software is eating the world data is digesting it DF2013
Viewers also liked
Big Data: Kunden auf der Spur
Vortrag vor dem Markting Club Dortmund am 31.3.2014
Keine Streuverluste mehr dank zielgenauer, bedarfsgerechter Kundeninformation: Big Data scheint möglich zu machen, wovon Marketer träumen.
Ob das wirklich so ist, was die schnelle Analyse massenhafter Daten tatsächlich kann und welchen Stellenwert die Kreativität künftiger Kampagnen noch einnehmen wird, nimmt Meinert Jacobson unter die Lupe. Meinert Jacobson ist Geschäftsführer der Gesellschaft für Marketing, Analysen und Consulting (MarAnCon).
Big Data Tutorial V4
Marko Grobelnik, Blaz Fortuna, Dunja Mladenic
International Semantic Web Conference (ISWC) 2013, Sydney
Big Data and E-Commerce
Dave Feinleib discusses Big Data and how that is helping both e-commerce retailers and brands to sell more.
Big Data Tutorial - Marko Grobelnik - 25 May 2012
The document discusses big data, including what it is, key factors enabling its growth like increased storage and processing power, and techniques for handling big data like distributed processing and NoSQL databases. It provides examples of tools and applications for big data and discusses challenges like ensuring patterns found in big data analysis are actually meaningful.
Big Data in e-Commerce
How to use the power of data in e-Commerce? Applying the Big Data solutions makes it possible to analyse data in real time. This allows us to use the data not for reports only, but to translate them into action.
How to Spark Creativity, Drive Innovation, and Ensure Sustainability
Our individual, interpersonal, and organizational working lives all interconnect. By examining these connections, we learn new ways to create, innovate, adapt, and lead. The newest problems of the world find solutions in the oldest timeless practices like mindfulness, authenticity, and devotion--because everything connects. Connectivity is a sense of journey, to the sense of purpose--it is an individual, lonely pursuit and a collective, companionable one at the same time.
Viewers also liked (8)
How to Spark Creativity, Drive Innovation, and Ensure Sustainability
How to Spark Creativity, Drive Innovation, and Ensure Sustainability
Similar to Big data tutorial_part4
Big data tutorial_part4
The document discusses big data, including what it is, key factors enabling its growth, and examples of tools and techniques used for big data. It also provides examples of big data applications in areas like recommendations, network monitoring, social media analysis, and discusses challenges like dealing with data volume, velocity, and complexity.
Big data tutorial_part4
The document discusses big data, including what it is, key factors enabling its growth like increased storage and processing power, and techniques and tools used for working with big data like NoSQL databases, MapReduce, and Hadoop. It provides examples of using big data at large internet companies for personalized recommendations and detecting trends from user click streams.
Big data tutorial_part4
The document discusses big data, including what it is, key factors enabling its growth, and examples of tools and techniques used for big data. It also provides examples of big data applications in areas like recommendations, network monitoring, social media analysis, and discusses challenges like dealing with data volume, velocity, and complexity.
Big data tutorial
The document discusses big data, including what it is, key factors enabling its growth, and examples of tools and techniques used for big data. It also provides examples of big data applications in areas like recommendations, network monitoring, social media analysis, and discusses challenges like dealing with data volume, velocity, and complexity.
ESWC SS 2012 - Friday Keynote Marko Grobelnik: Big Data Tutorial
The document discusses big data techniques, tools, and applications. It describes how big data is enabled by increases in storage capacity, processing power, and data availability. It outlines common approaches to distributed processing, storage, and programming models for big data, including MapReduce, NoSQL databases, and cloud computing. It also provides examples of applications involving log file analysis, network alarm monitoring, media content analysis, and social network analysis.
EDF2013: Big Data Tutorial: Marko Grobelnik
Big Data Tutorial by Marko Grobelnik, Jozef Stefan Institute, at the European Data Forum 2013, 10 April 2013 in Dublin, Ireland
Big Data - Umesh Bellur
Prof. Bellur discusses the concepts of big data and fast data. Big data is characterized by volume, variety, and velocity, with large amounts of data coming from many sources at a high speed that is difficult to process using traditional tools. Fast data must be processed in real-time from continuous streams as it arrives, with no ability to revisit data. This presents challenges like limited memory, noise, and requiring rapid responses. Standards are emerging to help with adoption of solutions for processing both big and fast data across various domains.
Big data business case
This document discusses big data concepts including what constitutes big data, dimensions of big data, the big data lifecycle, and where big data is used. It provides examples of big data use cases in industries like automotive, healthcare, and retail. Key aspects of big data technologies like Hadoop, HDFS, MapReduce, and NoSQL databases are explained at a high level. The challenges of big data including the CAP theorem and approaches to distributed and parallel processing are also summarized.
Big data
This document discusses big data, including key enablers like increased storage and processing power. It notes that 90% of data today was created in the last two years. Big data comes from sources like mobile devices, sensors, and social media. The challenge is managing and analyzing large amounts of diverse data in a timely way. Common big data types include structured, unstructured, semi-structured, text, graph, and streaming data. Big data analytics can provide value across many domains. Issues include privacy, regulation, and ensuring analysis solves meaningful problems. The big data industry is large and growing rapidly.
MongoDB Days Silicon Valley: Jumpstart: The Right and Wrong Use Cases for Mon...
Presented by Sigfrido Narvaez, Senior Solutions Architect, MongoDB
Experience level: Introductory
When it comes time to select database software for your project, there are a bewildering number of choices. How do you know if your project is a good fit for a relational database, or whether one of the many NoSQL options is a better choice? In this session you will learn when to use MongoDB and how to evaluate if MongoDB is a fit for your project. You will see how MongoDB's flexible document model is solving business problems in ways that were not previously possible, and how MongoDB's built-in features allow running at scale.
SKILLWISE-BIGDATA ANALYSIS
General outline created by SKILLWISE about BIGDATA, which gives you a clear idea about what is bigdata?
Big Data By Vijay Bhaskar Semwal
This document provides an introduction to a training course on big data analytics. It discusses why big data has become important due to the exponential growth in data volume, velocity, and variety. The course aims to focus on cloud-based storage and processing of big data using systems like HDFS, MapReduce, HBase and Storm. It emphasizes that learning involves actively asking questions. Big data is introduced by explaining the three V's of volume, velocity and variety. Examples of big data usage are given in areas like baseball analytics, political campaigns and election predictions. Challenges of big data integration and processing large volumes of heterogeneous data are also covered.
Big data and Internet
Big data is very large data that is difficult to process using traditional methods. It is characterized by high volume, velocity, and variety. Examples of real-life big data implementations include using social media to understand customer behavior, tracking social media for marketing campaigns, and analyzing medical data to predict readmissions. Challenges include integrating diverse data sources and ensuring ethical access. Common techniques for processing big data are parallel database management systems and MapReduce frameworks like Hadoop.
Big Data et eGovernment
A l'occasion de l'eGov Innovation Day 2014 - DONNÉES DE L’ADMINISTRATION, UNE MINE (qui) D’OR(t) - Philippe Cudré-Mauroux présente Big Data et eGovernment.
Big Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science have become increasingly imperative areas in both industry and academia to the extent that every company wants to hire a Data Scientist and every university wants to start dedicated degree programs and centres of excellence in Data Science. Big Data and Data Science have led to technologies that have already shaped different aspects of our lives such as learning, working, travelling, purchasing, social relationships, entertainments, physical activities, medical treatments, etc. This talk will attempt to cover the landscape of some of the important topics in these exponentially growing areas of Data Science and Big Data including the state-of-the-art processes, commercial and open-source platforms, data processing and analytics algorithms (specially large scale Machine Learning), application areas in academia and industry, the best industry practices, business challenges and what it takes to become a Data Scientist.
ai based computer basic learning Lecture about Bigdata.ppt
ai based computer basic learning Lecture about Bigdata
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Talking about the ease of use and handling Big Data technologies in the Cloud. Using Google Cloud Platform and Amazon Web Services and all of the tools around it.
Showing the problems and how we can solve them with simple tools.
Big Data World
The document provides an overview of big data concepts including definitions, statistics on data generation and internet usage, applications and examples, challenges, and data types. It discusses key big data concepts such as the 3Vs of volume, velocity and variety; more Vs including veracity, value and visualization; data science areas and skills; the data workflow; and examples from companies like UPS, Walmart, eBay, and Kaiser Permanente.
Internet of Things
An introduction to the Internet of Things (IoT) and its underpinning technologies and platform such as Cloud Computing and Big Data.
Data analytics introduction
This document provides an introduction to big data concepts. It discusses the characteristics of big data, including volume, velocity, variety, veracity, and value. Volume refers to the large amount of data being generated. Velocity refers to the speed at which data is created and needs to be analyzed. Variety means data comes in different forms like text, images, video. Veracity refers to the quality and reliability of data. Value means the usefulness of data for businesses. The document also covers challenges in analyzing big data and different technologies used like Hadoop, Spark and cloud computing.
Similar to Big data tutorial_part4 (20)
ESWC SS 2012 - Friday Keynote Marko Grobelnik: Big Data Tutorial
ESWC SS 2012 - Friday Keynote Marko Grobelnik: Big Data Tutorial
MongoDB Days Silicon Valley: Jumpstart: The Right and Wrong Use Cases for Mon...
MongoDB Days Silicon Valley: Jumpstart: The Right and Wrong Use Cases for Mon...
Big Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science: The Technologies Shaping Our Lives
ai based computer basic learning Lecture about Bigdata.ppt
ai based computer basic learning Lecture about Bigdata.ppt
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Big Data made easy in the era of the Cloud - Demi Ben-Ari
Recently uploaded
一比一原版莱斯大学毕业证（rice毕业证）如何办理
原版一模一样【微信：741003700 】【莱斯大学毕业证（rice毕业证）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理莱斯大学毕业证（rice毕业证）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理莱斯大学毕业证（rice毕业证）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理莱斯大学毕业证（rice毕业证）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理莱斯大学毕业证（rice毕业证）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
原版一模一样【微信：741003700 】【(uom毕业证书)曼彻斯特大学毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理(uom毕业证书)曼彻斯特大学毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理(uom毕业证书)曼彻斯特大学毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理(uom毕业证书)曼彻斯特大学毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理(uom毕业证书)曼彻斯特大学毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
Gran libro. Lo recomiendo
社内勉強会資料_Hallucination of LLMs .
社内エンジニア・リサーチャー勉強会の発表資料「Hallucination of LLMs」を公開しました！
大規模言語モデル(LLM)に起こるハルシネーションのメカニズムと、それらを軽減する方法について紹介しています。
一比一原版澳洲西澳大学毕业证（uwa毕业证书）如何办理
原版一模一样【微信：741003700 】【澳洲西澳大学毕业证（uwa毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理澳洲西澳大学毕业证（uwa毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理澳洲西澳大学毕业证（uwa毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理澳洲西澳大学毕业证（uwa毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理澳洲西澳大学毕业证（uwa毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
一比一原版加拿大渥太华大学毕业证（uottawa毕业证书）如何办理
原版一模一样【微信：741003700 】【渥太华大学毕业证（uottawa毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理渥太华大学毕业证（uottawa毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版美国帕森斯设计学院毕业证（parsons毕业证书）如何办理
原版一模一样【微信：741003700 】【美国帕森斯设计学院毕业证（parsons毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理美国帕森斯设计学院毕业证（parsons毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
tim.spann@zilliz.com
https://www.linkedin.com/in/timothyspann/
https://x.com/paasdev
https://github.com/tspannhw
https://github.com/milvus-io/milvus
Get Milvused!
https://milvus.io/
Read my Newsletter every week!
https://github.com/tspannhw/FLiPStackWeekly/blob/main/142-17June2024.md
For more cool Unstructured Data, AI and Vector Database videos check out the Milvus vector database videos here
https://www.youtube.com/@MilvusVectorDatabase/videos
Unstructured Data Meetups -
https://www.meetup.com/unstructured-data-meetup-new-york/
https://lu.ma/calendar/manage/cal-VNT79trvj0jS8S7
https://www.meetup.com/pro/unstructureddata/
https://zilliz.com/community/unstructured-data-meetup
https://zilliz.com/event
Twitter/X: https://x.com/milvusio https://x.com/paasdev
LinkedIn: https://www.linkedin.com/company/zilliz/ https://www.linkedin.com/in/timothyspann/
GitHub: https://github.com/milvus-io/milvus https://github.com/tspannhw
Invitation to join Discord: https://discord.com/invite/FjCMmaJng6
Blogs: https://milvusio.medium.com/ https://www.opensourcevectordb.cloud/ https://medium.com/@tspann
Expand LLMs' knowledge by incorporating external data sources into LLMs and your AI applications.
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
We are pleased to share with you the latest VCOSA statistical report on the cotton and yarn industry for the month of March 2024.
Starting from January 2024, the full weekly and monthly reports will only be available for free to VCOSA members. To access the complete weekly report with figures, charts, and detailed analysis of the cotton fiber market in the past week, interested parties are kindly requested to contact VCOSA to subscribe to the newsletter.
一比一原版英国赫特福德大学毕业证（hertfordshire毕业证书）如何办理
原版一模一样【微信：741003700 】【英国赫特福德大学毕业证（hertfordshire毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理英国赫特福德大学毕业证（hertfordshire毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理英国赫特福德大学毕业证（hertfordshire毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理英国赫特福德大学毕业证（hertfordshire毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理英国赫特福德大学毕业证（hertfordshire毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
一比一原版加拿大麦吉尔大学毕业证（mcgill毕业证书）如何办理
原版一模一样【微信：741003700 】【加拿大麦吉尔大学毕业证（mcgill毕业证书）成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理加拿大麦吉尔大学毕业证（mcgill毕业证书）【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Build applications with generative AI on Google Cloud
We will explore Vertex AI - Model Garden powered experiences, we are going to learn more about the integration of these generative AI APIs. We are going to see in action what the Gemini family of generative models are for developers to build and deploy AI-driven applications. Vertex AI includes a suite of foundation models, these are referred to as the PaLM and Gemini family of generative ai models, and they come in different versions. We are going to cover how to use via API to: - execute prompts in text and chat - cover multimodal use cases with image prompts. - finetune and distill to improve knowledge domains - run function calls with foundation models to optimize them for specific tasks. At the end of the session, developers will understand how to innovate with generative AI and develop apps using the generative ai industry trends.
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
We are pleased to share with you the latest VCOSA statistical report on the cotton and yarn industry for the month of May 2024.
Starting from January 2024, the full weekly and monthly reports will only be available for free to VCOSA members. To access the complete weekly report with figures, charts, and detailed analysis of the cotton fiber market in the past week, interested parties are kindly requested to contact VCOSA to subscribe to the newsletter.
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
原版一模一样【微信：741003700 】【(lbs毕业证书)伦敦商学院毕业证成绩单】【微信：741003700 】学位证，留信认证（真实可查，永久存档）原件一模一样纸张工艺/offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原。
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
【主营项目】
一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等！
二.真实使馆公证(即留学回国人员证明,不成功不收费)
三.真实教育部学历学位认证（教育部存档！教育部留服网站永久可查）
四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度)
如果您处于以下几种情况：
◇在校期间，因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】
◇面对父母的压力，希望尽快拿到；
◇不清楚认证流程以及材料该如何准备；
◇回国时间很长，忘记办理；
◇回国马上就要找工作，办给用人单位看；
◇企事业单位必须要求办理的
◇需要报考公务员、购买免税车、落转户口
◇申请留学生创业基金
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】外观非常简单，由纸质材料制成，上面印有校徽、校名、毕业生姓名、专业等信息。
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】格式相对统一，各专业都有相应的模板。通常包括以下部分：
校徽：象征着学校的荣誉和传承。
校名:学校英文全称
授予学位：本部分将注明获得的具体学位名称。
毕业生姓名：这是最重要的信息之一，标志着该证书是由特定人员获得的。
颁发日期：这是毕业正式生效的时间，也代表着毕业生学业的结束。
其他信息：根据不同的专业和学位，可能会有一些特定的信息或章节。
办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】价值很高，需要妥善保管。一般来说，应放置在安全、干燥、防潮的地方，避免长时间暴露在阳光下。如需使用，最好使用复印件而不是原件，以免丢失。
综上所述，办理(lbs毕业证书)伦敦商学院毕业证【微信：741003700 】是证明身份和学历的高价值文件。外观简单庄重，格式统一，包括重要的个人信息和发布日期。对持有人来说，妥善保管是非常重要的。
Recently uploaded (20)
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
reading_sample_sap_press_operational_data_provisioning_with_sap_bw4hana (1).pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
Overview IFM June 2024 Consumer Confidence INDEX Report.pdf
06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Senior Software Profiles Backend Sample - Sheet1.pdf
Senior Software Profiles Backend Sample - Sheet1.pdf
Big data tutorial_part4
- 1. Marko Grobelnik marko.grobelnik@ijs.si Jozef Stefan Institute Ljubljana, Slovenia Stavanger, May 8th 2012
- 2. Introduction ◦ What is Big data? ◦ Why Big-Data? ◦ When Big-Data is really a problem? Techniques Tools Applications Literature
- 5. ‘Big-data’ is similar to ‘Small-data’, but bigger …but having data bigger consequently requires different approaches: ◦ techniques, tools & architectures …to solve: ◦ New problems… ◦ …and old problems in a better way.
- 6. From “Understanding Big Data” by IBM
- 8. Big-Data
- 9. Key enablers for the growth of “Big Data” are: ◦ Increase of storage capacities ◦ Increase of processing power ◦ Availability of data
- 20. NoSQL ◦ DatabasesMongoDB, CouchDB, Cassandra, Redis, BigTable, Hbase, Hypertable, Voldemort, Riak, ZooKeeper MapReduce ◦ Hadoop, Hive, Pig, Cascading, Cascalog, mrjob, Caffeine, S4, MapR, Acunu, Flume, Kafka, Azkaban, Oozie, Greenplum Storage ◦ S3, Hadoop Distributed File System Servers ◦ EC2, Google App Engine, Elastic, Beanstalk, Heroku Processing ◦ R, Yahoo! Pipes, Mechanical Turk, Solr/Lucene, ElasticSearch, Datameer, BigSheets, Tinkerpop
- 22. …when the operations on data are complex: ◦ …e.g. simple counting is not a complex problem ◦ Modeling and reasoning with data of different kinds can get extremely complex Good news about big-data: ◦ Often, because of vast amount of data, modeling techniques can get simpler (e.g. smart counting can replace complex model based analytics)… ◦ …as long as we deal with the scale
- 23. Research areas (such as IR, KDD, ML, NLP, SemWeb, …) are sub- cubes within the data cube Scalability Dynamicity Context Quality Usage
- 26. Good recommendations can make a big difference when keeping a user on a web site ◦ …the key is how rich context model a system is using to select information for a user ◦ Bad recommendations <1% users, good ones >5% users click Contextual personalized recommendations generated in ~20ms
- 27. Domain Sub-domain Page URL URL sub-directories Page Meta Tags Page Title Page Content Named Entities Has Query Referrer Query Referring Domain Referring URL Outgoing URL GeoIP Country GeoIP State GeoIP City Absolute Date Day of the Week Day period Hour of the day User Agent Zip Code State Income Age Gender Country Job Title Job Industry
- 28. Log Files (~100M page clicks per day) User profiles NYT articles Stream of profiles Advertisers Segment Keywords Stock Market Stock Market, mortgage, banking, investors, Wall Street, turmoil, New York Stock Exchange Health diabetes, heart disease, disease, heart, illness Green Energy Hybrid cars, energy, power, model, carbonated, fuel, bulbs, Hybrid cars Hybrid cars, vehicles, model, engines, diesel Travel travel, wine, opening, tickets, hotel, sites, cars, search, restaurant … … Segments Trend Detection System Stream of clicks Trends and updated segments Campaign to sell segments $ Sales
- 29. 50Gb of uncompressed log files 10Gb of compressed log files 0.5Gb of processed log files 50-100M clicks 4-6M unique users 7000 unique pages with more then 100 hits Index size 2Gb Pre-processing & indexing time ◦ ~10min on workstation (4 cores & 32Gb) ◦ ~1hour on EC2 (2 cores & 16Gb)
- 31. Alarms Explorer Server implements three real-time scenarios on the alarms stream: 1. Root-Cause-Analysis – finding which device is responsible for occasional “flood” of alarms 2. Short-Term Fault Prediction – predict which device will fail in next 15mins 3. Long-Term Anomaly Detection – detect unusual trends in the network …system is used in British Telecom Alarms Server Alarms Explorer Server Live feed of data Operator Big board display Telecom Network (~25 000 devices) Alarms ~10-100/sec
- 32. Presented in “Planetary-Scale Views on a Large Instant-Messaging Network” by Jure Leskovec and Eric Horvitz WWW2008
- 33. Observe social and communication phenomena at a planetary scale Largest social network analyzed to date Research questions: How does communication change with user demographics (age, sex, language, country)? How does geography affect communication? What is the structure of the communication network? 33
- 34. We collected the data for June 2006 Log size: 150Gb/day (compressed) Total: 1 month of communication data: 4.5Tb of compressed data Activity over June 2006 (30 days) ◦ 245 million users logged in ◦ 180 million users engaged in conversations ◦ 17,5 million new accounts activated ◦ More than 30 billion conversations ◦ More than 255 billion exchanged messages 34
- 35. 35
- 36. 36
- 37. Count the number of users logging in from particular location on the earth 37
- 38. Logins from Europe 38
- 39. 6 degrees of separation [Milgram ’60s] Average distance between two random users is 6.6 90% of nodes can be reached in < 8 hops Hops Nodes 1 10 2 78 3 396 4 8648 5 3299252 6 28395849 7 79059497 8 52995778 9 10321008 10 1955007 11 518410 12 149945 13 44616 14 13740 15 4476 16 1542 17 536 18 167 19 71 20 29 21 16 22 10 23 3 24 2 25 3