0629venmoplus
•
0 likes•118 views
Introducing VenmoPlus.com, a service that allows users to explore their Venmo network in more depth. The document discusses the challenges of calculating and querying the graph distance between users in real-time as the graph is constantly changing. It describes how the service uses two databases - a real-time database to track new transactions and a historical database to store older transaction data. It also discusses algorithms and optimizations used to query distances and recommend connections between users efficiently.
Report
Share
Report
Share
Download to read offline
Recommended
VenmoPlus demo week6
Introducing VenmoPlus.com, a service that allows users to explore their Venmo network and query transaction histories and relationships between users in real time. It uses Redis to store the graph structure defining relationships between users and ElasticSearch to store all other data for querying. Algorithms and optimizations were developed to quickly calculate distances in the large and dynamic graph in real time using a combination of Redis and ElasticSearch.
Test Trend Analysis : Towards robust, reliable and timely tests
Slides from my talk at Selenium Conference 2016.
In this talk you will get ideas about how you can instrument test result information to provide actionable data, paving the way for more robust, reliable and timely test results.
By capturing this information over time, and when combined with visualization tools, we can answer different questions than with existing solutions (Allure / CI tool build history). Some examples of these are:
Which tests are consistently flaky
What are the common causes of failure across tests
Which tests consistently take a long time to run
Using this information we can move away from the ‘re-run’ culture and better support continuous integration goals of having quick, reliable, deterministic tests
Video of the talk is here: https://youtu.be/29fPYx7OJnE?list=PL_7kBU2XBlbKuRNVHeqjXUygXtToqMHsn
Data pipelines observability: OpenLineage & Marquez
This document discusses OpenLineage and Marquez, which aim to provide standardized metadata and data lineage collection for data pipelines. OpenLineage defines an open standard for collecting metadata as data moves through pipelines, similar to metadata collected by EXIF for images. Marquez is an open source implementation of this standard, which can collect metadata from various data tools and store it in a graph database for querying lineage and understanding dependencies. This collected metadata helps with tasks like troubleshooting, impact analysis, and understanding how data flows through complex pipelines over time.
Test trend analysis: Towards robust reliable and timely tests
This document discusses test trend analysis and making tests more robust, reliable, and timely. It proposes collecting test results data and storing it in Elasticsearch. Visualizations would then be created using Kibana to analyze test failures, slow tests, error messages, and step times. This would provide insights and help identify issues to make tests less flaky.
Data_Size_statistics
The document provides details on data sizes for various projects worked on using Hadoop/Spark, including the Panera LLC Capacity Planning and Predictive Analytics projects, AT&T Insights production and non-production projects, a CTL data lake ingestion project, and an AT&T Telegence Mobility project. It notes that the total data size across all projects is approximately 52.5 TB, with unstructured data making up 36.2 TB (69%), structured data accounting for 9 TB (17%), and semi-structured data consisting of 7.3 TB (14%).
Semantic Web Challenges for Visualisation and Visual Analytics
The document discusses the semantic web and challenges for visualizing semantic web data. It defines the semantic web as a web of linked data described using technologies like RDF, OWL, and triples. It notes there are three paths to creating semantic web data: adding markup to existing HTML, converting existing data like CSV files into RDF triples, or describing raw data as RDF triples. The document outlines challenges in understanding raw data, converting it to RDF, dealing with issues of identity and exceptions, and linking semantic data in an open web of incomplete and heterogeneous linked data.
Remodeling the Getty Provenance Index as Linked Open Data
Presented at 2016 Digital Provenance Symposium at Carnegie Museum of Art. Covers history of Getty Provenance Index, experiences encountered and lessons learned while remodeling the databases for publication as Linked Open Data, including mapping auction sales catalogs to the CIDOC-CRM, developing a URI scheme, and reconciling these resources with the Getty Vocabularies.
10 ways to stumble with big data
Many companies have data with great potential. There are many ways to go wrong with Big Data projects, however; the difference between a successful and a failed project can be huge, both in cost and the return of investment. In this talk. we will describe the most common pitfalls, and how to avoid them. You will learn to:
- Be aware of the existing risk factors in your organisation that may cause a data project to fail.
- Learn how to recognise the most common and costly causes of project failure.
- Learn how to avoid or mitigate project problems in order to ensure return of investment in a lean manner.
Recommended
VenmoPlus demo week6
Introducing VenmoPlus.com, a service that allows users to explore their Venmo network and query transaction histories and relationships between users in real time. It uses Redis to store the graph structure defining relationships between users and ElasticSearch to store all other data for querying. Algorithms and optimizations were developed to quickly calculate distances in the large and dynamic graph in real time using a combination of Redis and ElasticSearch.
Test Trend Analysis : Towards robust, reliable and timely tests
Slides from my talk at Selenium Conference 2016.
In this talk you will get ideas about how you can instrument test result information to provide actionable data, paving the way for more robust, reliable and timely test results.
By capturing this information over time, and when combined with visualization tools, we can answer different questions than with existing solutions (Allure / CI tool build history). Some examples of these are:
Which tests are consistently flaky
What are the common causes of failure across tests
Which tests consistently take a long time to run
Using this information we can move away from the ‘re-run’ culture and better support continuous integration goals of having quick, reliable, deterministic tests
Video of the talk is here: https://youtu.be/29fPYx7OJnE?list=PL_7kBU2XBlbKuRNVHeqjXUygXtToqMHsn
Data pipelines observability: OpenLineage & Marquez
This document discusses OpenLineage and Marquez, which aim to provide standardized metadata and data lineage collection for data pipelines. OpenLineage defines an open standard for collecting metadata as data moves through pipelines, similar to metadata collected by EXIF for images. Marquez is an open source implementation of this standard, which can collect metadata from various data tools and store it in a graph database for querying lineage and understanding dependencies. This collected metadata helps with tasks like troubleshooting, impact analysis, and understanding how data flows through complex pipelines over time.
Test trend analysis: Towards robust reliable and timely tests
This document discusses test trend analysis and making tests more robust, reliable, and timely. It proposes collecting test results data and storing it in Elasticsearch. Visualizations would then be created using Kibana to analyze test failures, slow tests, error messages, and step times. This would provide insights and help identify issues to make tests less flaky.
Data_Size_statistics
The document provides details on data sizes for various projects worked on using Hadoop/Spark, including the Panera LLC Capacity Planning and Predictive Analytics projects, AT&T Insights production and non-production projects, a CTL data lake ingestion project, and an AT&T Telegence Mobility project. It notes that the total data size across all projects is approximately 52.5 TB, with unstructured data making up 36.2 TB (69%), structured data accounting for 9 TB (17%), and semi-structured data consisting of 7.3 TB (14%).
Semantic Web Challenges for Visualisation and Visual Analytics
The document discusses the semantic web and challenges for visualizing semantic web data. It defines the semantic web as a web of linked data described using technologies like RDF, OWL, and triples. It notes there are three paths to creating semantic web data: adding markup to existing HTML, converting existing data like CSV files into RDF triples, or describing raw data as RDF triples. The document outlines challenges in understanding raw data, converting it to RDF, dealing with issues of identity and exceptions, and linking semantic data in an open web of incomplete and heterogeneous linked data.
Remodeling the Getty Provenance Index as Linked Open Data
Presented at 2016 Digital Provenance Symposium at Carnegie Museum of Art. Covers history of Getty Provenance Index, experiences encountered and lessons learned while remodeling the databases for publication as Linked Open Data, including mapping auction sales catalogs to the CIDOC-CRM, developing a URI scheme, and reconciling these resources with the Getty Vocabularies.
10 ways to stumble with big data
Many companies have data with great potential. There are many ways to go wrong with Big Data projects, however; the difference between a successful and a failed project can be huge, both in cost and the return of investment. In this talk. we will describe the most common pitfalls, and how to avoid them. You will learn to:
- Be aware of the existing risk factors in your organisation that may cause a data project to fail.
- Learn how to recognise the most common and costly causes of project failure.
- Learn how to avoid or mitigate project problems in order to ensure return of investment in a lean manner.
ROI in Linking Content to CRM by Applying the Linked Data Stack
Today, decision makers in enterprises have to rely more and more on a variety of data sets that are internally but also externally available in heterogeneous formats. Therefore, intelligent processes are required to build an integrated knowledge-base. Unfortunately, the adoption of the Linked Data lifecycle within enterprises, which targets the extraction, interlinking, publishing and analytics of distributed data, lags behind the public domain due to missing frameworks that are efficiently to deploy and ease to use. In this paper, we present our adoption of the lifecycle through our generic, enterprise-ready Linked Data workbench. To judge its benefits, we describe its application within a real-world Customer Relationship Management scenario. It shows (1) that sales employee could significantly reduce their workload and (2) that the integration of sophisticated Linked Data tools come with an obvious positive Return on Investment.
MLSD18. Basic Transformations - QCRI
This document summarizes a presentation on programming by example (PBE) and two PBE systems, FlashFill and Foofah. It discusses how PBE works by taking input-output examples to synthesize a program for transforming raw data. It also provides examples of possible data transformations and demonstrates FlashFill and Foofah for transforming structured and unstructured data.
MLSD18. Data Cleaning
Data Profiling, Data Cleaning, and Error Detection Data Extraction.
MLSD18: 1st edition of the Machine Learning School in Doha.
MLSD18. Summary of Morning Sessions
The document summarizes a morning session at the Machine Learning School in Doha on November 4-5, 2018. It discusses machine learning and traditional programming approaches. It then covers the ideal and actual machine learning workflows, the importance of preparing clean machine learning ready data, and various machine learning algorithms like classification, regression, anomaly detection and clustering. It also discusses techniques for transforming data like joins, aggregations and pivoting. Finally, it discusses programming by example as a way to synthesize programs from input-output examples to transform raw data.
MLSD18. Real-World Use Case I
Preparing your Data for Machine Learning.
MLSD18: 1st edition of the Machine Learning School in Doha.
Research Automation for Data-Driven Discovery
This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Globus and the Materials Data Facility are presented. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
CCCB Germline Variant Analysis on Cloud Platform
How to request CCCB germline variant analysis on cloud platform to rapidly generate vcf files from sequence files
Enabling Clinical Research in the Real World
Speaker: Brenda Deverell Cortez, Software Developer, Koneksa Health
Speaker: Tom Hosford, VP Engineering, Koneksa Health
Level: 100 (Beginner)
Track: Developer
Koneksa Health has built an application from the ground up to capture real world patient data in clinical trials. Patients with respiratory disease who formerly could only visit a hospital to measure key data points are now able to capture data with a small device from their own home. This example driven presentation will focus on how a full stack javascript development approach, with Mongo and Node, has been used to build the Koneksa Compare application, with emphasis on the technical challenges encountered and how these have been overcome by the application developers.
What You Will Learn:
- Data modeling and join strategies leveraging Mongoose and NodeJS
- Performance techniques for building a fast and efficient user experience
- Fault tolerance strategies for building a rock-solid, reliable application
Honey on the Wire KohaCon18
The document discusses various data and technology concepts relevant for libraries, including blockchain, data lakes, FRBR linked data models, and using a "Brainz" model. Blockchain could be used for digital first sale programs, patron data privacy, and other applications. Data lakes store raw data for archival, statistics and searches. Data lakes can provide data as a service through subscriptions and reports. FRBR models are reexamined to allow for non-canonical work identifiers. The document suggests libraries could implement a "Koha brainz" model inspired by Music Brainz and Book Brainz for organizing metadata.
[Data Innovation Summit 2015] Belga Big Content Platform
Big Content is a scalable big data platform that uses Hadoop, Spark, and other open source tools to process streaming and stored data from various public sources. It addresses common big data challenges like processing large datasets, real-time analytics, and integrating diverse data types through modular components like its content indexer, analysis chain, and interfaces for management and search. The platform helps customers gain insights from massive and growing amounts of information.
Real-time Data Analytics mit Elasticsearch
Elasticsearch hat sich in kurzer Zeit zu einer Top Open-Source-Suchtechnologie etabliert. Fernab von Such-Szenarien zeigt Elasticsearch auch im Bereich Real-time Data Analytics auf "Big Data" seine Stärken. Warum? Es ist skalierbarer Document Store und bietet performante Analysemethoden. Es bietet eine Hadoop Integration zu Pig, Hive, MapReduce und das Analyse-Webfront Kibana für interaktive Dashboards und Enduser-Analysen ist auch mit dabei. In diesem Vortrag möchte ich dem Publikum näherbringen, wie Elasticsearch für Real-time Data Analytics auf großen Datenvolumina genutzt werden kann.
A Data Decomposition Method for Stepwise Migration of Complex Legacy Data
Sooner or later, in almost every company, the maintenance and further development of large enterprise IT applications reaches its limit. From the point of view of cost as well as technical capability, legacy applications must eventually be replaced by new enterprise IT applications. Data migration is an inevitable part of making this switch. While different data migration strategies can be applied, incremental data migration is one of the most popular strategies, due to its low level of risk: e entire data volume is split into several data tranches, which are then migrated in individual migration steps. The key to a successful migration is the strategy for
decomposing the data into suitable tranches.
PageRank and Related Methods
This document discusses PageRank and related methods for ranking web pages. It provides an overview that includes: what PageRank is based on links as votes; algorithms for calculating PageRank using a simplified method and damping factor; different PageRank values from actual, toolbars, and ODP; manipulating PageRank ratings; and the relationship between PageRank and the semantic web. It also mentions two similar works from 1998 - Kleinberg's authoritative sources algorithm and Chakrabarti's experiments in topic distillation which added factors to PageRank.
GDBinSV_Meetup_DBMS_Trends_10062016
The DBMS market trends focused on the Graph DBMS. The benefit of the Graph Database and its forecasted the growth rate. The Advice from the renowned market research institute.
JSON-LD and SHACL for Knowledge Graphs
Enterprise systems are increasingly complex, often requiring data and software components to be accessed and maintained by different company departments. This complexity often becomes an organization’s biggest challenge as changing data fields and adding new applications rapidly grow to meet business demands for increased customer insights.
These slides are from a Webinar discussing how using SHACL and JSON-LD with AllegroGraph helps our customers simplify the complexity of enterprise systems through the ability to loosely combine independent elements, while allowing the overall system to function smoothly.
In this Webinar we will demonstrate how AllegroGraph’s SHACL validation engine confirms whether JSON-LD data is conforming to the desired requirements. We will describe how SHACL provides a way for a Data Graph to specify the Shapes Graph that should be used for validation and describes how a given shape is linked to targets in the data.
The recording is at youtube.com/allegrograph
Data tools ecosystem for non-programmers
Presentation by @oscarmarinmiro at Itnig, November 8th 2013, about non-programming resources for getting, analyzing and visualizing data
GraphDB
This document discusses graph databases and the graph database Neo4j. It provides an introduction to NoSQL databases and graph theory, including graph algorithms. It outlines some common uses of graph databases such as social networking, recommendations, and identity and access management. It also provides examples of Cypher queries that can be used with Neo4j to find and create nodes and relationships.
Scaling collaborative data science with Globus and Jupyter
The Globus service simplifies the utilization of large and distributed data on the Jupyter platform. Ian Foster explains how to use Globus and Jupyter to seamlessly access notebooks using existing institutional credentials, connect notebooks with data residing on disparate storage systems, and make data securely available to business partners and research collaborators.
Georeferenciar Imágenes (GOOGLE EARTH, raster desing Y CIVIL 3D 2015,2016)
El documento describe el proceso de georreferenciar imágenes en Google Earth y Civil 3D. Primero, se crea un rectángulo en Civil 3D para representar el área de la imagen y se exporta a Google Earth. Luego, se guarda una captura de pantalla de la imagen dentro del rectángulo y se inserta en Civil 3D usando Raster Design. Finalmente, se georrefencia la imagen seleccionando puntos de referencia para alinearla con el rectángulo original.
The Future of Search Marketing
Elisabeth (@elisabethos) is a veteran in the search engine marketing industry, having worked as an SEO consultant and search marketing analyst since 1999. As the Executive Features Editor for Search Engine Land, she is responsible for managing all editorial content from guest contributors and daily articles from expert columnists.
To learn more about the Insight Summit Series please visit http://www.insightsummitseries.com
Receta para una vejez feliz
El documento proporciona una receta para una vejez feliz que incluye evitar el tabaco y el alcohol, mantener una dieta saludable rica en frutas y verduras, realizar actividad física regularmente, mantener una vida sexual activa, tener una casa segura libre de riesgos de caídas, y mantener la mente sana a través de actividades intelectuales y sociales.
More Related Content
What's hot
ROI in Linking Content to CRM by Applying the Linked Data Stack
Today, decision makers in enterprises have to rely more and more on a variety of data sets that are internally but also externally available in heterogeneous formats. Therefore, intelligent processes are required to build an integrated knowledge-base. Unfortunately, the adoption of the Linked Data lifecycle within enterprises, which targets the extraction, interlinking, publishing and analytics of distributed data, lags behind the public domain due to missing frameworks that are efficiently to deploy and ease to use. In this paper, we present our adoption of the lifecycle through our generic, enterprise-ready Linked Data workbench. To judge its benefits, we describe its application within a real-world Customer Relationship Management scenario. It shows (1) that sales employee could significantly reduce their workload and (2) that the integration of sophisticated Linked Data tools come with an obvious positive Return on Investment.
MLSD18. Basic Transformations - QCRI
This document summarizes a presentation on programming by example (PBE) and two PBE systems, FlashFill and Foofah. It discusses how PBE works by taking input-output examples to synthesize a program for transforming raw data. It also provides examples of possible data transformations and demonstrates FlashFill and Foofah for transforming structured and unstructured data.
MLSD18. Data Cleaning
Data Profiling, Data Cleaning, and Error Detection Data Extraction.
MLSD18: 1st edition of the Machine Learning School in Doha.
MLSD18. Summary of Morning Sessions
The document summarizes a morning session at the Machine Learning School in Doha on November 4-5, 2018. It discusses machine learning and traditional programming approaches. It then covers the ideal and actual machine learning workflows, the importance of preparing clean machine learning ready data, and various machine learning algorithms like classification, regression, anomaly detection and clustering. It also discusses techniques for transforming data like joins, aggregations and pivoting. Finally, it discusses programming by example as a way to synthesize programs from input-output examples to transform raw data.
MLSD18. Real-World Use Case I
Preparing your Data for Machine Learning.
MLSD18: 1st edition of the Machine Learning School in Doha.
Research Automation for Data-Driven Discovery
This document discusses research automation and data-driven discovery. It notes that data volumes are growing much faster than computational power, creating a productivity crisis in research. However, most labs have limited resources to handle these large data volumes. The document proposes applying lessons from industry to create cloud-based science services with standardized APIs that can automate and outsource common tasks like data transfer, sharing, publishing, and searching. This would help scientists focus on their core research instead of computational infrastructure. Examples of existing services from Globus and the Materials Data Facility are presented. The goal is to establish robust, scalable, and persistent cloud platforms to help address the challenges of data-driven scientific discovery.
CCCB Germline Variant Analysis on Cloud Platform
How to request CCCB germline variant analysis on cloud platform to rapidly generate vcf files from sequence files
Enabling Clinical Research in the Real World
Speaker: Brenda Deverell Cortez, Software Developer, Koneksa Health
Speaker: Tom Hosford, VP Engineering, Koneksa Health
Level: 100 (Beginner)
Track: Developer
Koneksa Health has built an application from the ground up to capture real world patient data in clinical trials. Patients with respiratory disease who formerly could only visit a hospital to measure key data points are now able to capture data with a small device from their own home. This example driven presentation will focus on how a full stack javascript development approach, with Mongo and Node, has been used to build the Koneksa Compare application, with emphasis on the technical challenges encountered and how these have been overcome by the application developers.
What You Will Learn:
- Data modeling and join strategies leveraging Mongoose and NodeJS
- Performance techniques for building a fast and efficient user experience
- Fault tolerance strategies for building a rock-solid, reliable application
Honey on the Wire KohaCon18
The document discusses various data and technology concepts relevant for libraries, including blockchain, data lakes, FRBR linked data models, and using a "Brainz" model. Blockchain could be used for digital first sale programs, patron data privacy, and other applications. Data lakes store raw data for archival, statistics and searches. Data lakes can provide data as a service through subscriptions and reports. FRBR models are reexamined to allow for non-canonical work identifiers. The document suggests libraries could implement a "Koha brainz" model inspired by Music Brainz and Book Brainz for organizing metadata.
[Data Innovation Summit 2015] Belga Big Content Platform
Big Content is a scalable big data platform that uses Hadoop, Spark, and other open source tools to process streaming and stored data from various public sources. It addresses common big data challenges like processing large datasets, real-time analytics, and integrating diverse data types through modular components like its content indexer, analysis chain, and interfaces for management and search. The platform helps customers gain insights from massive and growing amounts of information.
Real-time Data Analytics mit Elasticsearch
Elasticsearch hat sich in kurzer Zeit zu einer Top Open-Source-Suchtechnologie etabliert. Fernab von Such-Szenarien zeigt Elasticsearch auch im Bereich Real-time Data Analytics auf "Big Data" seine Stärken. Warum? Es ist skalierbarer Document Store und bietet performante Analysemethoden. Es bietet eine Hadoop Integration zu Pig, Hive, MapReduce und das Analyse-Webfront Kibana für interaktive Dashboards und Enduser-Analysen ist auch mit dabei. In diesem Vortrag möchte ich dem Publikum näherbringen, wie Elasticsearch für Real-time Data Analytics auf großen Datenvolumina genutzt werden kann.
A Data Decomposition Method for Stepwise Migration of Complex Legacy Data
Sooner or later, in almost every company, the maintenance and further development of large enterprise IT applications reaches its limit. From the point of view of cost as well as technical capability, legacy applications must eventually be replaced by new enterprise IT applications. Data migration is an inevitable part of making this switch. While different data migration strategies can be applied, incremental data migration is one of the most popular strategies, due to its low level of risk: e entire data volume is split into several data tranches, which are then migrated in individual migration steps. The key to a successful migration is the strategy for
decomposing the data into suitable tranches.
PageRank and Related Methods
This document discusses PageRank and related methods for ranking web pages. It provides an overview that includes: what PageRank is based on links as votes; algorithms for calculating PageRank using a simplified method and damping factor; different PageRank values from actual, toolbars, and ODP; manipulating PageRank ratings; and the relationship between PageRank and the semantic web. It also mentions two similar works from 1998 - Kleinberg's authoritative sources algorithm and Chakrabarti's experiments in topic distillation which added factors to PageRank.
GDBinSV_Meetup_DBMS_Trends_10062016
The DBMS market trends focused on the Graph DBMS. The benefit of the Graph Database and its forecasted the growth rate. The Advice from the renowned market research institute.
JSON-LD and SHACL for Knowledge Graphs
Enterprise systems are increasingly complex, often requiring data and software components to be accessed and maintained by different company departments. This complexity often becomes an organization’s biggest challenge as changing data fields and adding new applications rapidly grow to meet business demands for increased customer insights.
These slides are from a Webinar discussing how using SHACL and JSON-LD with AllegroGraph helps our customers simplify the complexity of enterprise systems through the ability to loosely combine independent elements, while allowing the overall system to function smoothly.
In this Webinar we will demonstrate how AllegroGraph’s SHACL validation engine confirms whether JSON-LD data is conforming to the desired requirements. We will describe how SHACL provides a way for a Data Graph to specify the Shapes Graph that should be used for validation and describes how a given shape is linked to targets in the data.
The recording is at youtube.com/allegrograph
Data tools ecosystem for non-programmers
Presentation by @oscarmarinmiro at Itnig, November 8th 2013, about non-programming resources for getting, analyzing and visualizing data
GraphDB
This document discusses graph databases and the graph database Neo4j. It provides an introduction to NoSQL databases and graph theory, including graph algorithms. It outlines some common uses of graph databases such as social networking, recommendations, and identity and access management. It also provides examples of Cypher queries that can be used with Neo4j to find and create nodes and relationships.
Scaling collaborative data science with Globus and Jupyter
The Globus service simplifies the utilization of large and distributed data on the Jupyter platform. Ian Foster explains how to use Globus and Jupyter to seamlessly access notebooks using existing institutional credentials, connect notebooks with data residing on disparate storage systems, and make data securely available to business partners and research collaborators.
What's hot (18)
ROI in Linking Content to CRM by Applying the Linked Data Stack
ROI in Linking Content to CRM by Applying the Linked Data Stack
[Data Innovation Summit 2015] Belga Big Content Platform
[Data Innovation Summit 2015] Belga Big Content Platform
A Data Decomposition Method for Stepwise Migration of Complex Legacy Data
A Data Decomposition Method for Stepwise Migration of Complex Legacy Data
Scaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and Jupyter
Viewers also liked
Georeferenciar Imágenes (GOOGLE EARTH, raster desing Y CIVIL 3D 2015,2016)
El documento describe el proceso de georreferenciar imágenes en Google Earth y Civil 3D. Primero, se crea un rectángulo en Civil 3D para representar el área de la imagen y se exporta a Google Earth. Luego, se guarda una captura de pantalla de la imagen dentro del rectángulo y se inserta en Civil 3D usando Raster Design. Finalmente, se georrefencia la imagen seleccionando puntos de referencia para alinearla con el rectángulo original.
The Future of Search Marketing
Elisabeth (@elisabethos) is a veteran in the search engine marketing industry, having worked as an SEO consultant and search marketing analyst since 1999. As the Executive Features Editor for Search Engine Land, she is responsible for managing all editorial content from guest contributors and daily articles from expert columnists.
To learn more about the Insight Summit Series please visit http://www.insightsummitseries.com
Receta para una vejez feliz
El documento proporciona una receta para una vejez feliz que incluye evitar el tabaco y el alcohol, mantener una dieta saludable rica en frutas y verduras, realizar actividad física regularmente, mantener una vida sexual activa, tener una casa segura libre de riesgos de caídas, y mantener la mente sana a través de actividades intelectuales y sociales.
Genre survey results
Most respondents to the survey were female, between 16-18 years old. The survey was shared on social media where younger audiences are more likely to see it. The author was surprised that over half of respondents preferred action genres for a trailer over comedy or romance. This feedback will influence the author to create an action trailer rather than a romance one, which would attract a smaller audience. Audio and a wide range of shot types were favored elements for trailers, more so than characters or style. While romance was not a preferred main genre, it scored well as a sub-genre element, showing audiences may enjoy romance as a component versus the full focus.
Letter sound- key word quiz three
This document provides a scrambled letter-sound-key word quiz for lesson 1. Students are instructed to say the letter, sound and key word for each letter shown to practice phonics. The document contains two letters - "Qu" and "qu" - to be decoded in this phonics exercise.
Atlas prici decision
This document provides an executive summary and marketing plan for Atlas, a new technology company launching in Nepal. The summary includes:
1) Atlas aims to launch a new revolution in technology and target specific consumer and business market segments.
2) The primary objectives are to achieve break-even within 18 months by selling 55,000 units and achieve sales revenue of 1.5 billion rupees in the first year, turning initial losses to less than 20% of expected turnover.
3) The marketing plan outlines objectives to increase brand awareness and market share through promotional activities, reduce prices, and expand distribution over the next year.
Christina_2014 CV
Christina Adelshafik is an Egyptian architect currently working in business development at Pavilion Architects in Cairo. She received her bachelor's degree in architecture from Helwan University in 2007, graduating with distinction for her project on establishing an institute for monument restoration and conservation. Her work experience includes over 10 years in international business development roles for architecture and consulting firms, managing client relationships, proposals, and new business opportunities in multiple countries.
Scalable. Digital. Success.
Scott is a seasoned eCommerce & Digital Marketing professional with a career spanning over 15 years building eCommerce systems and managing the Digital presence for Fortune 500 companies and currently is the Web Services Division Manager at SERVPRO Industries, Inc.
To learn more about the Insight Summit Series please visit http://www.insightsummitseries.com
Evaluation question 2
The document discusses how the media product represents its target social group of 16-25 year old males. It uses a color scheme of black, red, and grey/white that is associated with young men. A messy, "grungy" font style is used to appeal to this audience. The front cover features a female artist, as the target group is stereotypically interested in girls. The price of £2.00 per monthly issue and inclusion of festival content makes the magazine affordable and relevant to its target "e" demographic group of students. The double page article is in a Q&A format for easier reading by males in this age range.
Letter sound- key word2
This document provides a list of letter-sound associations to teach basic phonics. Each letter of the alphabet from A to Z is paired with a simple key word beginning with that letter to demonstrate the associated sound. For example, A is paired with "apple" and B is paired with "bat" to teach the sounds of those letters.
Q2 final
The student created a short film for a coursework project along with two ancillary texts: a poster and online review. These texts were designed to advertise and promote the film to its target audience of 15-24 year olds. While the poster and film shared similar visual styles and color schemes, the review used the style and layout of the Rotten Tomatoes website to seem more authentic, though it lacked visual consistency with the other pieces. Overall, the ancillary texts worked together to create an advertising campaign for the short film while breaking some conventions by differing in visual presentation.
Bethany 22
Twenty two years ago a wonderful little girl came into our lives. We were forever changed! Love you Bethany and happy birthday!
Q2 final
The student created a short film for a coursework project along with two ancillary texts: a poster and online review. These texts were designed to advertise and promote the film to its target audience of 15-24 year olds. While the poster and film shared similar visual styles and color schemes, the review used the style and layout of the Rotten Tomatoes website to seem more authentic, though it lacked visual consistency with the other pieces. Overall, the ancillary texts worked together to create an advertising campaign for the short film while breaking some conventions by differing in visual presentation.
Aula de la naturaleza
El documento habla sobre la contaminación por plásticos en el Atolón de Midway y sus efectos en los albatros. Los albatros están muriendo después de ingerir plásticos que confunden con comida. La contaminación por plásticos es un problema grave que afecta a la vida silvestre en este atolón.
Recticel
This document presents an analysis of financial ratios for Recticel, a Belgian chemicals company. It calculates and compares Recticel's current ratio, quick ratio, EBITDA, and profit margin to benchmarks like Tessenderlo from 2009-2012. The current ratio is above 1, indicating Recticel is liquid, but the quick ratio is below 1, suggesting it may not be liquid in theory. EBITDA has increased steadily each year. Profit margin comparisons show Recticel performs well against competitors like Tessenderlo, Solvay, and Ter beke. The conclusion is that Recticel is a stable company, but there may be better investment options.
Evaluation 2
The document discusses the planning and research process the author undertook to produce magazines as part of a college coursework project. They conducted extensive online and print research to analyze existing magazine formats, styles, layouts, and content. This research informed the planning and production of their college magazine and music magazine, from developing initial layout concepts to determining appropriate content and target audiences. The author reflects on areas they could have improved, such as better time management and blog organization. Overall, the research and planning process played an important role in the successful production of professional-looking magazines.
Itg investor presentation_30apr15
- ITG has achieved international growth, with Europe and Asia Pacific setting new revenue records in Q1 2015 and international operations now comprising 46% of revenues and almost 72% of pre-tax income.
- ITG has a robust balance sheet with $206M in cash/equivalents and only $15M in long-term debt as well as an active share buyback program.
- ITG offers a range of trading, analytics and research products globally, including its leading dark pool POSIT, algorithms, smart routing, and TCA tools, and sees opportunities to expand in new asset classes and markets internationally.
Viewers also liked (20)
Georeferenciar Imágenes (GOOGLE EARTH, raster desing Y CIVIL 3D 2015,2016)
Georeferenciar Imágenes (GOOGLE EARTH, raster desing Y CIVIL 3D 2015,2016)
Similar to 0629venmoplus
Qingpeng zhang week5
This document introduces VenmoPlus.com, a service that allows users to explore their Venmo payment network. It discusses some of the challenges in building the system, including calculating graph distances and querying connections in real time for a network with 5 million users and over 88 million payment transactions. The solution uses two databases - Redis to store the graph data for fast updates and traversal, and Elasticsearch for storing transaction data and full-text search capabilities. Spark is used to distribute the ETL processing between these databases.
Introducing VenmoPlus.com 6/27 version
project for the Insight Data Engineering Program
venmoplus.com
https://github.com/qingpeng/VenmoPlus
Qingpeng zhang 0711
This document describes VenmoPlus.com, a service that builds on top of Venmo transaction and social network data. It features functions like searching usernames, labeling relationships between users, friend recommendations, searching transactions within a user's friend circle, and listing friends. The system uses Redis to store the social graph and Elasticsearch for all other data. Algorithms like breadth-first search and bidirectional search are used to find distances between users in the dynamic graph. Query optimizations improve performance, such as removing unnecessary data from Elasticsearch.
Qingpeng zhang 0713
This document introduces VenmoPlus.com, a service that allows users to explore their Venmo network. It provides features such as fuzzy username searching, labeling relationships between payers and receivers, friend recommendations, searching transactions within a friend circle, and listing friends. It uses Redis to store the graph structure and Elasticsearch to store all other data. Algorithms like breadth-first search and bidirectional search are used to find the distance between nodes in the dynamic graph in real time. The system employs a two database approach with Redis and Elasticsearch to support these features efficiently.
VenmoPlus
VenmoPlus.com is a service that allows users to explore their Venmo network through additional features like fuzzy username searching, labeling relationships between payers and receivers, friend recommendations, searching transactions within a friend circle, and listing friends. It uses Redis to store the graph structure and Elasticsearch to store all other data. Algorithms like breadth-first search and bidirectional search are used to find distances between nodes in the dynamic graph in real time. The system is optimized through the use of two databases, algorithm design, and query optimizations to support features like searching relationships and transactions within a user's friend circle.
VenmoPlus0708
VenmoPlus.com is a service that allows users to explore their Venmo network through additional features like fuzzy username searching, labeling relationships between payers and receivers, friend recommendations, searching transactions within a friend circle, and listing friends. It uses Redis to store the graph structure and Elasticsearch to store all other data. Algorithms like bidirectional breadth-first search and optimizations are used to enable real-time queries of the dynamic graph. The service is built using a pipeline of raw Venmo data processed in a distributed manner across multiple databases and cloud resources.
Graph Analytics with ArangoDB
This document discusses using graphs and graph databases for machine learning. It provides an overview of graph analytics algorithms that can be used to solve problems with graph data, including recommendations, fraud detection, and network analysis. It also discusses using graph embeddings and graph neural networks for tasks like node classification and link prediction. Finally, it discusses how graphs can be used for machine learning infrastructure and metadata tasks like data provenance, audit trails, and privacy.
Data Science as Scale
The document provides an introduction to data science at scale and distributed thinking. It discusses the motivation for data science at scale due to increasing data volumes, varieties, and velocities. It distinguishes between data science, which focuses on accuracy, and data engineering, which focuses on scale, performance, and reliability. The document then provides a crash course on data engineering concepts like distributed computation and the SMACK stack. It introduces Spark as a framework that can scale data processing. Finally, it discusses probabilistic algorithms as an approach for processing large datasets that may be inexact but use less resources than exact algorithms.
Data Discovery and Metadata
Talk on Data Discovery and Metadata by Mark Grover from July 2019.
Goes into detail of the problem, build/buy/adopt analysis and Lyft's solution - Amundsen, along with thoughts on the future.
Open Analytics Environment
I summarize requirements for an "Open Analytics Environment" (aka "the Cauldron"), and some work being performed at the University of Chicago and Argonne National Laboratory towards its realization.
Data platform architecture principles - ieee infrastructure 2020
This document discusses principles for building a healthy data platform, including:
1. Establishing explicit contracts between teams to define dependencies and service level agreements.
2. Abstracting the data platform into services for ingesting, storing, and processing data in motion and at rest.
3. Enabling observability of data pipelines through metadata collection and integration with tools like Marquez to provide lineage, availability, and change management visibility.
Democratizing Data within your organization - Data Discovery
n this talk, we talk about the challenges at scale in an organization like Lyft. We delve into data discovery as a challenge towards democratizing data within your organization. And, go in detail about the solution to solve the challenge of data discovery.
Graph Gurus Episode 1: Enterprise Graph
This document summarizes an introductory webinar on building an enterprise knowledge graph from RDF data using TigerGraph. It introduces RDF and knowledge graphs, demonstrates loading DBpedia data into a TigerGraph graph database using a universal schema, and provides examples of queries to extract information from the graph such as related people, publishers by location, and related topics for a given predicate. The webinar encourages attendees to learn more about graph databases and TigerGraph through additional resources and future webinar episodes.
Lambda Architecture and open source technology stack for real time big data
The document discusses the Lambda Architecture, which is an approach for building data systems to handle large volumes of real-time streaming data. It proposes using three main design principles: handling human errors by making the system fault-tolerant, storing raw immutable data, and enabling recomputation of results from the raw data. The document then provides two case studies of applying Lambda Architecture principles to analyze mobile app usage data and process high-volume web logs in real-time. It concludes with lessons learned, such as studying Lambda concepts, collecting any available data, and turning data into useful insights.
Big data real time architectures
Big data real time architectures -
How do to big data processing in real time?
What architectures are out there to support this paradigm?
Which one should we choose?
What Advantages / Pitfalls they contain.
Clickstream data with spark
The document describes analyzing clickstream data using Spark clustering algorithms. It discusses parsing raw clickstream data containing user agent strings to extract useful fields like device type, OS, and timezone. It then applies distributed K-modes clustering in Spark to group users into subsets exhibiting similar patterns. The results show 10 clusters with different proportions of country, timezone, device and other fields, revealing distinct user types within the dataset.
GraphGen: Conducting Graph Analytics over Relational Databases
Applying graph analytics on data stored in relational databases can provide tremendous value in many application domains. We discuss the importance of leveraging these analyses, and the challenges in enabling them. We present a tool, called GraphGen, that allows users to visually explore, and rapidly analyze (using NetworkX) different graph structures present in their databases.
GraphGen: Conducting Graph Analytics over Relational Databases
This document discusses GraphGen, a tool for conducting graph analytics over relational databases. It begins by introducing graph analytics and its applications. It then discusses the current state of graph analytics, which is fragmented with no single solution. Most organizations store data relationally and have "hidden" graphs that can be extracted. GraphGen provides a declarative language to define nodes and edges to extract these graphs without ETL. It supports various interfaces like Java, Python, and a web application to enable graph analytics over relational data in an intuitive way.
SDSC18 and DSATL Meetup March 2018
This presentation was given in one of the DSATL Mettups in March 2018 in partnership with Southern Data Science Conference 2018 (www.southerndatascience.com)
Pydata talk
This document provides an overview of SFrame, a scalable dataframe for machine learning developed by Dato. SFrame was created to handle large datasets and enable fast machine learning. It uses a columnar storage format and lazy evaluation to optimize performance. SFrame can handle datasets with billions of rows and columns efficiently using its out-of-core design. It also includes an SGraph extension to handle graph analytics on very large graphs with billions of edges. A variety of machine learning algorithms are built on SFrame to leverage its scalability.
Similar to 0629venmoplus (20)
Data platform architecture principles - ieee infrastructure 2020
Data platform architecture principles - ieee infrastructure 2020
Democratizing Data within your organization - Data Discovery
Democratizing Data within your organization - Data Discovery
Lambda Architecture and open source technology stack for real time big data
Lambda Architecture and open source technology stack for real time big data
GraphGen: Conducting Graph Analytics over Relational Databases
GraphGen: Conducting Graph Analytics over Relational Databases
GraphGen: Conducting Graph Analytics over Relational Databases
GraphGen: Conducting Graph Analytics over Relational Databases
Recently uploaded
Digital Marketing with a Focus on Sustainability
Digital Marketing best practices including influencer marketing, content creators, and omnichannel marketing for Sustainable Brands at the Sustainable Cosmetics Summit 2024 in New York
Best practices for project execution and delivery
A select set of project management best practices to keep your project on-track, on-cost and aligned to scope. Many firms have don't have the necessary skills, diligence, methods and oversight of their projects; this leads to slippage, higher costs and longer timeframes. Often firms have a history of projects that simply failed to move the needle. These best practices will help your firm avoid these pitfalls but they require fortitude to apply.
The Genesis of BriansClub.cm Famous Dark WEb Platform
BriansClub.cm, a famous platform on the dark web, has become one of the most infamous carding marketplaces, specializing in the sale of stolen credit card data.
Understanding User Needs and Satisfying Them
https://www.productmanagementtoday.com/frs/26903918/understanding-user-needs-and-satisfying-them
We know we want to create products which our customers find to be valuable. Whether we label it as customer-centric or product-led depends on how long we've been doing product management. There are three challenges we face when doing this. The obvious challenge is figuring out what our users need; the non-obvious challenges are in creating a shared understanding of those needs and in sensing if what we're doing is meeting those needs.
In this webinar, we won't focus on the research methods for discovering user-needs. We will focus on synthesis of the needs we discover, communication and alignment tools, and how we operationalize addressing those needs.
Industry expert Scott Sehlhorst will:
• Introduce a taxonomy for user goals with real world examples
• Present the Onion Diagram, a tool for contextualizing task-level goals
• Illustrate how customer journey maps capture activity-level and task-level goals
• Demonstrate the best approach to selection and prioritization of user-goals to address
• Highlight the crucial benchmarks, observable changes, in ensuring fulfillment of customer needs
Structural Design Process: Step-by-Step Guide for Buildings
The structural design process is explained: Follow our step-by-step guide to understand building design intricacies and ensure structural integrity. Learn how to build wonderful buildings with the help of our detailed information. Learn how to create structures with durability and reliability and also gain insights on ways of managing structures.
Creative Web Design Company in Singapore
At Techbox Square, in Singapore, we're not just creative web designers and developers, we're the driving force behind your brand identity. Contact us today.
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Know what your zodiac sign says about your taste in food! Explore how the 12 zodiac signs influence your culinary preferences with insights from MyPandit. Dive into astrology and flavors!
-- June 2024 is National Volunteer Month --
Check out our June display of books on voluntary organisations
Business storytelling: key ingredients to a story
Storytelling is an incredibly valuable tool to share data and information. To get the most impact from stories there are a number of key ingredients. These are based on science and human nature. Using these elements in a story you can deliver information impactfully, ensure action and drive change.
How to Implement a Real Estate CRM Software
To implement a CRM for real estate, set clear goals, choose a CRM with key real estate features, and customize it to your needs. Migrate your data, train your team, and use automation to save time. Monitor performance, ensure data security, and use the CRM to enhance marketing. Regularly check its effectiveness to improve your business.
Creative Web Design Company in Singapore
At Techbox Square, in Singapore, we're not just creative web designers and developers, we're the driving force behind your brand identity. Contact us today.
Hamster Kombat' Telegram Game Surpasses 100 Million Players—Token Release Sch...
Hamster Kombat' Telegram Game Surpasses 100 Million Players—Token Release Schedule Unveiled
How MJ Global Leads the Packaging Industry.pdf
MJ Global's success in staying ahead of the curve in the packaging industry is a testament to its dedication to innovation, sustainability, and customer-centricity. By embracing technological advancements, leading in eco-friendly solutions, collaborating with industry leaders, and adapting to evolving consumer preferences, MJ Global continues to set new standards in the packaging sector.
Digital Transformation Frameworks: Driving Digital Excellence
[To download this presentation, visit:
https://www.oeconsulting.com.sg/training-presentations]
This presentation is a curated compilation of PowerPoint diagrams and templates designed to illustrate 20 different digital transformation frameworks and models. These frameworks are based on recent industry trends and best practices, ensuring that the content remains relevant and up-to-date.
Key highlights include Microsoft's Digital Transformation Framework, which focuses on driving innovation and efficiency, and McKinsey's Ten Guiding Principles, which provide strategic insights for successful digital transformation. Additionally, Forrester's framework emphasizes enhancing customer experiences and modernizing IT infrastructure, while IDC's MaturityScape helps assess and develop organizational digital maturity. MIT's framework explores cutting-edge strategies for achieving digital success.
These materials are perfect for enhancing your business or classroom presentations, offering visual aids to supplement your insights. Please note that while comprehensive, these slides are intended as supplementary resources and may not be complete for standalone instructional purposes.
Frameworks/Models included:
Microsoft’s Digital Transformation Framework
McKinsey’s Ten Guiding Principles of Digital Transformation
Forrester’s Digital Transformation Framework
IDC’s Digital Transformation MaturityScape
MIT’s Digital Transformation Framework
Gartner’s Digital Transformation Framework
Accenture’s Digital Strategy & Enterprise Frameworks
Deloitte’s Digital Industrial Transformation Framework
Capgemini’s Digital Transformation Framework
PwC’s Digital Transformation Framework
Cisco’s Digital Transformation Framework
Cognizant’s Digital Transformation Framework
DXC Technology’s Digital Transformation Framework
The BCG Strategy Palette
McKinsey’s Digital Transformation Framework
Digital Transformation Compass
Four Levels of Digital Maturity
Design Thinking Framework
Business Model Canvas
Customer Journey Map
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
On episode 272 of the Digital and Social Media Sports Podcast, Neil chatted with Brian Fitzsimmons, Director of Licensing and Business Development for Barstool Sports.
What follows is a collection of snippets from the podcast. To hear the full interview and more, check out the podcast on all podcast platforms and at www.dsmsports.net
Innovation Management Frameworks: Your Guide to Creativity & Innovation
Innovation Management Frameworks: Your Guide to Creativity & InnovationOperational Excellence Consulting
[To download this presentation, visit:
https://www.oeconsulting.com.sg/training-presentations]
This PowerPoint compilation offers a comprehensive overview of 20 leading innovation management frameworks and methodologies, selected for their broad applicability across various industries and organizational contexts. These frameworks are valuable resources for a wide range of users, including business professionals, educators, and consultants.
Each framework is presented with visually engaging diagrams and templates, ensuring the content is both informative and appealing. While this compilation is thorough, please note that the slides are intended as supplementary resources and may not be sufficient for standalone instructional purposes.
This compilation is ideal for anyone looking to enhance their understanding of innovation management and drive meaningful change within their organization. Whether you aim to improve product development processes, enhance customer experiences, or drive digital transformation, these frameworks offer valuable insights and tools to help you achieve your goals.
INCLUDED FRAMEWORKS/MODELS:
1. Stanford’s Design Thinking
2. IDEO’s Human-Centered Design
3. Strategyzer’s Business Model Innovation
4. Lean Startup Methodology
5. Agile Innovation Framework
6. Doblin’s Ten Types of Innovation
7. McKinsey’s Three Horizons of Growth
8. Customer Journey Map
9. Christensen’s Disruptive Innovation Theory
10. Blue Ocean Strategy
11. Strategyn’s Jobs-To-Be-Done (JTBD) Framework with Job Map
12. Design Sprint Framework
13. The Double Diamond
14. Lean Six Sigma DMAIC
15. TRIZ Problem-Solving Framework
16. Edward de Bono’s Six Thinking Hats
17. Stage-Gate Model
18. Toyota’s Six Steps of Kaizen
19. Microsoft’s Digital Transformation Framework
20. Design for Six Sigma (DFSS)
To download this presentation, visit:
https://www.oeconsulting.com.sg/training-presentationsAnny Serafina Love - Letter of Recommendation by Kellen Harkins, MS.
This letter, written by Kellen Harkins, Course Director at Full Sail University, commends Anny Love's exemplary performance in the Video Sharing Platforms class. It highlights her dedication, willingness to challenge herself, and exceptional skills in production, editing, and marketing across various video platforms like YouTube, TikTok, and Instagram.
Building Your Employer Brand with Social Media
Presented at The Global HR Summit, 6th June 2024
In this keynote, Luan Wise will provide invaluable insights to elevate your employer brand on social media platforms including LinkedIn, Facebook, Instagram, X (formerly Twitter) and TikTok. You'll learn how compelling content can authentically showcase your company culture, values, and employee experiences to support your talent acquisition and retention objectives. Additionally, you'll understand the power of employee advocacy to amplify reach and engagement – helping to position your organization as an employer of choice in today's competitive talent landscape.
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart➒➌➎➏➑➐➋➑➐➐Dpboss Matka Guessing Satta Matka Kalyan Chart Indian Matka
SATTA MATKA SATTA FAST RESULT KALYAN TOP MATKA RESULT KALYAN SATTA MATKA FAST RESULT MILAN RATAN RAJDHANI MAIN BAZAR MATKA FAST TIPS RESULT MATKA CHART JODI CHART PANEL CHART FREE FIX GAME SATTAMATKA ! MATKA MOBI SATTA 143 spboss.in TOP NO1 RESULT FULL RATE MATKA ONLINE GAME PLAY BY APP SPBOSSRecently uploaded (20)
The Genesis of BriansClub.cm Famous Dark WEb Platform
The Genesis of BriansClub.cm Famous Dark WEb Platform
Structural Design Process: Step-by-Step Guide for Buildings
Structural Design Process: Step-by-Step Guide for Buildings
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Hamster Kombat' Telegram Game Surpasses 100 Million Players—Token Release Sch...
Hamster Kombat' Telegram Game Surpasses 100 Million Players—Token Release Sch...
Digital Transformation Frameworks: Driving Digital Excellence
Digital Transformation Frameworks: Driving Digital Excellence
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
Innovation Management Frameworks: Your Guide to Creativity & Innovation
Innovation Management Frameworks: Your Guide to Creativity & Innovation
Anny Serafina Love - Letter of Recommendation by Kellen Harkins, MS.
Anny Serafina Love - Letter of Recommendation by Kellen Harkins, MS.
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
0629venmoplus
- 1. Introducing VenmoPlus.com -Explore your Venmo network! Qingpeng “Q.P.” Zhang, Insight Data Engineering Fellow
- 3. 2013 Biggest Challenge: ● Calculate/Query graph distance in real time
- 4. Solutions ● Two databases ● Graph algorithm optimizations ● Query/search optimizations ● S3⇔Redis S3⇔ Elasticsearch distributedly with Spark ● ...
- 5. Historical transactions Real time transactions A Tale of Two Databases
- 6. Two Databases 420890 Graham Hadley 1630476 Leon Tang 810029 Harminder Toor 1371353 Ephraim Park 562884 Paul Min 420890 set(14935158, 562884) 1630476 set(1371353) 810029 set(190230,14935158) 1371353 set(810029,971156) 562884 set(196371,1371353)
- 9. About Me ● Postdoc in Lawrence Berkeley National Lab ● PhD in Computer Science, Michigan State Certified Volunteers: ● Software Carpentry ● Data Carpentry ● American Red Cross Christmas Eve 2014, ice storm, Michigan
- 10. Algorithm 1 Shortest distance -> intersection of sets (friend lists) ● 1st degree friends of A ∩ 1st degree friends of B == [] ? ● 2nd degree friends of A ∩ 1st degree friends of B == []?
- 11. Algorithm 2 Query distance between vertices in a historic moment in a constantly changing graph (because we don’t pre-calculate the distance….) ● A recent transaction for a user is history and has changed the graph ● Query distance of the two users at that moment. ○ not considering that specific transaction) ○ Remove the influence of that specific transaction temporarily and restore ■ Test if that transaction is the first between the pair of users.
- 12. Pipeline, raw data, in distributed way
- 13. This, or that? - to build graph
- 14. This, or that? - for fast searching
- 15. Query/Search Optimizations 1. Remove aggregation for better performance… (trade-off) 2. Friend recommender: a. Using Counter to get only 5 users with the most common friends 3. Search message in friend circle a. Combine query of Elasticsearch and Redis
- 17. Lesson learned
- 20. ● Cache of 2nd degree friends list ● Partitioned GraphDB ● Good for Linkedin (hundreds of million users, with higher degree) ● 5 million vertices (users) ● 32 million distinct edges (transactions) ● 88 million total edges (transactions)
- 21. ● Cache of 2nd degree friends list ● Partitioned GraphDB ● Good for Linkedin (hundreds of million users, with higher degree) ● 5 million vertices (users) ● 32 million distinct edges (transactions) ● 88 million total edges (transactions) No cache (precalculation)? No GraphDB?