Truecaller towards a data-driven company
•
1 like•1,501 views
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog Marek Wiewiórka and Tomasz Żukowski had a pleasure to give a presentation at Data Science Summit 2017 which took place in Warsaw on May 26th. They came forth with one of the most popular startup in Sweden - Trucaller. Please take a look at the presentation.
Report
Share
Report
Share
Download to read offline
Recommended
Assisting millions of active users in real-time - Alexey Brodovshuk, Kcell; K...
Nowadays many companies become data rich and intensive. They have millions of users generating billions of interactions and events per day.
These massive streams of complex events can be processed and reacted upon to e.g. offer new products, next best actions, communicate to users or detect frauds, and quicker we can do it, the higher value we can generate.
In this talk we will present, how in joint development with our client and in just few months effort we have built from ground up a complex event processing platform for their intensive data streams. We will share how the system runs marketing campaigns or detect frauds by following behavior of millions users in real-time and reacting on it instantly. The platform designed and built with Big Data technologies to infinitely and cost-effectively scale already ingests and processes billions of messages or terabytes of data per day on a still small cluster. We will share how we leveraged the current best of breed open-source projects including Apache Flink, Apache Nifi and Apache Kafka, but also what interesting problems we needed to solve. Finally, we will share where we’re heading next, what next use cases we’re going to implement and how.
Ray: Enterprise-Grade, Distributed Python
Ray (ray.io) is an open-source, distributed framework from U.C. Berkeley’s RISELab that easily scales Python applications from a laptop to a cluster.
Model serving made easy using Kedro pipelines - Mariusz Strzelecki, GetInData
If you want to stay up to date, subscribe to our newsletter here: https://bit.ly/3tiw1I8
Presentation from the performance given by Mariusz during the Data Science Summit ML Edition.
Author: Mariusz Strzelecki
Linkedin: https://www.linkedin.com/in/mariusz-strzelecki/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Automated Deployments with Ansible
This presentation starts with an introduction to the rationale behind automated deployments in Continuous Delivery and DevOps. Then, I compare agent-based architectures, such as Chef and Puppet with the agentless architecture of the server orchestration engine Ansible. The presentation concludes with an automated deployment of Dynatrace into a simulated production environment.
AIOps - The next 5 years
AIOps is becoming imperative to the management of today’s complex IT systems and their ability to support changing business conditions. This slide explains the role that AIOps can and will play in the enterprise of the future, how the scope of AIOps platforms will expand, and what new functionality may be deployed.
Watch the webinar here. https://www.moogsoft.com/resources/aiops/webinar/aiops-the-next-five-years
Product management class rookie to pro
Building a Product? the knowledge you will acquire will help with product management and the use of agile scrum to build products. The training provides fundamental guide to building the best solution in the world with some of the best tips, templates and guides in terms of leading trends. This will bring your IDEAS to Live.
Snowflake: The Good, the Bad, and the Ugly
Learn how to solve the top 3 challenges Snowflake customers face, and what you can do to ensure high-performance, intelligent analytics at any scale. Ideal for those currently using Snowflake and those considering it. Learn more at: https://kyligence.io/
Recommended
Assisting millions of active users in real-time - Alexey Brodovshuk, Kcell; K...
Nowadays many companies become data rich and intensive. They have millions of users generating billions of interactions and events per day.
These massive streams of complex events can be processed and reacted upon to e.g. offer new products, next best actions, communicate to users or detect frauds, and quicker we can do it, the higher value we can generate.
In this talk we will present, how in joint development with our client and in just few months effort we have built from ground up a complex event processing platform for their intensive data streams. We will share how the system runs marketing campaigns or detect frauds by following behavior of millions users in real-time and reacting on it instantly. The platform designed and built with Big Data technologies to infinitely and cost-effectively scale already ingests and processes billions of messages or terabytes of data per day on a still small cluster. We will share how we leveraged the current best of breed open-source projects including Apache Flink, Apache Nifi and Apache Kafka, but also what interesting problems we needed to solve. Finally, we will share where we’re heading next, what next use cases we’re going to implement and how.
Ray: Enterprise-Grade, Distributed Python
Ray (ray.io) is an open-source, distributed framework from U.C. Berkeley’s RISELab that easily scales Python applications from a laptop to a cluster.
Model serving made easy using Kedro pipelines - Mariusz Strzelecki, GetInData
If you want to stay up to date, subscribe to our newsletter here: https://bit.ly/3tiw1I8
Presentation from the performance given by Mariusz during the Data Science Summit ML Edition.
Author: Mariusz Strzelecki
Linkedin: https://www.linkedin.com/in/mariusz-strzelecki/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Automated Deployments with Ansible
This presentation starts with an introduction to the rationale behind automated deployments in Continuous Delivery and DevOps. Then, I compare agent-based architectures, such as Chef and Puppet with the agentless architecture of the server orchestration engine Ansible. The presentation concludes with an automated deployment of Dynatrace into a simulated production environment.
AIOps - The next 5 years
AIOps is becoming imperative to the management of today’s complex IT systems and their ability to support changing business conditions. This slide explains the role that AIOps can and will play in the enterprise of the future, how the scope of AIOps platforms will expand, and what new functionality may be deployed.
Watch the webinar here. https://www.moogsoft.com/resources/aiops/webinar/aiops-the-next-five-years
Product management class rookie to pro
Building a Product? the knowledge you will acquire will help with product management and the use of agile scrum to build products. The training provides fundamental guide to building the best solution in the world with some of the best tips, templates and guides in terms of leading trends. This will bring your IDEAS to Live.
Snowflake: The Good, the Bad, and the Ugly
Learn how to solve the top 3 challenges Snowflake customers face, and what you can do to ensure high-performance, intelligent analytics at any scale. Ideal for those currently using Snowflake and those considering it. Learn more at: https://kyligence.io/
Which Application Modernization Pattern Is Right For You?
App modernization projects are hard. Enterprises are looking to cloud-native platforms like Pivotal Cloud Foundry to run their applications, but they’re worried about the risks inherent to any replatforming effort.
Fortunately, several repeatable patterns of successful incremental migration have emerged.
In this webcast, Google Cloud’s Prithpal Bhogill and Pivotal’s Shaun Anderson will discuss best practices for app modernization and securely and seamlessly routing traffic between legacy stacks and Pivotal Cloud Foundry.
Bring Satellite and Drone Imagery into your Data Science Workflows
Overhead imagery from satellites and drones have entered the mainstream of how we explore, understand, and tell stories about our world.
Machine Learning and AI
The breath and depth of Azure products that fall under the AI and ML umbrella can be difficult to follow. In this presentation I’ll first define exactly what AI, ML, and deep learning is, and then go over the various Microsoft AI and ML products and their use cases.
SRE vs DevOps
In this presentation I will speak how are the SRE and DevOps, what is a reliability. Also about the reliability approach in Competitive Gaming in Wargaming and show a few cases.
KNIME Software Overview
Are you curious about KNIME Software?
Do you know the difference between KNIME Analytics Platform and KNIME Server?
Which data sources can KNIME connect to?
Can you run an R script from within a KNIME workflow? A Python script? Which other integrations are available?
How can KNIME help with ETL, data preparation, and general data manipulation? Which machine learning algorithms can KNIME offer?
This webinar answers all of these questions! There’s also information about connecting to big data clusters and how you can run the whole or part of your analysis on a big data platform. It also covers everything you need to know about Microsoft Azure and Amazon AWS
Enterprise cloud strategy - PDF
Lead the journey to the cloud and drive innovation! Discover what makes the cloud so compelling to enterprises; with which applications you should start your cloud journey; how your organization will change, and how skill sets will evolve; how to measure progress; how to think about security, compliance, and business buy-in; and how to exploit the ever-growing feature set that the cloud offers to gain strategic and competitive advantage.
Ursa Labs and Apache Arrow in 2019
Update on Apache Arrow project and not-for-profit Ursa Labs org for 2019 https://ursalabs.org/. Active projects and development objectives
Preparing for Growth - Architecting Giant Apps for Scalability and Build Speed
SwiftHeroes 2021 Bruno Rocha
Introduction to Spark with Python
The presentation I gave at Big Talk event #2, in Istanbul on April 2018.
How to See and Resolve Office 365 Performance Challenges
Take control of your Office 365 deployments with ThousandEyes and learn how to monitor, detect and resolve performance issues. Find out:
- What the common misconceptions about Office 365 and how they affect performance
- Monitoring best practices for readiness, deployment and operational phases of your Office 365 rollout
- Real-world case studies explained through a demo
Dataiku & Snowflake Meetup Berlin 2020
From the Data Work Out event:
Performant and scalable Data Science with Dataiku DSS and Snowflake
Managing the whole process of setting up a machine learning environment from end-to-end becomes significantly easier when using cloud-based technologies. The ability to provision infrastructure on demand (IaaS) solves the problem of manually requesting virtual machines. It also provides immediate access to compute resources whenever they are needed. But that still leaves the administrative overhead of managing the ML software and the platform to store and manage the data.
A fully managed end-to-end machine learning platform like Dataiku Data Science Studio (DSS) that enables data scientists, machine learning experts, and even business users to quickly build, train and host machine learning models at scale, needs to access data from many different sources and can also access data provided by Snowflake. Storing data in Snowflake has three significant advantages: a single source of truth, shorten the data preparation cycle, scale as you go.
Dominant resource fairness fair allocation of multiple resource types
DRF algorithm provides fair resource allocation in a system containing different resource types. Dominant Resource Fairness (DRF), is a generalization of max-min fairness to multiple resource types. This algorithm is used in resource allocation of hadoop cluster.
OpenTelemetry For Operators
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the latest concepts in observability and then demonstrates how to configure and deploy various OpenTelemetry components to effectively meet your SLO's.
Predicting Startup Market Trends based on the news and social media - Albert ...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
Nowadays, one tweet can have impact on the value of the company or cryptocurrency. It becomes important for companies to be able to know everything what's happening in the market, especially for startups or when entering the new market. The presentation is about presenting the complex platform used for creating and verifying the strategy for a startup from the Wellbeing market. We go through web scraping-based data ingestion to ElasticSearch, NLP pipelines to understand what people write and what is the possible future of each market predicted by PySpark job.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Real time analytics that controls 50% of mobile network in Poland - Maciej Br...
Did you like it? Check out more about this presentation here: https://www.youtube.com/watch?v=SW_GqhQJd8U&t=14s
The ability to analyze data in real time for mobile network is crucial for diagnostics and ensuring the quality of the service for end customers. To achieve this we have built a real-time ingestion and analytics platform that processes 2.2 billions messages a day from mobile networks hardware. During the talk we will show how we used Flink and Flink SQL to build this platform. The solution includes calculation of more than 5000 KPIs and 1500 aggregation defined in SQL, on 750 Kafka topics. We will describe how we manage Flink jobs at scale using Ververica and Kubernetes, how we monitor the platform using Clickhouse and what problems we need to overcome in the project.
Watch our webinar here: https://www.youtube.com/watch?v=SW_GqhQJd8U&t=115s
Speakers:
Maciej Bryński - https://www.linkedin.com/in/brynski/
Rafał Małanij
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
More Related Content
What's hot
Which Application Modernization Pattern Is Right For You?
App modernization projects are hard. Enterprises are looking to cloud-native platforms like Pivotal Cloud Foundry to run their applications, but they’re worried about the risks inherent to any replatforming effort.
Fortunately, several repeatable patterns of successful incremental migration have emerged.
In this webcast, Google Cloud’s Prithpal Bhogill and Pivotal’s Shaun Anderson will discuss best practices for app modernization and securely and seamlessly routing traffic between legacy stacks and Pivotal Cloud Foundry.
Bring Satellite and Drone Imagery into your Data Science Workflows
Overhead imagery from satellites and drones have entered the mainstream of how we explore, understand, and tell stories about our world.
Machine Learning and AI
The breath and depth of Azure products that fall under the AI and ML umbrella can be difficult to follow. In this presentation I’ll first define exactly what AI, ML, and deep learning is, and then go over the various Microsoft AI and ML products and their use cases.
SRE vs DevOps
In this presentation I will speak how are the SRE and DevOps, what is a reliability. Also about the reliability approach in Competitive Gaming in Wargaming and show a few cases.
KNIME Software Overview
Are you curious about KNIME Software?
Do you know the difference between KNIME Analytics Platform and KNIME Server?
Which data sources can KNIME connect to?
Can you run an R script from within a KNIME workflow? A Python script? Which other integrations are available?
How can KNIME help with ETL, data preparation, and general data manipulation? Which machine learning algorithms can KNIME offer?
This webinar answers all of these questions! There’s also information about connecting to big data clusters and how you can run the whole or part of your analysis on a big data platform. It also covers everything you need to know about Microsoft Azure and Amazon AWS
Enterprise cloud strategy - PDF
Lead the journey to the cloud and drive innovation! Discover what makes the cloud so compelling to enterprises; with which applications you should start your cloud journey; how your organization will change, and how skill sets will evolve; how to measure progress; how to think about security, compliance, and business buy-in; and how to exploit the ever-growing feature set that the cloud offers to gain strategic and competitive advantage.
Ursa Labs and Apache Arrow in 2019
Update on Apache Arrow project and not-for-profit Ursa Labs org for 2019 https://ursalabs.org/. Active projects and development objectives
Preparing for Growth - Architecting Giant Apps for Scalability and Build Speed
SwiftHeroes 2021 Bruno Rocha
Introduction to Spark with Python
The presentation I gave at Big Talk event #2, in Istanbul on April 2018.
How to See and Resolve Office 365 Performance Challenges
Take control of your Office 365 deployments with ThousandEyes and learn how to monitor, detect and resolve performance issues. Find out:
- What the common misconceptions about Office 365 and how they affect performance
- Monitoring best practices for readiness, deployment and operational phases of your Office 365 rollout
- Real-world case studies explained through a demo
Dataiku & Snowflake Meetup Berlin 2020
From the Data Work Out event:
Performant and scalable Data Science with Dataiku DSS and Snowflake
Managing the whole process of setting up a machine learning environment from end-to-end becomes significantly easier when using cloud-based technologies. The ability to provision infrastructure on demand (IaaS) solves the problem of manually requesting virtual machines. It also provides immediate access to compute resources whenever they are needed. But that still leaves the administrative overhead of managing the ML software and the platform to store and manage the data.
A fully managed end-to-end machine learning platform like Dataiku Data Science Studio (DSS) that enables data scientists, machine learning experts, and even business users to quickly build, train and host machine learning models at scale, needs to access data from many different sources and can also access data provided by Snowflake. Storing data in Snowflake has three significant advantages: a single source of truth, shorten the data preparation cycle, scale as you go.
Dominant resource fairness fair allocation of multiple resource types
DRF algorithm provides fair resource allocation in a system containing different resource types. Dominant Resource Fairness (DRF), is a generalization of max-min fairness to multiple resource types. This algorithm is used in resource allocation of hadoop cluster.
OpenTelemetry For Operators
The monolith to cloud-native, microservices evolution has driven a shift from monitoring to observability. OpenTelemetry, a merger of the OpenTracing and OpenCensus projects, is enabling Observability 2.0. This talk covers the latest concepts in observability and then demonstrates how to configure and deploy various OpenTelemetry components to effectively meet your SLO's.
What's hot (20)
Which Application Modernization Pattern Is Right For You?
Which Application Modernization Pattern Is Right For You?
Bring Satellite and Drone Imagery into your Data Science Workflows
Bring Satellite and Drone Imagery into your Data Science Workflows
Preparing for Growth - Architecting Giant Apps for Scalability and Build Speed
Preparing for Growth - Architecting Giant Apps for Scalability and Build Speed
How to See and Resolve Office 365 Performance Challenges
How to See and Resolve Office 365 Performance Challenges
Dominant resource fairness fair allocation of multiple resource types
Dominant resource fairness fair allocation of multiple resource types
Similar to Truecaller towards a data-driven company
Predicting Startup Market Trends based on the news and social media - Albert ...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
Nowadays, one tweet can have impact on the value of the company or cryptocurrency. It becomes important for companies to be able to know everything what's happening in the market, especially for startups or when entering the new market. The presentation is about presenting the complex platform used for creating and verifying the strategy for a startup from the Wellbeing market. We go through web scraping-based data ingestion to ElasticSearch, NLP pipelines to understand what people write and what is the possible future of each market predicted by PySpark job.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Real time analytics that controls 50% of mobile network in Poland - Maciej Br...
Did you like it? Check out more about this presentation here: https://www.youtube.com/watch?v=SW_GqhQJd8U&t=14s
The ability to analyze data in real time for mobile network is crucial for diagnostics and ensuring the quality of the service for end customers. To achieve this we have built a real-time ingestion and analytics platform that processes 2.2 billions messages a day from mobile networks hardware. During the talk we will show how we used Flink and Flink SQL to build this platform. The solution includes calculation of more than 5000 KPIs and 1500 aggregation defined in SQL, on 750 Kafka topics. We will describe how we manage Flink jobs at scale using Ververica and Kubernetes, how we monitor the platform using Clickhouse and what problems we need to overcome in the project.
Watch our webinar here: https://www.youtube.com/watch?v=SW_GqhQJd8U&t=115s
Speakers:
Maciej Bryński - https://www.linkedin.com/in/brynski/
Rafał Małanij
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Data-Driven Fast Track: Introduction to data-drivenness with Piotr Menclewicz
Watch video here: https://youtu.be/sfowpU90zFM
Piotr's presentation about GetInData’s Data-Driven Fast Track, the 3-step framework for data transformation.
You will learn:
➡ How to assess how data-driven your company is
➡ How to generate ideas for new initiatives to push your company towards better decisions
➡ How to think about implementing these initiatives to increase your chances of success
If you miss it live don't despair. Watch the video and feel free to diagnose your company by filling out the survey prepared by our team here: https://bit.ly/3fKcRrb! After completing the survey, you will receive a tailored summary report with insights from one of our experts.
Below you'll find links to all the materials mentioned in the workshop needed for exercises.
LINKS TO MATERIALS ABOUT DATA-DRIVEN:
Data-driven fast-track: 3 steps to make your company more data-driven: https://getindata.com/blog/data-drive...
Is my company data-driven? Here’s how you can find out: https://getindata.com/blog/is-my-comp...
If you:
➡ have questions about webinar topic,
➡ want to talk about your data-driven transformation,
➡ want to become more data-driven and you need consultations,
don't hesitate to write to us: hello@getindata.com
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Master the Multi-Clustered Data Warehouse - Snowflake
Snowflake is one of the most powerful, efficient data warehouses on the market today—and we joined forces with the Snowflake team to show you how it works!
In this webinar:
- Learn how to optimize Snowflake
- Hear insider tips and tricks on how to improve performance
- Get expert insights from Craig Collier, Technical Architect from Snowflake, and Kalyan Arangam, Solution Architect from Matillion
- Find out how leading brands like Converse, Duo Security, and Pets at Home use Snowflake and Matillion ETL to make data-driven decisions
- Discover how Matillion ETL and Snowflake work together to modernize your data world
- Learn how to utilize the impressive scalability of Snowflake and Matillion
Complex event processing platform handling millions of users - Krzysztof Zarz...
If you want to learn more about it, check out our webinar here: https://www.youtube.com/watch?v=EfGPY_NyYQ8&t=77s
The webinar was organized by GetinData on 2020. During the webinar, we shared our lessons learnt from building and running stream processing platform in production for over 2 years.
Watch more here: https://www.youtube.com/watch?v=EfGPY_NyYQ8
Author: Krzysztof Zarzycki
Linkedin: https://www.linkedin.com/in/kzarzycki/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Take Action: The New Reality of Data-Driven Business
The Briefing Room with Dr. Robin Bloor and WebAction
Live Webcast on July 23, 2014
Watch the archive:
https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=360d371d3a49ad256942f55350aa0a8b
The waiting used to be the hardest part, but not anymore. Today’s cutting-edge enterprises can seize opportunities faster than ever, thanks to an array of technologies that enable real-time responsiveness across the spectrum of business processes. Early adopters are solving critical business challenges by enabling the rapid-fire design, development and production of very specific applications. Functionality can range from improved customer engagement to dynamic machine-to-machine interactions.
Register for this episode of The Briefing Room to learn from veteran Analyst Dr. Robin Bloor, who will tout a new era in data-driven organizations, and why a data flow architecture will soon be critical for industry leaders. He’ll be briefed by Sami Akbay of WebAction, who will showcase his company’s real-time data management platform, which combines all the component parts needed to access, process and leverage data big and small. He’ll explain how this new approach can provide game-changing power to organizations of all types and sizes.
Visit InsideAnlaysis.com for more information.
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Ravi Pillala, Chief Data Architect & Distinguished Engineer at Intuit
TurboTax is one of the well known consumer software brand which at its peak serves 385K+ concurrent users. In this session, We start with looking at how user behavioral data & tax domain events are captured in real time using the event bus and analyzed to drive real time personalization with various TurboTax data pipelines. We will also look at solutions performing analytics which make use of these events, with the help of Kafka, Apache Flink, Apache Beam, Spark, Amazon S3, Amazon EMR, Redshift, Athena and Amazon lambda functions. Finally, we look at how SageMaker is used to create the TurboTax model to predict if a customer is at risk or needs help.
Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025
Introduction to Tibco Spotfire + Streambase for advanced analytics and data streaminig
There are 250 Database products, are you running the right one?
Slide deck from a webinar from Aerospike featuring Forrester Research
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
https://www.bigdataspain.org/2016/program/thu-big-data-analytics-machine-learning-to-real-time-processing.html
https://www.youtube.com/watch?v=QiQ6LKKvd9k&t=3s&index=19&list=PL6O3g23-p8Tr5eqnIIPdBD_8eE5JBDBik
Streaming Analytics Comparison of Open Source Frameworks, Products, Cloud Ser...
Streaming Analytics Comparison of Open Source Frameworks, Products and Cloud Services. Includes Apache Storm, Flink, Spark, TIBCO, IBM, AWS Kinesis, Striim, Zoomdata, ...
This session discusses the technical concepts of stream processing / streaming analytics and how it is related to big data, mobile, cloud and internet of things. Different use cases such as predictive fault management or fraud detection are used to show and compare alternative frameworks and products for stream processing and streaming analytics.
The focus of the session lies on comparing
- different open source frameworks such as Apache Apex, Apache Flink or Apache Spark Streaming
- engines from software vendors such as IBM InfoSphere Streams, TIBCO StreamBase
- cloud offerings such as AWS Kinesis.
- real time streaming UIs such as Striim, Zoomdata or TIBCO Live Datamart.
Live demos will give the audience a good feeling about how to use these frameworks and tools.
The session will also discuss how stream processing is related to Apache Hadoop frameworks (such as MapReduce, Hive, Pig or Impala) and machine learning (such as R, Spark ML or H2O.ai).
Streaming Analytics - Comparison of Open Source Frameworks and Products
Stream Processing is a concept used to create a high-performance system for rapidly building applications that analyze and act on real-time streaming data. Benefits, amongst others, are faster processing and reaction to real-time complex event streams and the flexibility to quickly adapt to changing business and analytic needs. Big data, cloud, mobile and internet of things are the major drivers for stream processing and streaming analytics.
This session discusses the technical concepts of stream processing and how it is related to big data, mobile, cloud and internet of things. Different use cases such as predictive fault management or fraud detection are used to show and compare alternative frameworks and products for stream processing and streaming analytics.
The audience will understand when to use open source frameworks such as Apache Storm, Apache Spark or Esper, and powerful engines from software vendors such as IBM InfoSphere Streams or TIBCO StreamBase. Live demos will give the audience a good feeling about how to use these frameworks and tools.
The session will also discuss how stream processing is related to Hadoop and statistical analysis with software such as SAS, Apache Spark’s MLlib or R language.
IT and OT Convergence
OpsRamp and Mystic River are joining forces to bring you this interactive webinar. Can we create gateways between IT and OT? IT/OT convergence has been defined as the integration of information technology (IT) systems used for data-centric computing with operational technology (OT) systems used to monitor events, processes and devices and make adjustments in enterprise and industrial operations. But what this will look like and what is the new role of IT operations management? What is changing in IT and OT to align these worlds? This Tech Talk features our partner Mystic River Consulting, a firm with proven, repeatable service delivery methodologies and a proprietary platform that drives transformational IT project results, at a game changing speed. We’ll discuss the convergence of IT and OT and dive into a demonstration to show what’s possible with OpsRamp.
Watch the recording: https://www.brighttalk.com/webcast/17791/416457
Learn more at https://www.opsramp.com
Also, follow us on social media channels to learn about product highlights, news, announcements, events, conferences and more:
Twitter - https://www.twitter.com/OpsRamp
LinkedIn - https://www.linkedin.com/company/opsramp
Facebook - https://www.facebook.com/OpsRampHQ/
Observability at Spotify
Talk #3 at Time Series Estonia #2 by Anastasia Khlebnikova (Senior Data Engineer at Spotify) on Observability at Spotify
Worldpay - Delivering Multi-Tenancy Applications in A Secure Operational Plat...
Worldpay - Delivering Multi-Tenancy Applications in A Secure Operational Plat...DataWorks Summit/Hadoop Summit
In 2015/16 Worldpay deployed it's Enterprise Data Platform - a highly secure cluster used for analysis of over 65 Billion card transactions and the subject of last years Hadoop Summit Keynote in Dublin. A year on and we are now rapidly expanding our platform with true multi-tenancy. For our first tenant we have build and deployed the analytics and reporting for our central platforms. Our second tenant is to deploy 'decision engines' into our core business systems. These allow Worldpay to make decisions derived from machine learning on how we authorise and route payments traffic and how these affect the consumer, merchant and other business partners. We are also developing other tenant for systems management and security. This talk will look at what it means to have truly have a single enterprise data lake and multiple tenants that share that data and look forward to how we will extend the platform in 2017 with Hadoop 3.Data Works Summit Munich 2017 - Worldpay - Multi Tenancy Clusters
A presentation from the Data Works Summit conference in 2017 that looks how Worldpay, a major payments provider, deployed a secure Hadoop cluster to support multiple business cases in a multi-tenancy cluster.
Predictive Analytics and the Industrial Internet of Manufacturing Things with...
Download our special report, IoT Tech for the Manager: http://bit.ly/report1-slideshare
Predictive Analytics and the Industrial Internet of Manufacturing Things as presented at the IoT Inc Business' eighteenth Meetup. See: http://www.iot-inc.com/predictive-analytics-and-the-industrial-internet-of-manufacturing-things-meetup/
In our eighteenth Meetup we have William Sobel, Chief Strategy Officer at System Insights and Chief Architect/Chair TSC at MTConnect Institute presenting “Predictive Analytics and the Industrial Internet of Manufacturing Things”.
Presentation Abstract
The Industrial Internet of Things has been hyped to take manufacturing into a new era; the German Industrie 4.0 initiative, NNMI in the US and 2025 goals in China are all aligned on the target of agile and smart manufacturing. Our current manufacturing systems have not changed much in the last 20 years and we are still using paper and pencil in many of our processes. There are many advanced technologies we can bring to bear today to help us along that path, but we still need to build the foundations to enable these advancements. Manufacturing requires special consideration for an IIOT system; an approach that does not take into consideration the context of the manufacturing process will not be able to transform the data from the equipment and sensors into actionable information. The solution is to build a standards based interoperable platform that allows services to fuse semantic data from multiple sources to provide the foundation for accelerated innovation in smart manufacturing. Will Sobel will discuss how this is a model for the new product and services to come and how this will enable outcome and intent based self-aware manufacturing systems.
HiveMQ & HighByte Presents: Building an Enterprise Unified Namespace (UNS) to...
Did you know that you could build an enterprise-grade unified namespace (UNS) using an MQTT broker and an Industrial DataOps solution? In this webinar, leaders from HiveMQ and HighByte discussed:
The UNS definition and the benefits of building an enterprise UNS for your company or your clients
The advantages of using an MQTT broker UNS architecture
How HighByte Intelligence Hub consumes data from multiple sources, assembles data into logical payloads, and publishes these payloads to the UNS
How HighByte Intelligence Hub subscribes to data in the UNS and publishes it to systems that don’t communicate over MQTT
How to work with topic structures in HiveMQ
The benefits of using HiveMQ to achieve a UNS with a publish-subscribe architecture
How to build an enterprise-grade unified namespace (UNS) using an MQTT broker and an Industrial DataOps solution
If you are not aware of UNS, it is a consolidated, abstracted structure by which all business applications can consume real-time industrial data in a consistent manner. The benefits of a UNS include reduced time to implement new integrations, reduced efforts to maintain data integrations, improved agility of integrations, access to new data, and improved data quality and security.
When you use UNS in combination with an MQTT broker, you can build a complete, easy to deploy, accessible, and structured solution to power your Smart Factory.
About the Speakers.
Ian Skerrett is Vice President of Marketing at HiveMQ. He has over 25 years of experience in the software industry and is one of the top experts for creating Open Source Strategies for enterprise software products. He was instrumental in creating the Eclipse IoT open source community with more than 15 different open source projects that provide the building blocks developers require for building IoT solutions. He is also a frequent speaker on IoT and MQTT.
John Harrington is the Chief Business Officer of HighByte, focused on defining the company’s business and product strategy. His areas of responsibility include market research, customer use cases, product priorities, go-to-market, financial planning, and sales. John is passionate about delivering technology that improves productivity and safety in manufacturing and industrial environments. He has spent his 25-year career both delivering software to manufacturers and working for manufacturers in operations roles. This experience has given him a unique perspective on how suppliers and end users each play an integral role in implementing new technology solutions. John has a Master in Business Administration from Babson College and a Bachelor of Science in Mechanical Engineering from Worcester Polytechnic Institute.
To watch the webinar recording: https://www.hivemq.com/webinars/building-an-enterprise-unified-namespace-to-power-your-smart-factory/
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data HubSouth Tyrol Free Software Conference
Tracking aeroplanes in real time with Open Source Software is possible. Aircrafts must continuously send their current flight parameters to air traffic controllers on the ground and to other aircrafts. This generates a lot of data, especially when planes are being tracked by multiple sensors.
The Open Data Hub on the other hand offers a great backbone for data storing and processing, where the correct datasets have to be identified and filtered. After all transformation on the data is done, it will be exposed via API to be further used by a web application.
Bringing together sensor generated data, the Open Data Hub and custom web applications, is a showcase on how the Open Data Hub can be used as a service: OaaS.Similar to Truecaller towards a data-driven company (20)
Predicting Startup Market Trends based on the news and social media - Albert ...
Predicting Startup Market Trends based on the news and social media - Albert ...
Real time analytics that controls 50% of mobile network in Poland - Maciej Br...
Real time analytics that controls 50% of mobile network in Poland - Maciej Br...
Data-Driven Fast Track: Introduction to data-drivenness with Piotr Menclewicz
Data-Driven Fast Track: Introduction to data-drivenness with Piotr Menclewicz
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
Complex event processing platform handling millions of users - Krzysztof Zarz...
Complex event processing platform handling millions of users - Krzysztof Zarz...
Take Action: The New Reality of Data-Driven Business
Take Action: The New Reality of Data-Driven Business
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025
Tibco Augmented Intelligence - Analytics, IoT, Big Data, Streaming 20161025
There are 250 Database products, are you running the right one?
There are 250 Database products, are you running the right one?
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Streaming Analytics Comparison of Open Source Frameworks, Products, Cloud Ser...
Streaming Analytics Comparison of Open Source Frameworks, Products, Cloud Ser...
Streaming Analytics - Comparison of Open Source Frameworks and Products
Streaming Analytics - Comparison of Open Source Frameworks and Products
Worldpay - Delivering Multi-Tenancy Applications in A Secure Operational Plat...
Worldpay - Delivering Multi-Tenancy Applications in A Secure Operational Plat...
Data Works Summit Munich 2017 - Worldpay - Multi Tenancy Clusters
Data Works Summit Munich 2017 - Worldpay - Multi Tenancy Clusters
Predictive Analytics and the Industrial Internet of Manufacturing Things with...
Predictive Analytics and the Industrial Internet of Manufacturing Things with...
HiveMQ & HighByte Presents: Building an Enterprise Unified Namespace (UNS) to...
HiveMQ & HighByte Presents: Building an Enterprise Unified Namespace (UNS) to...
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
SFSCON23 - Martin Rabanser - Real-time aeroplane tracking and the Open Data Hub
More from GetInData
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
How do we work with customers on Big Data / ML / Analytics Projects using Scr...
How do we work with our customers ? How does it look? What do the meetings look like ? How do we structure the cooperation? Who does what and when ?
We receive these kinds of questions quite often. They are very important questions as the customer should know the details before we start the project and it’s important for GetInData to be transparent on this so the client is well informed.
During the webinar our Project Lead, Rafał Zalewski talked about Scrum Framework we use in cooperation with our customers.
Watch here:
https://www.youtube.com/watch?v=uOWrgcaKwWo&t=32s
Speaker: Rafał Zalewski, GetInData: https://www.linkedin.com/in/rafalzalewski/
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
How NOT to win a Kaggle competition
If you want to stay up to date, subscribe to our newsletter here: https://bit.ly/3tiw1I8
Presentation from the performance given by Piotr Chaberski and Adrian Dembek the Data Science Summit ML Edition.
Authors: Piotr Chaberski, Adrian Dembek
Linkedin: https://www.linkedin.com/in/piotrchaberski/
https://www.linkedin.com/in/adriandembek/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
How to become good Developer in Scrum Team?
Speaker:
Rafał Zalewski, GetInData: https://www.linkedin.com/in/rafalzalewski/
Abstract:
To become good Developer in Scrum Team you need to understand not only Scrum Events but also Scrum fundaments like Scrum Pillars and Scrum Values. In this presentation you will learn and understand the mindset expectation from you as Developer in Scrum Team. You will also learn how Scrum mindset helps to achieve better development results.
____
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
OpenLineage & Airflow - data lineage has never been easier
If you want to stay up to date, subscribe to our newsletter here: https://bit.ly/3tiw1I8
Presentation from the performance given by Paweł during the Airflow Summit 2022.
Author: Paweł Leszczyński
Linkedin: https://www.linkedin.com/in/pawel-leszczynski/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Benefits of a Homemade ML Platform
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
Building your own platform is often ostracized these days. Everyone is encouraged to reuse existing solutions for known reasons. But using a ready-made platform / tool should not be a mindless process. Reusability is an art. During this presentation, you will learn why we decided to build our own MLOps platform while not re-inventing the wheel by using ready-made components with a touch of custom components. What are the benefits of this, but also what limitations and hurdles we have encountered. We hope that our experience will help you make the right decisions in your projects. Sometimes, maybe more risky ones.
Creating Real-Time Data Streaming powered by SQL on Kubernetes - Albert Lewan...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
This workshop focuses on creating a data streaming platform from scratch using an empty Kubernetes (or even Minikube) cluster. During the workshop, we go through the installation process, deploy the basic components for the platform, start Apache Flink, and monitor the process, using SQL to query available data.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
MLOps implemented - how we combine the cloud & open-source to boost data scie...
Check out more about this presentation here: https://www.youtube.com/watch?v=nSsssYHiylQ&t=17s
Presentation from the performance given by our team during the NSML Summit.
Authors: Krzysztof Zarzycki, Marek Wiewiórka
Linkedin: https://www.linkedin.com/in/kzarzycki/
https://www.linkedin.com/in/marekwiewiorka/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
Did you like it? Check out our E-book: Apache NiFi - A Complete Guide
https://ebook.getindata.com/apache-nifi-complete-guide
Apache NiFi is one of the most popular services for running ETL pipelines otherwise it’s not the youngest technology. During the talk, there are described all details about migrating pipelines from the old Hadoop platform to the Kubernetes, managing everything as the code, monitoring all corner cases of NiFi and making it a robust solution that is user-friendly even for non-programmers.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Feast + Amundsen Integration - Mariusz Strzelecki, GetInData
Read more here: https://getindata.com/blog/machine-learning-features-discovery-feast-amundsen
Author: Mariusz Strzelecki
Linkedin: https://www.linkedin.com/in/mariusz-strzelecki/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
More and more services are running in Kubernetes so it means that we can migrate our current data pipelines to the new environment. In case of Flink we have multiple ways to do real-time data streaming: use Lyft or GCP operator, go with official deployment and customize it or choose the Ververica Platform or create something on your own. The presentation shows how to choose the right solution for technical requirements and business needs to run Flink in Kubernetes at great scale with no issues.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Big data trends - Krzysztof Zarzycki, GetInData
If you want to stay up to date, subscribe to our newsletter here: https://bit.ly/3tiw1I8
Get more info here: https://getindata.com/blog/6-big-data-trends-2021-bigdata-blog/
Author: Krzysztof Zarzycki
Linkedin: https://www.linkedin.com/in/kzarzycki/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The talk is focused on administration, development and monitoring platform with Apache Spark, Apache Flink and Kubeflow in which the monitoring stack is based on Prometheus stack.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Analytics 101 - How to build a data-driven organisation? - Rafał Małanij, Get...
Check out more about this presentation here: https://www.youtube.com/watch?v=eqNToHn4yB0
The webinar was organized by GetinData on 2020. During the webinar we explaned what does it mean to build a data-driven company.
Watch more here: https://www.youtube.com/watch?v=eqNToHn4yB0
Speaker: Rafał Małanij
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Monitoring in Big Data Platform - Albert Lewandowski, GetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The webinar was organized by GetinData on 2020. During the webinar we explaned the concept of monitoring and observability with focus on data analytics platforms.
Watch more here: https://www.youtube.com/watch?v=qSOlEN5XBQc
Whitepaper - Monitoring ang Observability for Data Platform: https://getindata.com/blog/white-paper-big-data-monitoring-observability-data-platform/
Speaker: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Managing Big Data projects in a constantly changing environment - Rafał Zalew...
Watch our full performance given by our team during the Big Data Technology Warsaw Summit: https://www.youtube.com/watch?v=CBrq7z8ikaM
The nature of Big Data projects are nowadays one of its kind - they are not like the data warehousing initiatives in the old days, nor like cloud native applications projects, at least not yet. Variety of technologies, complicated architectures and rapidly changing landscape are just a few challenges that the IT Department is facing in such projects. When you add the number of stakeholders from different departments involved and that Big Data project is sometimes more like an R&D with unpredictable outcome, this makes a mix where the objectives can be easily lost. It is not a surprise that up to 85% of Big Data projects were pure failures (Gartner 2016).
In this talk we will share our experience in planning and executing Big Data initiatives in the organisations, with some use cases and good practices in mind
Watch our webinar here: https://www.youtube.com/watch?v=CBrq7z8ikaM
Speakers:
Rafał Małanij
Rafał Zalewski
Linkedin: https://www.linkedin.com/in/rafalzalewski/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
NLP for videos: Understanding customers' feelings in videos - Albert Lewandow...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
Currently there are more and more created videos distributed via multiple social media channels. It becomes more and more important to monitor all of them by companies to verify their customers' feedback, reviews, opinions. During the talk, we talk about extracting text from videos, analyzing language and prepare robust, scalable infrastructure for it. The idea behind platform is about having the mix between managed and self-managed service for Big Data processing. The keynote shows the case study of the MVP of the platform for marketing companies.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Strategies for on premise to Google Cloud migration - Mateusz Pytel, GetInData
If you want to learn more about it watch our video: https://www.youtube.com/watch?v=WjAY9mnqD4s
The webinar was organized by GetinData on 2020, June 9th. We discussed various approaches one can take to migrate the infrastructure, pointing out their pros and cons. The webinar was based on our real-life, production-grade experiences, however, the strategies and insights can be easily applied to all kinds of IT business.
Watch our webinar here: https://www.youtube.com/watch?v=WjAY9mnqD4s&t=1s
Speaker: Mateusz Pytel
Linkedin: https://www.linkedin.com/in/em-pe/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Monitoring environment based on satellite data with Python and PySpark - Albe...
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
Satellite data is not well-known although it provides a lot useful information for different sectors like agriculture, industry or military.During the workshops participants will learn how to get the satellite imageries, process them by using Python and PySpark and what are the most common use cases. There will be a knowledge sharing session about issues with satellite data and how these difficulties can be overcome.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
_____
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Welcome to MLOps candy shop and choose your flavour! - Mateusz Pytel & Marius...
If you want to learn more, check out our performance given by our team during the Big Data Technology Warsaw Summit: https://www.youtube.com/watch?v=gdinM_9R2GA&t=22s
Operationalizing Machine Learning operations (features delivery, model training, deployment and serving) is nowadays one of the most challenging areas in fast-growing data-driven companies. Variety of open source components (Kubeflow, Mlflow, Kedro to name a few) and set of specialized managed services provided by every major cloud provider drive solution architects nuts.
In GetInData we have a solution for it - we used to call it GetInData MLOps Platform. Set of reusable components, following the Unix toolset pattern ("do one thing and be best at it") and portable to any environment. Also - thanks to loose coupling - adjustable to current and future clients' ML-related challenges, like a candy shop where the first person needs super-fast online predictions, the second one requires robust hyperparameter tuning for best possible models and the third person aims for scalable collaboration on features extraction within many data science teams.
During the presentation we will show you two components we're really excited for - Kedro-Kubeflow integration and Feast-based feature store - how we implement these and what clients' use them for. Welcome to our MLOps candy shop that no pandemic can close
Watch our webinar here: https://www.youtube.com/watch?v=gdinM_9R2GA&t=10s
Speakers:
Mateusz Pytel - Google Certified Professional: https://www.linkedin.com/in/em-pe/
Mariusz Strzelecki - Senior Machine Learning Engineer: https://www.linkedin.com/in/mariusz-strzelecki/
___
Company:
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
More from GetInData (20)
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdf
How do we work with customers on Big Data / ML / Analytics Projects using Scr...
How do we work with customers on Big Data / ML / Analytics Projects using Scr...
OpenLineage & Airflow - data lineage has never been easier
OpenLineage & Airflow - data lineage has never been easier
Creating Real-Time Data Streaming powered by SQL on Kubernetes - Albert Lewan...
Creating Real-Time Data Streaming powered by SQL on Kubernetes - Albert Lewan...
MLOps implemented - how we combine the cloud & open-source to boost data scie...
MLOps implemented - how we combine the cloud & open-source to boost data scie...
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
Best Practices for ETL with Apache NiFi on Kubernetes - Albert Lewandowski, G...
Feast + Amundsen Integration - Mariusz Strzelecki, GetInData
Feast + Amundsen Integration - Mariusz Strzelecki, GetInData
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Kubernetes and real-time analytics - how to connect these two worlds with Apa...
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...
Analytics 101 - How to build a data-driven organisation? - Rafał Małanij, Get...
Analytics 101 - How to build a data-driven organisation? - Rafał Małanij, Get...
Monitoring in Big Data Platform - Albert Lewandowski, GetInData
Monitoring in Big Data Platform - Albert Lewandowski, GetInData
Managing Big Data projects in a constantly changing environment - Rafał Zalew...
Managing Big Data projects in a constantly changing environment - Rafał Zalew...
NLP for videos: Understanding customers' feelings in videos - Albert Lewandow...
NLP for videos: Understanding customers' feelings in videos - Albert Lewandow...
Strategies for on premise to Google Cloud migration - Mateusz Pytel, GetInData
Strategies for on premise to Google Cloud migration - Mateusz Pytel, GetInData
Monitoring environment based on satellite data with Python and PySpark - Albe...
Monitoring environment based on satellite data with Python and PySpark - Albe...
Welcome to MLOps candy shop and choose your flavour! - Mateusz Pytel & Marius...
Welcome to MLOps candy shop and choose your flavour! - Mateusz Pytel & Marius...
Recently uploaded
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentationszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Mind map of terminologies used in context of Generative AI
Mind map of common terms used in context of Generative AI.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
By Design, not by Accident - Agile Venture Bolzano 2024
As presented at the Agile Venture Bolzano, 4.06.2024
Recently uploaded (20)
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Truecaller towards a data-driven company
- 1. Truecaller towards a data-driven company Marek Wiewiórka, Tomasz Żukowski
- 2. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Agenda 1. Truecaller - a global phonebook 2. Evolution of the company’s data architecture 3. Data as a company asset
- 3. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Truecaller ■ World's largest mobile phone community ( > 250 mln users)
- 4. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Truecaller In Numbers ■ +6 billion application events daily ■ +3 TB of compressed user generated data daily ■ +65M active users and 250k application installations daily ■ +28M identified spam calls every day ■ ...
- 5. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Not ly Data-Driven Beginnings... ■ Data layer and analytics powered by MySQL databases ■ No separation of OLTP and OLAP domains ■ Daily ETL processes that used to take longer than one day ;) ■ Problems with storing and querying historical (cold) data ■ Basic reporting without possibility of doing real data science ■ Almost no DWH design principles in place...
- 6. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Towards ly Scalable Data Architecture DWH Data ingestion Schema repo
- 7. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Towards ly Scalable Data Architecture ■ Both data ingestion and data storage/analytics layers are horizontally scalable ■ High availability for both master and worker nodes ■ Apache Avro with schema evolution features and centralized schema repository makes adding new event types seamless for ETL processes ■ Clear separation of staging (raw - Avro format) and reporting (cleaned and enriched in ORC format) data
- 8. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Towards ly Self-Service Analytics
- 9. © Copyright. All rights reserved. Not to be reproduced without prior written consent. ly Analytical Tools
- 10. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Jupyter Notebooks
- 11. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What Can We Do With These Data? ■ Calculate spammer score
- 12. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What Can We Do With These Data? ■ Calculate spammer score ■ Visualize our business
- 13. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What Can We Do With These Data? ■ Calculate spammer score ■ Visualize our business ■ Monitor KPIs after upgrades
- 14. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What Can We Do With These Data? ■ Calculate spammer score ■ Visualize our business ■ Monitor KPIs after upgrades ■ Better target ads
- 15. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What Can We Do With These Data? ■ Calculate spammer score ■ Visualize our business ■ Monitor KPIs after upgrades ■ Better target ads ■ Detect fraudulent user behaviour
- 16. © Copyright. All rights reserved. Not to be reproduced without prior written consent. LTV - how to calculate
- 17. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Market Share Estimation
- 18. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Is Brexit ly a Problem? ■ Calculated on anonymized data of 200k users in the UK ■ Analysis prepared just after Brexit referendum
- 19. © Copyright. All rights reserved. Not to be reproduced without prior written consent. Is Brexit ly a Problem?
- 20. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
- 21. © Copyright. All rights reserved. Not to be reproduced without prior written consent. What’s next ■ More digging into data (a lot of areas not even touched yet) ■ More advanced modelling ■ Streaming analytics
- 22. © Copyright. All rights reserved. Not to be reproduced without prior written consent. ?