How we built analytics from scratch (in seven easy steps)

•

1 like•968 views

plumbee

Technology

How we built analytics from scratch
(in seven easy steps)
Jodi Moran, Co-founder & CTO
1

Plumbee’s growth
3
Oct 2011
• 3 founders & 3
founding
employees
• 0 in data
March 2012
• Mirrorball Slots
on Facebook
launch
• 15 staff
• 0 in data
Dec 2012
• Mirrorball Slots
on iOS beta
launch
• 29 staff
• 4 in data
Today
• 1.2M MAU
• 250K DAU
• 39 staff
• 5 in data

“Build, measure, learn”
4
Timing and targeting of offers
Balancing of the virtual economy
Creation of engaging features
Cost-effective acquisition

Goals
5
Never say “we don’t have that data”
Breadth of data use
Depth of data use
Agile data use
Scalable foundation for the future

Step #1:
7
Blank slate No time
No
bandwidth
No
experience
3rd party analytics

Third-party analytics
• Low opportunity
cost
• Full stack solution
• Lots of choices
• Get useful data to
everyone fast
8

Step #2:
10
3rd party
systems lack
flexibility
Want to own
the data
Don’t know
what we want
to know
Analytics is
strategic
Collect everything

What is everything?
• State-changing calls from client to server
• Changes of state
• State-changing calls from client to third
parties (Facebook)
Yes, this is a lot of data: 450m events (45 GB
compressed) per day.
Using Amazon Web Services makes this
possible.
11

Why we like it
No need:
– To test instrumentation
– To add instrumentation of new features
– To touch transactional databases
– To worry we won’t have the data
Easy and fast to implement
... but we still miss things.
13

Step #3:
15
Lots and lots
of data
Need access
Data is
unstructured
No time to
build
structure
Elastic MapReduce & Hive

The secret to success
17
The
right
analyst
Technical
skills
Unstructured
data
Data
architecture

Step #4:
18
Only access
via SQL
Lack of
visibility
Want data to
be everyday
Google Spreadsheets

Step #5:
21
Want to know
what worked
Can’t
separate
factors
Want
flexibility
In-house split testing

It’s easy to serve experiments…
• Server-side random assignment of users
• Second tier allows deep tests (bonus:
canary deployments)
• Tool for configuration-only tests
• Test & variant pairs attached to every
analytics event
22

… but it’s hard to analyse experiments
23
Web
analytics
Conversion
rate
Binomial
distribution
Simple
tests
•Measuring variables that don’t satisfy
“conversion rate” assumptions
•The need for an Overall Evaluation Criterion

Step #6:
24
All data
processing is
manual
This is getting
expensive
And it takes a
long time to
run
Automation & optimization

(Basic) optimization
• Spot instances
• Output compression with snappy
• Python streaming jobs
• There’s a lot more we could do…
26

Step #7:
27
Expensive
Hive clusters
Queries take a
long time to
run
Hive
functionality
is limited
Relational data mart

Why Hive AND a traditional database?
15 GB of
aggregates
20 TB total
28

Goals
30
Never say “we don’t have that data”
Breadth of data use
Depth of data use
Agile data use
Scalable foundation for the future

The results: average daily spenders
31
Month

But we have tons to do.Engineering
• Replace our custom event
aggregators with Flume
• Replace pull-based Hive &
Python streaming jobs with
Cascading + JVM-based
languages
• Change event storage from
JSON to Avro
• Better dashboards and tools
• Consider in-memory
processing, e.g. Spark/Shark
• Toward “big data”
Analysis
• More “actionable”, less
“interesting”
• Continuous optimization: split
/ multivariate testing, multi-
armed bandit
• Better predictive models
• Clustering, segmentation,
personalization
• Toward “data science”
32

33
Jodi Moran jobs.plumbee.com
jodi@plumbee.com www.plumbee.com
@jodi_p_moran apps.facebook.com/mirrorballslots
www.facebook.com/jodipmoran
www.linkedin.com/in/jmoran
Questions? Get in touch!

Similar to How we built analytics from scratch (in seven easy steps)

Games Industry Analytics Forum 2 - Plumbee

GIAF

Infectious Media runs on data. But, as an ad-tech company that records hundreds of thousands of web events per second, they have have to deal with data at a scale not seen by most companies. You can not make decisions with data when people need to write manual SQL only for queries take 10-20 minutes to return. Infectious Media made the switch to Google BigQuery and Looker and now every member of every team can get the data they need in seconds. Infectious Media shares: - Why they chose their current stack - Why faster data means happier customers - Advantages and practical implications of storing and processing that much data Check out the recording at https://info.looker.com/h/i/308848878-power-to-the-people-a-stack-to-empower-every-user-to-make-data-driven-decisions

Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions

Looker

Mainframe (z/OS®) & IBM i systems are incredibly important for many businesses but are often outside the holistic IT Operations Analytics (ITOA) observability available in Splunk. Exclude them & you will miss valuable, critical insights. Ironstream delivers valuable log data, events and intelligence from both IBM mainframe and IBM i environments into Splunk, providing a true 360-degree view of your IT infrastructure. This session will examine how adding more agility; faster MTTI/MTTR; deeper visibility & increased efficiencies address problems seen by real-world customers over the last 7 years. Effectively addressing these problems has resulted in millions of dollars of cost-savings. See how you can use Precisely Ironstream to: - Tap into mainframe and IBM i machine data the easy way! - Get machine data into your Cloud, On-Prem, ITSI, Enterprise Security, or AIOps environments - Address the high-value use cases - Realize fast ROI & be ready for the future!

Learnings from 7 Years of Integrating Mission-Critical IBM Z® and IBM i with ...

Precisely

Like a lot of online businesses today, PrepSportswear’s success is 100% dependent on the availability, scalability and performance of their digital online services. If the website is down, the business stops. They knew they had to transform their business from that of a retailer with a website to a high caliber IT company that sells products online. In these webinar slides, Richard Dominguez, PrepSportswear’s Developer in Operations, shares their journey. They transformed from a team operating a monolithic app using waterfall development methodology on an old, hard to maintain code base, to a modern IT organization applying new practices from Agile development, DevOps and a Service-Oriented Architectural approach. The Impact? PrepSportswear’s Most Successful Online Holiday Shopping Season in Company History! Join us to: Learn how to identify if you are running a monolithic application that is dragging you down. Get tips on hiring the right people to inject a DevOps cultural mindset into your organization. Understand how to break the monolith into smaller pieces that support key lines of business. Discover where to automate monitoring into your pipeline and platform. Identify metrics for individual stakeholders (dev vs. test vs. business). Go forward, celebrate, learn from, and repeat success! Richard will be joined by Andreas Grabner, Performance Advocate at Dynatrace who will support why monitoring, application and end user metrics have to be a key part of your own transformation! Richard Dominguez has 9+ years’ experience as both a System Analyst and Software Developer in Test. He has worked on many high profile projects in Microsoft such as Hyper-V, Windows 7 Client Performance, and Windows Phone Services. Richard now works at PrepSportswear as the company’s DevOps engineer. His responsibilities include site reliability, external synthetic testing, release management and overall site performance. Andreas Grabner has 15+ years’ experience as an architect and developer in the Java and .NET space. In his current role, Andi works as an advocate for high performing applications in both the development and operations areas. He is a regular expert and contributor to large performance communities, a frequent speaker at technology conferences and regularly publishes articles blogs on blog.dynatrace.com

6 ways DevOps helped PrepSportswear move from monolith to microservices

Dynatrace

Spca2014 sp buy orbuild goedhart

NCCOMMS

Watch full webinar here: https://bit.ly/3j7E9Jo Consumers are increasingly using digital banking tools and insurance models, and these numbers will only continue to grow. Financial and insurance organizations have to adapt to the new and always changing situation while complying with new regulations, such as IFRS17, and embracing ESG criteria. At the heart of any digital transformation is data. Therefore, it is not a stretch to say that data management and analytics strategies differentiate many of the leaders from the laggards in the banking, financial services and insurance (BFSI) industry. BFSI organizations still relying on slow, traditional systems and data management processes will find themselves falling behind their competition. In addition, as many adopt cloud strategies, these traditional approaches fill the cloud modernization process with downtime and end user frustration. In fact, according to a McKinsey article, cloud combined with distributed data infrastructure will define how consumers and providers adopt digital insurance models for the next decade. Hear how the BFSI industry is leveraging data virtualization to deploy data fabric or data mesh architectures for enterprise-wide digital transformation. Join this webinar to learn: - The latest trends in BFSI for 2023 and how data and analytics is reshaping the industry - How a logical data architecture can help you capitalize on your data - How Denodo customers digitally transformed themselves using the Denodo Platform

How to Place Data at the Center of Digital Transformation in BFSI

Denodo

Democratizing Data Science in the Enterprise

Jesus Rodriguez

This presentation is designed to help companies strengthen their competitive advantage by leveraging publicly available Web sources. Entrepreneurs, global industry leaders and enterprises of all sizes are turning Web data into lucrative opportunities – creating new revenue-generating products, reducing costs and re-engineering workflows to optimize pricing, streamline reporting, ensure compliance, engage interactively with clients and more. This presentation uses a variety of success stories to illustrate ways in which businesses can use Web data to drive revenue and streamline operations.

Using Web Data to Drive Revenue and Reduce Costs

Connotate

Using Web Data to Drive Revenue and Reduce Costs

Connotate

Data Stack Considerations: Build vs. Buy at Tout

Looker

Sabermetrics turned the baseball world upside down by challenging decades-old measures of individual performance and their perceived linkage to team success. After cementing their legacy as the Lovable Losers for 108 years, the Chicago Cubs were able to leverage a data-driven approach to finally win a World Series. A high-school football coach, devoted to statistical analysis, has won three state championships—by never punting. Formula 1 racing teams collect staggering amounts of telemetry data from their race cars for the purpose of eking out seconds during the course of a race. Geoff Meyer shares how Dell leverages data from sources as varied as engineering, sales, and the factory to augment test operations. From the identification of high-value test cases and test configurations to determining which test scripts should be retired or re-factored, Geoff explains how you can take advantage of big data to optimize the planning, execution, and communication of your organization’s test efforts.

Leverage Big Data and Analytics for Testing

TechWell

Scaling Pinterest's Monitoring

Brian Overstreet

Intro to SEO Forensics >> bit.ly/audit-blog-2018 (Mostly) Free SEO Tools >> bit.ly/seo-tools-2018 SVS Tutorial >> bit.ly/svs-tutorial SVS Matrix >> bit.ly/pubcon-svs Revisions made post-Pubcon Florida for the MKEsearch event: When to do log file analysis -- NEW Mobile assessment for mobile-first indexing -- NEW Rendering assessment from Google I/O -- NEW Event >> https://mkesearch.org/upcoming-events/seo-planning-site-migrations-and-csi-seo-audits

MKEsearch 2018 | CSI: Forensic SEO Audits

Renee Girard

The Wyoming Community College State System made the strategic decision to migrate seven community colleges to the Colleague by Ellucian SQL platform and deploy BusinessObjects as their reporting solution with the strategic goal to develop one state-wide reporting solution. This session will feature Caper College's journey, the collaboration with all six colleges, and a review of the data design and reporting strategy from the reporting solutions consultant.

Ellucian Live 2014 Presentation on Reporting and BI

Kent Brooks

Demystifying Data by Pallas Horwitz, Senior Data Scientist at Blue Shell Games. The Games Industry Analytics Forum (GIAF) returned to San Francisco for its 9th meet-up. The GIAF is a unique opportunity for games analytics practitioners looking to generate insight and value from big data game analytics. If you are interested in attending future GIAF events, please sign up to our mailing list at www.deltadna.com/mailing-list-signup/

GIAF USA Spring 2015 - Demystifying data

Lauren Cormack

Finding The Perfect Donor Database In An Imperfect World

4Good.org

This webinar was co-hosted by Curiosity and RCG Global Services on January 20th, 2022. Watch the webinar on demand: https://www.curiositysoftware.ie/solving-test-data-webinar Outdated test data management practices are today a sinkhole for testing and development time. They stifle release velocity, risk costly legislative non-compliance, and yet still do not provide the data needed to protect releases from damaging bugs. To achieve true quality at speed, the test data paradigm must shift. Enterprises must move beyond slowly copying large sets of production data to a limited number of out-of-date test environments. In this webinar, Global Head of Quality Engineering at RCG, Niko Mangahas, draws on extensive project experience to define the test data challenges facing enterprises today. He then helps you identify the right test data solution for your organisation, setting out principles for effective requirements gathering and program design. Niko then hands over to veteran test data inventor, Huw Price, who demoes some of the latest techniques for making complete and compliant data available on-the-fly during parallel testing, development, and CI/CD.

Curiosity Software and RCG Global Services Present - Solving Test Data: the g...

Curiosity Software Ireland

No doubt Visualization of Data is a key component of our industry. The path data travels since it is created till it takes shape in a chart is sometimes obscure and overlooked as it tends to live in the engineering side (when volume is relevant), an area where Data Scientist tend to visit but not the usual Web/Marketing Data Analyst. Nowadays the options to tame all that journey and make the best of it are many and they don't require extensive engineering knowledge. Small or Big Data, let's see what "Store, Extract, Transform, Load, Visualize" is all about.

Store, Extract, Transform, Load, Visualize. Untagged Conference

Ani Lopez

Watch full webinar here: https://bit.ly/2EpHGyd Presented at Data Champions, Online Asia 2020 Businesses and individuals around the world are experiencing the impact of a global pandemic. With many workers and potential shoppers still sequestered, COVID-19 is proving to have a momentous impact on the global economy. Regardless of the current situation and post-pandemic era, real-time data becomes even more critical to healthcare practitioners, business owners, government officials, and the public at large where holistic and timely information are important to make quick decisions. It enables doctors to make quick decisions about where to focus the care, business owners to alter production schedules to meet the demand, government agencies to contain the epidemic, and the public to be informed about prevention. In this on-demand session, you will learn about the capabilities of data virtualization as a modern data integration technique and how can organisations: - Rapidly unify information from disparate data sources to make accurate decisions and analyse data in real-time - Build a single engine for security that provides audit and control by geographies - Accelerate delivery of insights from your advanced analytics project

A Key to Real-time Insights in a Post-COVID World (ASEAN)

Denodo

Whether your company is a digital native or you’re just starting or well on your way to digital transformation, you know that your web services, mobile apps, and API programs need faster time-to-market without increasing risk. More full-stack and backend developers as well as QA/testing, architecture, and DevOps leaders are trying to inject rocket fuel into their agile development by doing more API testing. But how much UI testing can be moved over to much faster API testing? And then, how short can you make your testing sprints with API testing? With the right provisioning of UI, visual, and API testing, your organization can measurably improve DevOps metrics as you begin to do far more testing in much less time - without increasing on-hand IT resources. In this webinar, Sauce Labs experts will clarify what parts of the “quality spectrum” are better covered by UI testing vs. API testing. Learn why API testing can be much faster and more reliable than UI testing, and how you can leverage API testing to achieve quality at speed. Key Takeaways: - Easily extend your Selenium and Appium UI testing suite with API testing automation for much faster, more reliable functional, integration, and load/performance testing - Implement a CI/CD pipeline and DevOps workflows safely and securely with Sauce Labs full quality spectrum testing - Achieve quality at speed with significant improvements in time to market and quality to market

Quality at Speed: More API Testing, Less UI Testing

Sauce Labs

Similar to How we built analytics from scratch (in seven easy steps) (20)

Games Industry Analytics Forum 2 - Plumbee

Power to the People: A Stack to Empower Every User to Make Data-Driven Decisions

Learnings from 7 Years of Integrating Mission-Critical IBM Z® and IBM i with ...

6 ways DevOps helped PrepSportswear move from monolith to microservices

Spca2014 sp buy orbuild goedhart

How to Place Data at the Center of Digital Transformation in BFSI

Democratizing Data Science in the Enterprise

Using Web Data to Drive Revenue and Reduce Costs

Data Stack Considerations: Build vs. Buy at Tout

Leverage Big Data and Analytics for Testing

Scaling Pinterest's Monitoring

MKEsearch 2018 | CSI: Forensic SEO Audits

Ellucian Live 2014 Presentation on Reporting and BI

GIAF USA Spring 2015 - Demystifying data

Finding The Perfect Donor Database In An Imperfect World

Curiosity Software and RCG Global Services Present - Solving Test Data: the g...

Store, Extract, Transform, Load, Visualize. Untagged Conference

A Key to Real-time Insights in a Post-COVID World (ASEAN)

Quality at Speed: More API Testing, Less UI Testing

Recently uploaded

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Slack Application Development 101 Slides

praypatel2

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

CNv6 Instructor Chapter 6 Quality of Service

giselly40

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

How to convert PDF to text with Nanonets

naman860154

Histor y of HAM Radio presentation slide

vu2urc

Sara Mae O’Brien Scott and Tatiana Baquero Cakici, Senior Consultants at Enterprise Knowledge (EK), presented “AI Fast Track to Search-Focused AI Solutions” at the Information Architecture Conference (IAC24) that took place on April 11, 2024 in Seattle, WA. In their presentation, O’Brien-Scott and Cakici focused on what Enterprise AI is, why it is important, and what it takes to empower organizations to get started on a search-based AI journey and stay on track. The presentation explored the complexities of enterprise search challenges and how IA principles can be leveraged to provide AI solutions through the use of a semantic layer. O’Brien-Scott and Cakici showcased a case study where a taxonomy, an ontology, and a knowledge graph were used to structure content at a healthcare workforce solutions organization, providing personalized content recommendations and increasing content findability. In this session, participants gained insights about the following: Most common types of AI categories and use cases; Recommended steps to design and implement taxonomies and ontologies, ensuring they evolve effectively and support the organization’s search objectives; Taxonomy and ontology design considerations and best practices; Real-world AI applications that illustrated the value of taxonomies, ontologies, and knowledge graphs; and Tools, roles, and skills to design and implement AI-powered search solutions.

IAC 2024 - IA Fast Track to Search Focused AI Solutions

Enterprise Knowledge

Microsoft's Threat Matrix for Kubernetes helps organizations understand the attack surface a Kubernetes deployment introduces to their environments. This ensures that adequate detections and mitigations are in place. By covering over 40 different attacker techniques, defenders can learn about Kubernetes-specific mitigations and controls to deploy to their environments. In this session, we will explore the MS-TA9013 Host Path Mount technique, which is commonly used by attackers to perform privilege escalation in a Kubernetes cluster. Attendees will learn how attackers and defenders can: * Escape the container's host volume mount to gain persistence on an underlying node * Move laterally from the underlying node into the customer's cloud environment * Analyze Kubernetes audit logs to detect pods deployed with a hostPath mount * Deploy an admission controller that prevents new pods from using a hostPath mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Puma Security, LLC

Choosing the right accounts payable services provider is a strategic decision that can significantly impact your business's financial performance and operational efficiency. By considering factors such as expertise, range of services, technology infrastructure, scalability, cost, and reputation, businesses can make informed decisions and select a provider that aligns with their unique needs and objectives. Partnering with the right provider can streamline accounts payable processes, drive cost savings, and position your business for long-term success. https://katprotech.com/accounts-payable-and-purchase-order-automation/

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Katpro Technologies

Recently uploaded (20)

Automating Google Workspace (GWS) & more with Apps Script

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Slack Application Development 101 Slides

🐬 The future of MySQL is Postgres 🐘

CNv6 Instructor Chapter 6 Quality of Service

Handwritten Text Recognition for manuscripts and early printed texts

Boost PC performance: How more available memory can improve productivity

Advantages of Hiring UIUX Design Service Providers for Your Business

[2024]Digital Global Overview Report 2024 Meltwater.pdf

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

How to convert PDF to text with Nanonets

Histor y of HAM Radio presentation slide

IAC 2024 - IA Fast Track to Search Focused AI Solutions

Breaking the Kubernetes Kill Chain: Host Path Mount

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

How we built analytics from scratch (in seven easy steps)

1. How we built analytics from scratch (in seven easy steps) Jodi Moran, Co-founder & CTO 1

2. Plumbee: social casino games 2

3. Plumbee’s growth 3 Oct 2011 • 3 founders & 3 founding employees • 0 in data March 2012 • Mirrorball Slots on Facebook launch • 15 staff • 0 in data Dec 2012 • Mirrorball Slots on iOS beta launch • 29 staff • 4 in data Today • 1.2M MAU • 250K DAU • 39 staff • 5 in data

4. “Build, measure, learn” 4 Timing and targeting of offers Balancing of the virtual economy Creation of engaging features Cost-effective acquisition

5. Goals 5 Never say “we don’t have that data” Breadth of data use Depth of data use Agile data use Scalable foundation for the future

6. In the beginning… 6

7. Step #1: 7 Blank slate No time No bandwidth No experience 3rd party analytics

8. Third-party analytics • Low opportunity cost • Full stack solution • Lots of choices • Get useful data to everyone fast 8

9. 9

10. Step #2: 10 3rd party systems lack flexibility Want to own the data Don’t know what we want to know Analytics is strategic Collect everything

11. What is everything? • State-changing calls from client to server • Changes of state • State-changing calls from client to third parties (Facebook) Yes, this is a lot of data: 450m events (45 GB compressed) per day. Using Amazon Web Services makes this possible. 11

12. 12

13. 12

14. 12

15. 12

16. 12

17. Why we like it No need: – To test instrumentation – To add instrumentation of new features – To touch transactional databases – To worry we won’t have the data Easy and fast to implement ... but we still miss things. 13

18. 14

19. 14

20. Step #3: 15 Lots and lots of data Need access Data is unstructured No time to build structure Elastic MapReduce & Hive

21. 16

22. 16

23. The secret to success 17 The right analyst Technical skills Unstructured data Data architecture

24. Step #4: 18 Only access via SQL Lack of visibility Want data to be everyday Google Spreadsheets

25. 19

26. 20

27. 20

28. Step #5: 21 Want to know what worked Can’t separate factors Want flexibility In-house split testing

29. It’s easy to serve experiments… • Server-side random assignment of users • Second tier allows deep tests (bonus: canary deployments) • Tool for configuration-only tests • Test & variant pairs attached to every analytics event 22

30. … but it’s hard to analyse experiments 23 Web analytics Conversion rate Binomial distribution Simple tests •Measuring variables that don’t satisfy “conversion rate” assumptions •The need for an Overall Evaluation Criterion

31. Step #6: 24 All data processing is manual This is getting expensive And it takes a long time to run Automation & optimization

32.

33.

34.

35. (Basic) optimization • Spot instances • Output compression with snappy • Python streaming jobs • There’s a lot more we could do… 26

36. Step #7: 27 Expensive Hive clusters Queries take a long time to run Hive functionality is limited Relational data mart

37. Why Hive AND a traditional database? 15 GB of aggregates 20 TB total 28

38. 29

39. 29

40. 29 Plumbee analytics today

41. Goals 30 Never say “we don’t have that data” Breadth of data use Depth of data use Agile data use Scalable foundation for the future

42. The results: average daily spenders 31 Month

43. But we have tons to do.Engineering • Replace our custom event aggregators with Flume • Replace pull-based Hive & Python streaming jobs with Cascading + JVM-based languages • Change event storage from JSON to Avro • Better dashboards and tools • Consider in-memory processing, e.g. Spark/Shark • Toward “big data” Analysis • More “actionable”, less “interesting” • Continuous optimization: split / multivariate testing, multi- armed bandit • Better predictive models • Clustering, segmentation, personalization • Toward “data science” 32

44. 33 Jodi Moran jobs.plumbee.com jodi@plumbee.com www.plumbee.com @jodi_p_moran apps.facebook.com/mirrorballslots www.facebook.com/jodipmoran www.linkedin.com/in/jmoran Questions? Get in touch!

How we built analytics from scratch (in seven easy steps)

Recommended

Recommended

More Related Content

Similar to How we built analytics from scratch (in seven easy steps)

Similar to How we built analytics from scratch (in seven easy steps) (20)

Recently uploaded

Recently uploaded (20)

How we built analytics from scratch (in seven easy steps)