The document provides an overview of workshops on optimizing Google AdWords accounts. It discusses developing reporting backends to analyze campaign performance data at scale. It also covers optimizing keywords by discovering new ideas and traffic estimates using the Targeting Idea Service and Traffic Estimator Service. The document introduces the Kratu tool for finding optimization opportunities and provides an example of an epsilon greedy inspired semi-automatic keyword optimization strategy.
Data Driven Attribution in BigQuery with Shapley Values and Markov ChainsChristopher Gutknecht
This talk covers the journey of implementing two data driven attribution models in BigQuery and the findings so far. The packages Fractribution (Google) and Channel Attribution were used to model Shapley values and markov chains respectively.
One of the biggest time sinks and challenges for mobile application developers is developing, accessing, and managing all of the disparate data sources that are involved in delivering delightful, collaborative, and real-time mobile experiences for users while also enabling offline capabilities for when a user is not connected, but still wants to use the app. In this session, you be introduced to the new AWS AppSync service that speed and simplifies these tasks for developers using GraphQL to provide a data abstraction layer and easy query and update statements without having to know the details of the underlying data sources.
Data Driven Attribution in BigQuery with Shapley Values and Markov ChainsChristopher Gutknecht
This talk covers the journey of implementing two data driven attribution models in BigQuery and the findings so far. The packages Fractribution (Google) and Channel Attribution were used to model Shapley values and markov chains respectively.
One of the biggest time sinks and challenges for mobile application developers is developing, accessing, and managing all of the disparate data sources that are involved in delivering delightful, collaborative, and real-time mobile experiences for users while also enabling offline capabilities for when a user is not connected, but still wants to use the app. In this session, you be introduced to the new AWS AppSync service that speed and simplifies these tasks for developers using GraphQL to provide a data abstraction layer and easy query and update statements without having to know the details of the underlying data sources.
Slidedeck presented at http://devternity.com/ around MongoDB internals. We review the usage patterns of MongoDB, the different storage engines and persistency models as well has the definition of documents and general data structures.
Large scale data capture and experimentation platform at GrabRoman
In this video I'm presenting how we built a system to experiment and rollout features across hundreds of microservices at Grab.
The talk also describes a high-performance event tracking system which captures billions of events per day from mobile apps and backend services and makes them easily queryable through SQL with 1 minute end-to-end latency.
We'll go through feature toggles, experimentation platform and a custom, special-purpose database we built on top of Presto to be able to SQL-query everything.
Related blog posts:
- https://engineering.grab.com/building-grab-s-experimentation-platform
- https://engineering.grab.com/feature-toggles-ab-testing
- https://engineering.grab.com/big-data-real-time-presto-talariadb
- https://engineering.grab.com/experimentation-platform-data-pipeline
[WSO2Con Asia 2018] Patterns for Building Streaming AppsWSO2
This slide deck explains how to enable digital transformation through streaming analytics and how easily streaming applications can be implemented
Learn more: https://wso2.com/library/conference/2018/08/wso2con-asia-2018-patterns-for-building-streaming-apps/
Building Data Products with BigQuery for PPC and SEO (SMX 2022)Christopher Gutknecht
In this data management session, Christopher describes how to build robust and reliable data products in BigQuery and dbt, for PPC and SEO use cases. After an introduction to the modern data stack, six principles of reliable data products are presented, followed by the following use cases:
- Google Ads Conversion upload
- SEO sitemap efficiency report
- Google Shopping product rating sync
- Large-Scale link checker with advertools
- Inventory-based PPC campaigns with dbt
Here is the referenced selection of gists on github: https://gist.github.com/ChrisGutknecht
Modern Thinking: Cómo el Big Data y Cognitive están cambiando la estrategia de Marketing
Por: Ismael Yuste, Strategic Cloud Engineer Google Cloud
Presentación: Introducción a las soluciones Big Data de Google
Optimizing a React application for Core Web VitalsJuan Picado
The performance of your web application can define the success of your website, the core web vitals are key metrics that help you to keep track and improve the user experience. This talk we will see how to optimize and measure a React application performance using some basic techniques, like code splitting with webpack, SEO optimization and bottleneck resolutions with examples.
Applying BigQuery ML on e-commerce data analyticsMárton Kodok
With BigQuery ML, you can build machine learning models without leaving the database environment and training it on massive datasets. We are going to demonstrate common marketing Machine Learning use cases we do at REEA.net to build, train, eval and predict, your own scalable machine learning models using SQL language in Google BigQuery and to address the following use cases:
Customer Segmentation
Customer Lifetime Value (LTV) prediction
Conversion/Purchase prediction
The audience will get first hand experience how to write CREATE MODEL sql syntax to build machine learning models such as:
Multiclass logistic regression for classification
K-means clustering
Import TensorFlow models for prediction in BigQuery
Models are trained and accessed in BigQuery using SQL — a language data analysts know. This enables business decision making through predictive analytics across the organization without leaving the query editor
Deep Visibility: Logging From Distributed MicroservicesAaronLieberman5
Visibility into any system is a key component of creating a supportable platform. Without proper logging, support can be costly and inefficient. With the emergence of APIs, microservices, and distributed, decoupled architectures, logging becomes even more important because there are more components that make up a system than ever before. This is beneficial from the standpoint of creating reliable systems, but logging frameworks need to adapt to this architecture because the premise of logging remains the same as it always has: log clear messages that are easy to read with the goal of enhancing visibility into a system.
In this Meetup hosted by Big Compass, we will explore techniques of logging from the typical iPaas or always-on managed system like a custom application on an EC2, and we will balance that with a discussion on logging from serverless microservices such as AWS Lambda also. We’ll walk through a real system we have created and discuss how a logging framework can be created using AWS serverless services to enhance the visibility and supportability of the system.
You will learn:
• Common best practices and blind spots of logging
• Differences of logging from always-on systems versus serverless services (AWS Lambda)
• Successful use cases where logging has been implemented to improve supportability of a system
Who should attend:
• IT leaders who want to decrease support cost and have a system visibility pain point
• Developers struggling with implementing a robust, highly visible logging solution
• Anyone considering using serverless technology for an upcoming implementation
Reasons to attend:
• Create a logging framework that garners deep visibility and a great experience for users, no matter the underlying architecture
In this session, you will learn: how to embed feature flagging sitewide to deliver safer, faster releases, best practices for implementing feature flags in a services-oriented architecture, and the latest enhancements you need to help your team recover faster when ship happens.
James Turner (Caplin) - Enterprise HTML5 Patternsakqaanoraks
Most HTML5 web applications are relatively small scale – they are maintained by a single team and contain relatively little JavaScript, CSS and HTML5 code.
At Caplin we build "thick client" replacement financial trading systems containing considerable business logic implemented by hundreds of thousands of lines of JavaScript code. The code is maintained by multiple development teams spread across multiple business units. The talk describes the problems faced and how they can be solved using componetization, loose coupling, services, event bus, design patterns, BDD, the best open source libraries, test by contract, and test automation etc.
Real-Time Personalized Customer Experiences at Bonobos (RET203) - AWS re:Inve...Amazon Web Services
In this session, learn how Bonobos, an online retailer for men's clothing and accessories, powers their personalized customer experiences on top of AWS. We start by exploring the foundational elements required to build an effective retail data platform as well as the building blocks provided by AWS to deliver these experiences. Learn how Bonobos leverages Segment in their architecture, and hear from Bonobos and Segment on the objectives, challenges, and outcomes realized by Bonobos through their journey in constructing and deploying their personalization platform.
Design for Scale - Building Real Time, High Performing Marketing Technology p...Amazon Web Services
DynamoDB presented by David Pearson from AWS
Bizo Business Audience Marketing success story on AWS by Alex Boisvert, Director of Engineering, Bizo
In today's world, consumer habits change fast and marketing decisions need to be made within seconds, not days. Delivering engaging advertising experiences requires real time, high performing architectures that provide digital advertisers the ability to measure and improve the performance of their campaigns and tie them more closely to corporate goals. The insights gleaned from the massive amounts of data collected can then be used to dynamically adjust media spend and creative execution for optimal performance. The AWS Cloud enables you to deliver marketing content and advertisements with the levels of availability, performance, and personalization that your customers expect. Plus, AWS lowers your costs. Join us to learn about how big data and low latency / high performing architectures are changing the game for digital advertising.
Slidedeck presented at http://devternity.com/ around MongoDB internals. We review the usage patterns of MongoDB, the different storage engines and persistency models as well has the definition of documents and general data structures.
Large scale data capture and experimentation platform at GrabRoman
In this video I'm presenting how we built a system to experiment and rollout features across hundreds of microservices at Grab.
The talk also describes a high-performance event tracking system which captures billions of events per day from mobile apps and backend services and makes them easily queryable through SQL with 1 minute end-to-end latency.
We'll go through feature toggles, experimentation platform and a custom, special-purpose database we built on top of Presto to be able to SQL-query everything.
Related blog posts:
- https://engineering.grab.com/building-grab-s-experimentation-platform
- https://engineering.grab.com/feature-toggles-ab-testing
- https://engineering.grab.com/big-data-real-time-presto-talariadb
- https://engineering.grab.com/experimentation-platform-data-pipeline
[WSO2Con Asia 2018] Patterns for Building Streaming AppsWSO2
This slide deck explains how to enable digital transformation through streaming analytics and how easily streaming applications can be implemented
Learn more: https://wso2.com/library/conference/2018/08/wso2con-asia-2018-patterns-for-building-streaming-apps/
Building Data Products with BigQuery for PPC and SEO (SMX 2022)Christopher Gutknecht
In this data management session, Christopher describes how to build robust and reliable data products in BigQuery and dbt, for PPC and SEO use cases. After an introduction to the modern data stack, six principles of reliable data products are presented, followed by the following use cases:
- Google Ads Conversion upload
- SEO sitemap efficiency report
- Google Shopping product rating sync
- Large-Scale link checker with advertools
- Inventory-based PPC campaigns with dbt
Here is the referenced selection of gists on github: https://gist.github.com/ChrisGutknecht
Modern Thinking: Cómo el Big Data y Cognitive están cambiando la estrategia de Marketing
Por: Ismael Yuste, Strategic Cloud Engineer Google Cloud
Presentación: Introducción a las soluciones Big Data de Google
Optimizing a React application for Core Web VitalsJuan Picado
The performance of your web application can define the success of your website, the core web vitals are key metrics that help you to keep track and improve the user experience. This talk we will see how to optimize and measure a React application performance using some basic techniques, like code splitting with webpack, SEO optimization and bottleneck resolutions with examples.
Applying BigQuery ML on e-commerce data analyticsMárton Kodok
With BigQuery ML, you can build machine learning models without leaving the database environment and training it on massive datasets. We are going to demonstrate common marketing Machine Learning use cases we do at REEA.net to build, train, eval and predict, your own scalable machine learning models using SQL language in Google BigQuery and to address the following use cases:
Customer Segmentation
Customer Lifetime Value (LTV) prediction
Conversion/Purchase prediction
The audience will get first hand experience how to write CREATE MODEL sql syntax to build machine learning models such as:
Multiclass logistic regression for classification
K-means clustering
Import TensorFlow models for prediction in BigQuery
Models are trained and accessed in BigQuery using SQL — a language data analysts know. This enables business decision making through predictive analytics across the organization without leaving the query editor
Deep Visibility: Logging From Distributed MicroservicesAaronLieberman5
Visibility into any system is a key component of creating a supportable platform. Without proper logging, support can be costly and inefficient. With the emergence of APIs, microservices, and distributed, decoupled architectures, logging becomes even more important because there are more components that make up a system than ever before. This is beneficial from the standpoint of creating reliable systems, but logging frameworks need to adapt to this architecture because the premise of logging remains the same as it always has: log clear messages that are easy to read with the goal of enhancing visibility into a system.
In this Meetup hosted by Big Compass, we will explore techniques of logging from the typical iPaas or always-on managed system like a custom application on an EC2, and we will balance that with a discussion on logging from serverless microservices such as AWS Lambda also. We’ll walk through a real system we have created and discuss how a logging framework can be created using AWS serverless services to enhance the visibility and supportability of the system.
You will learn:
• Common best practices and blind spots of logging
• Differences of logging from always-on systems versus serverless services (AWS Lambda)
• Successful use cases where logging has been implemented to improve supportability of a system
Who should attend:
• IT leaders who want to decrease support cost and have a system visibility pain point
• Developers struggling with implementing a robust, highly visible logging solution
• Anyone considering using serverless technology for an upcoming implementation
Reasons to attend:
• Create a logging framework that garners deep visibility and a great experience for users, no matter the underlying architecture
In this session, you will learn: how to embed feature flagging sitewide to deliver safer, faster releases, best practices for implementing feature flags in a services-oriented architecture, and the latest enhancements you need to help your team recover faster when ship happens.
James Turner (Caplin) - Enterprise HTML5 Patternsakqaanoraks
Most HTML5 web applications are relatively small scale – they are maintained by a single team and contain relatively little JavaScript, CSS and HTML5 code.
At Caplin we build "thick client" replacement financial trading systems containing considerable business logic implemented by hundreds of thousands of lines of JavaScript code. The code is maintained by multiple development teams spread across multiple business units. The talk describes the problems faced and how they can be solved using componetization, loose coupling, services, event bus, design patterns, BDD, the best open source libraries, test by contract, and test automation etc.
Real-Time Personalized Customer Experiences at Bonobos (RET203) - AWS re:Inve...Amazon Web Services
In this session, learn how Bonobos, an online retailer for men's clothing and accessories, powers their personalized customer experiences on top of AWS. We start by exploring the foundational elements required to build an effective retail data platform as well as the building blocks provided by AWS to deliver these experiences. Learn how Bonobos leverages Segment in their architecture, and hear from Bonobos and Segment on the objectives, challenges, and outcomes realized by Bonobos through their journey in constructing and deploying their personalization platform.
Design for Scale - Building Real Time, High Performing Marketing Technology p...Amazon Web Services
DynamoDB presented by David Pearson from AWS
Bizo Business Audience Marketing success story on AWS by Alex Boisvert, Director of Engineering, Bizo
In today's world, consumer habits change fast and marketing decisions need to be made within seconds, not days. Delivering engaging advertising experiences requires real time, high performing architectures that provide digital advertisers the ability to measure and improve the performance of their campaigns and tie them more closely to corporate goals. The insights gleaned from the massive amounts of data collected can then be used to dynamically adjust media spend and creative execution for optimal performance. The AWS Cloud enables you to deliver marketing content and advertisements with the levels of availability, performance, and personalization that your customers expect. Plus, AWS lowers your costs. Join us to learn about how big data and low latency / high performing architectures are changing the game for digital advertising.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Monitoring Java Application Security with JDK Tools and JFR Events
Account Performance and Optimization
1. AdWords API Workshops 2013
Account Performance &
Optimization
Finding the needle in your haystack
Google Confidential and Proprietary
2. Overview
1. How to Optimize
○ The Optimization Cycle
2. Developing a solid reporting backend
○ Reporting Workflow
○ Strategies for scaling, processing and storage
○ Introduction to sample applications
3. Keyword Optimizations
○ What are TIS and TES
○ Discovering good ideas with TIS and TES
○ Automated keyword optimizations strategy
4. Discovering optimization opportunities with Kratu
○ Introduction to Kratu
○ AdWords Optimization Report in Kratu
○ Kratu backend example
Google Confidential and Proprietary
9. Reporting Workflow MCC Info
Managed
List<ManagedCustomer> Customer
Service
Report Definition
ReportType
Report Data Storage
DateRange Report Downloader Processor MySql
MongoDB
...
Fields
n-Threads
Google Confidential and Proprietary
10. Report Data Storage
Report Downloader Processor MySql
MongoDB
...
n-Threads
Process local temp files
Keys must guarantee
Based on Parses rows using a Uniqueness per
MultipleClient CSVReader ReportRow
ReportDownloader (Date/Day, Segments)
example Write calls to Storage
must be in batches Rows Updates are
Each Thread runs a frequent
ReportDownloder with: Batch size depends on
technology/memory Create indexes for:
● AdWordsSession
● ClientCustomerId Shard data based on ● AccountId
● ReportDefinition account, subMCCs, dates ● Date/Day
● CSV & GZip or common attributes ● CampaignId
● Segments...
Google Confidential and Proprietary
11. Implementing a MySql ReportProcessor
connection.setAutoCommit( false);
statement.execute(" SET UNIQUE_CHECKS=0; ALTER TABLE ... DISABLE KEYS;");
for (File file : localFiles) {
preparedStatement = "INSERT INTO .. (.) VALUES (.) ON DUPLICATE KEY UPDATE
(.)";
CSVReader csvReader = new CSVReader(new InputStreamReader (new FileInputStream
(
file.getAbsolutePath() + ".gunzip"), " UTF-8"), ',', '"', 1);
reportRows = new ModifiedCsvToBean <Report>(). parseReport(mapping, csvReader);
for (Report reportItem : reportRows) {
addInsertToBatch(reportItem, preparedStatement);
// Executing the Batch every 500 inserts to reduce memory usage
if (batchSize++ >= 500) {
preparedStatement.executeBatch();
connection.commit();
batchSize = 0;
}
}...
}...
statement.execute(" SET UNIQUE_CHECKS=1; ALTER TABLE ... ENABLE KEYS;");
Google Confidential and Proprietary
12. Implementing a MongoDB ReportProcessor
for (File file : localFiles) {
CSVReader csvReader = new CSVReader(new InputStreamReader(
new FileInputStream(file),"UTF-8"), ',', '"', 1);
List<Report> reportObjectList =
ModifiedCsvToBean<Report>().parseReport(mapping, csvReader);
csvReader.close();
// Save in MongoDB (there is also an AppEngine implementation)
// Iterates using Google Gson: gsonBuilder.create().toJson(Object)
// and DBObject: com.mongodb.util.JSON.parse(jsonObject)
noSqlStorage.save(reportObjectList);
}
Google Confidential and Proprietary
14. TIS SearchParameter[ ]
Keyword / Placement
TargetingIdeaSelector
Ideas / Stats
AttributeType[ ] (fields)
Targeting Idea
Service
TargetingIdea[ ]
Type_AttributeMapEntry[ ]
(field,value)
Google Confidential and Proprietary
15. TIS - Search Parameters & Fields
SearchParameter[ ] AttributeType[ ] (fields)
● AdSpecList ● Average Cpc
● CategoryProductsAndServices ● Average Targeted Monthly Searches
● Competition ● Category Products and Services
● ExcludedKeyword ● Competition
● IdeaTextFilter ● Criterion
● IncludeAdultContent ● Extracted from Webpage
● Language ● Global Monthly Searches
● Location ● Idea Type
● Network NEW ● Keyword Category
● PlacementType ● Keyword Text
● RelatedToQuery ● Negative Keywords
● RelatedToUrl ● Search Share
● SearchVolume ● Search Volume
● SeedAdGroupId ● Targeted Monthly Searches
...
Google Confidential and Proprietary
16. TIS EXAMPLE
LanguageSearchParameter = english
RelatedToUrlSearchParameter = http://www.someplumbers.com/
LocationSearchParameter = uk
RequestType. IDEAS, IdeaType. KEYWORD
{AttributeType. KEYWORD_TEXT, AttributeType. SEARCH_VOLUME,
AttributeType. AVERAGE_CPC ,AttributeType. COMPETITION}
Targeting Idea
Service
KEYWORD_TEXT searchVol avgCPC Competition
'london plumbing service' 16 5.13 0.858
'london plumbing services' 46 9.41 0.765
'emergency plumbing service' 28 6.71 0.912
... new
Google Confidential and Proprietary
17. TES
TrafficEstimatorSelector
CampaignEstimateRequest[ ] Criterion[ ], campaignId
AdGroupEstimateRequest[ ] adGroupId, maxCPC
KeywordEstimateRequest[ ] keyword, maxCpc, isNegative
Traffic Estimator Service
CampaignEstimate[ ] campaignId
AdGroupEstimate[ ] adGroupId
KeywordEstimate[ ] criterionId, Min & Max StatsEstimates
averageCpc, averagePosition, clicksPerDay, impressionsPerDay, clickThroughRate, totalCost
Google Confidential and Proprietary
18. campaignEstimateRequest.setCriteria(new Criterion[] { uk, english });
adGroupEstimateRequest.setMaxCpc(new Money(null, 1000000)); // 1€
adGroupEstimateRequest.setKeywordEstimateRequests( ...
'london plumbing service'
'emergency plumbing service london'
'emergency plumbing services london'... )...
Traffic Estimator Service
Keyword avgCPC avgPosition dayClicks totalCost CTR
dayImpressions
'london plumbing service', 0.87, 2.00, 0.02, 0.02, 0.04, 0.58
'plumbing services london', 0.71, 8.56, 0.54, 0.39, 0.01, 49.33
new new
...
Google Confidential and Proprietary
19. Example Optimization - Epsilon Greedy Inspired
http://en.wikipedia.org/wiki/Multi-armed_bandit
Automatic auto glass repair
● Regular car glass replacement
● Set and forget
● Needs close monitoring windshield repair
vehicle glass repair
Semi-automatic KW Research
● Manual lookup car window repair Using TIS+TES
● Suggest KWs for exchange fix car glass
● KW Research can be
manual car glass
glass window
glasses repair
KW window repair
Google Confidential and Proprietary
21. Kratu - Open Source Issue and Opportunity Discovery
Google Confidential and Proprietary
22. Kratu - Open Source Issue and Opportunity Discovery
● Open Source, Google driven
○ http://google.github.com/kratu/
● Client-side, runs in the browser. Implemented in JavaScript
● Includes prebuilt working example for AdWords
○ Starting point created by experienced Google Account Managers
○ Used internally at Google to display data for millions of accounts
○ Provided as-is. Use with caution. Google accepts no liabilities.
● BYOD - Bring Your Own Data
○ Anything that can be converted to a JS Array with key/value objects
○ Includes simple JSON and CSV loaders
● We offer a Google built sample backend
● Easily integratable with other, non-AdWords data
Google Confidential and Proprietary
24. Kratu - Implementation - Overview
Reporting
Data Storage backend
MySql Fetch
MongoDB Process
... Store
Web server
/kratu/
/rest/accounts/
Google Confidential and Proprietary
25. Kratu - Implementation
● Easy to add other signals
○ Add any key/value pair to your data, define a weight and a threshold
● Flexible model
○ Custom and built-in calculation of opportunity/issue
○ Compose new signals from multiple data points
○ Custom and built-in event handlers (integrate with other systems)
○ Custom and built-in formatting
● Paginations for larger number of accounts
● Re-use data and signals to produce new reports
○ Targeted reports, eg. upselling opportunities, mobile adoption
○ Personalized reports, eg. individual for Account managers
● End-to-end tutorial available here:
○ http://google.github.com/kratu/tutorial/
Google Confidential and Proprietary
26. Kratu - backend example
Quick example using Restlet and Gson:
router.attach("/accountreports/{accountId}", ReportRest.class);
String accountId = (String) getRequestAttributes().get("
accountId");
@Get
public JsonRepresentation getAccountPerformanceReportHandler() {
// Gets Reports from MongoDB or MySql as Java Objects
List<Report> listAccountReports = Report.getReportByAccountIdMonth(
accountId, date, "AccountReport");
String resultJsonList = gson.toJson(listAccountReports);
JsonRepresentation jsonRepresentation = new
JsonRepresentation(resultJsonList);
jsonRepresentation.setMediaType(MediaType.APPLICATION_JSON);
return jsonRepresentation;
}
Google Confidential and Proprietary