SEM analytics: A model for PPC bidding and budget
2. efficiency curve and optimization
3. a time series model of max CPC
4. budget allocation
5. best practices and experiment learning
Search marketers face a range of challenges managing PPC advertising campaigns: rising keyword
price, measurement of impact on and offline, ROI tracking, expansion of keyword, management of
multiple engines, click fraud, bid tool functionality, declining ROI, etc. Determining the maximum
CPC or willingness to pay for a given keyword phrase is perhaps the most challenging and most
rewarding task. On one hand, Maximum CPC value is dynamically depending on ad conversion rate,
profit margins, advertising budget and other factors. Marketers have to adopt different bidding
strategies at various grains such as keyword level, keyword group level, match type level and
search engine level. On the other hand, a powerful bidding strategy will deliver sustainable ROI or
other efficiency goals. Marketers need to take advantage of all the data that’s available through
adaptive learning algorithms. This paper demonstrates how to use simple time series modeling to
determine the max CPC. In addition, this paper reports a method to optimize the advertising budget
allocated to a keyword portfolio. The method is adopted through a variation of the Dorfman-steiner
PPC bidding management is both science and art to improve bidding and budget
allocation efficiency for PPC efforts. It is a science because marketers heavily
rely on quantitative analyses and routinely uses various data mining techniques
such as regression, segmentation, time series model, etc. It is also an art
because marketers need to find the perfect mix of max CPC, bidding frequency,
positioning, day parting, budget allocation and more. Creativity is required to
combine both science and art together to improve ROI or other efficiency goals.
This paper focuses on the science part by introducing a simple time series
model for bidding management and also a budget allocation method through
marginal analysis. First, a simple framework for PPC ROI management is set up
through efficiency curve. Second, efficiency is then defined as the optimal
strategies based on efficiency curve. Third, max CPC is modeled to reach the
optimal. Fourth, we make recommendations on how to allocate budget which is
consistent with the framework and optimal strategies. Finally, a few best
practices are proposed.
Efficiency Curve (EC) and optimization:
Efficiency curve, which is similar to the production curve in economics, is
defined as following:
y is the target revenue; x is the investment or spending. Fig. 1 shows a plot of
daily spend and daily revenue for a keyword portfolio over a period of time.
Even though there is a lot of daily variations of both spend and revenue
because of weekly trend, positioning, competition, and other factors, we still
see a nice curve with diminishing returns to scale. In addition, we have
observed that EC for a keyword may be much more fluid with even more
variations, especially true for lower volume keywords. However, the
diminishing nature of the efficiency curve is still present. The following
proposed methodologies should still apply at keyword level.
Fig. 1. Typical daily spend level vs. daily revenue for a keyword portfolio on a main
search engine. A power curve is fitted: y is the expected revenue and x is the spending.
There are many candidate functional forms as long as a few constraints could
be met. When x is zero, y needs to be zero. We also need diminishing returns
to scale. We have found that the functional form for SEM is best described as a
power curve as following:
y = 30.909x0.5375
R² = 0.8297
Power curve has many very suitable features to serve as the production
function for a KW portfolio. A power curve is called effective for optimization
if e is in (0,1) such that we have diminishing returns. Otherwise, it is called
ineffective for optimization.
Average return (AR) or ROI is then calculated as:
AR = y/x=a*x(e-1) ---(3)
Note the relationship between Revenue per Click (RPC) and Cost per Click (CPC)
is obtained through a simple transformation:
AR = (y/clicks) / (x/clicks) = RPC/CPC ---(4)
Marginal return (MR) could be easily calculated as following:
µ=MR =dy/dx= a*e*x^(e-1) =e * y / x ---(5)
Another term of interest to the bidding management of a keyword portfolio is
the elasticity of revenue to spend is:
Є=∆y/∆x= (dy/y)/(dx/x) = MR/AR=e ---(6)
Elasticity (Є) basically describes that revenue will increase only e*100% if
spending is increased by 100%.
Efficiency is simply defined as the maximum obtainable revenue (or other
conversion metrics) for a given ad spending level for certain time window (day,
week, month, etc) on the same efficiency curve. We assume external factors
such as competition or Search Engine algorithmic changes are uncontrollable.
In addition, we assume that internal controllable factors such as keyword
portfolio size, keyword grouping, CR and LP are optimal and static. Hence, the
only tasks at hand are to determine the max CPC for keyword queries and how
much to spend on a given KW portfolio.
If we relax these assumptions, we could define another term called Efficiency
Gain. Efficiency Gain is the percentage increase in revenue for a given spend
level after we shifted the efficiency curve upwards. Proposed methodologies in
this paper would still work after we relax these assumptions for various
test/control scenarios. Efficiency gain could be used to measure the power of
any enhancement of managing a keyword portfolio. We will provide usage of
EG in the end of the paper. Similarly, efficiency loss is the percentage decrease
in revenue for a given spend level if the curve is shifted downwards.
We will discuss a few methodologies based on value of keyword to reach
efficiency on static efficiency curve.
For simplicity, let’s first assume there is a linear relationship between max CPC
and actual average CPC as following:
Max CPC = s * Avg CPC ---(7)
s factor depends on uncontrollable external factors such as competition, search
engine algorithmic change, etc. It is normally quite stable but different by
keyword and search engines. However, the exact value doesn't impact
following proposed algorithm. We could even assume it is equal to 1.
First methodology for estimating max CPC is to identify break-even point such
that the historical average revenue is equal to actual average cost in a certain
Max CPC = s* Avg CPC = s* RPC ---(8)
By setting keyword bidding amount at average revenue, we can ensure
profitability or ROI goals. It is simple yet effective. Most practitioners
recommend this methodology as a starter bidding strategy. However, optimal
efficiency is not reached because this method is essentially AR=y/x=1.
Optimal efficiency could be reached only by setting MR=1 which is the solution
to maximizing profitability (P=y-x):
MR=e*y/x = 1 ---(9)
Transforming by adding clicks into the equation, we get following:
e*RPC =CPC ---(10)
Then we end up with:
Max CPC = s* avg CPC = s* e* RPC ---(11)
For the easy of usage, let’s define scaling factor (SF) as following.
SF = s*e ---(12)
This second methodology says that we should set maximum bidding amount
based on the value of the keyword and also the elasticity of the keyword. This
method obtained through marginal analysis is basically the Dorfman-steiner
theorem which states “a firm which can influence the demand for its product
by advertising will, in order to maximize its profits, choose the advertising
budget and price such that the increase in gross revenue resulting from a one
dollar increase in advertising expenditures is equal to the ordinary elasticity of
demand for the firm’s product”.
Dorfman-steiner theorem essentially says that advertising spend should be at
the level at which an additional dollar of spending just produces additional e
dollar of revenue. This is also our foundation for budget allocation in later part
of the paper where each additional dollar spent will bring in the same
additional revenue regardless where it is spent.
After establishing the optimal solution to our optimization problem, we need
an algorithm to model the process. Both AR and MR methodologies are similar
where we need figure out the RPC and SF for a keyword or a keyword portfolio.
RPC and SF are dynamic in nature. RPC has very big variance if the # of clicks is
small for a keyword. For example, keyword ‘mp3 player 8gb’ has only 50 clicks
so far since we started buying this keyword at some max CPC. The RPC is zero
still. If a conversion could happen at click number 51. Then the RPC jumps to
around 20 cents if the revenue per conversion is $10. It is apparent that simple
calculation of RPC won’t be effectively. We need to balance the variance and
stability of our max CPC if we want to apply our revenue based methodologies
because identifying a steady and optimal state of RPC is the foundation of our
optimization methodology. We will demonstrate how we can model the max
CPC through a simple time series model of RPC and SF in the following sections.
A time series model for estimating keyword level RPC
based on the EC framework:
Let’s continue using keyword ‘mp3 player 8gb’ as an example to illustrate our
time series model. Assume we have tracking and correlation in place to tell us
the cost and revenue for each and every click.
For simplicity, we target Max CPC divided by SF for a keyword since SF is
relatively independent of the RPC estimation. We will propose methodology for
SF estimation in the budget allocation section. After we have SF estimation, we
could easily put both estimated RPC and estimated SF together to reach
optimal bidding amount.
At time zero, we start buying the keyword at a default aRPC_0 (exact value
doesn’t matter, a portfolio average # is a good starting point) which is
sufficient for this keyword to be on the first page and generate some clicks:
Time 0: Max CPC_0/SF =aRPC_0 ---(13)
At time 1, we will be able to evaluate the effectiveness of this prior belief of
the value of this keyword since we can calculate the actual revenue per click
(aRPC_1) for whatever clicks came in so far. Based on the error or gap, we can
make linear or non-linear adjustment to our max CPC so that it could minimize
the error going forward:
Time 1: error = aRPC_1 – aRPC_0
Max CPC_1/SF = aRPC_0 + % * error
= aRPC_0 + % * (aRPC_1 – aRPC_0)
= aRPC_0 * (1-%) + aRPC_1 * % ---(14)
% is also called decaying factor because it reduces the impact of initial value
aRPC_0 over time.
At time t, after t iterations, we end up with following max CPC formula:
Time t: error =aRPC_t – max CPC_t-1
Max CPC_t/SF = max CPC_t-1 (1-%) +aRPC_t * %
=∑ (aRPCn ∗ (1 − %)t−n
𝑛=0 + aRPC_t *% ---(15)
Essentially, our scaled max CPC is an Auto Regression (AR) time series model.
To simplify the matter, we have found the AR(1) is sufficient for our
optimization problem with a linear adjustment factor based on # of clicks at
time t. We propose following formula for max CPC at any given time t:
Max CPC_t/SF = aRPC_t * % + aRPC_0 * (1-%) ---(16)
Where % is defined as following if we have k clicks so far:
% = max(k, c) / c ---(17)
Here c is the cut-off point (# of clicks) for calculating the decay factor. For
example, if the cut off clicks is 200, then ‘mp3 player 8gb’ with 50 clicks would
have a decaying factor of 50/200 = 0.25. Cut-off point largely depends on the
nature of the keyword portfolio and target metrics. Normally it can be
estimated by looking at the portfolio average conversion rates and CPC.
The above algorithm works well because it learns the dynamics of the bidding
over time and makes attempt to reach the optimal. However, we are not
optimal without applying the elasticity or scaling factor. The next question is
how to estimate the scaling factor. So far, we have simply created a learning
algorithm based on the revenue. The interaction between revenue and cost is
assumed away. Naturally, cost perspective should be in our algorithm. This
intuition goes like following: if a keyword is too expensive relatively speaking,
we should reduce its bidding amount. Otherwise, we should increase its
bidding amount. Following section will introduce a method to do so.
Budget allocation based on marginal analysis at subgroup
level of a keyword portfolio:
Target objective of PPC campaigns is normally one or more of following: pre-
defined budget (B), pre-defined return on ads spend (ROAS), and/or cost per
order. Regardless of the target, the proposed methodology should still apply
with minor modification of the algorithm. Here we will use the Dorfman-Steiner
theorem to optimally allocation B and recommend scaling factor.
let's split the keyword portfolio (S) into a mutually exclusive subgroups (S1,
S2,...,SN). Each subgroup has its own elasticity (e) and intercept (a) by fitting a
power curve on historical spend and revenue pairs. In case a subgroup has
abnormal power curve with e<0 or e>1, we need combine it with some other
group until we end up with a smooth and effective power curve such that the
marginal analysis could be conducted. Based on the D-S theorem, optimal
spend allocation for budget B is reached if following is true:
µ = MR1=MR2.... = MRN ---(18)
Since we already know elasticity’s (e1, e2, ..., eN) and intercepts (a1, a2, ....,
aN), it is quite easy to search for a µ such that each subgroup will get a spend
allocation of Bi which is the solution to following system functions:
n=1 = B
Where Bi = (µ/ai*ei)^(1/(ei-1) for i in (1,…,N) ---(19)
See fig. 2 for an illustration of budget allocation.
Fig. 2. Efficiency curves for 4 subgroups for a keyword portfolio. Slope on EC is the
marginal returns. Optimal spend is achieved by allocating budget such that MR for all
subgroups are the same.
Now that we have gained insights into the marginal returns based on the
budget allocated for each subgroup, we could easily calculate the scaling
factor by transforming the MR back into CPC and RPC:
µ = (ai*ei*X^(ei-1)) =ei*y/x (20)
x=ei*y/ µ ---(21)
After plugging in clicks, we get following:
CPC = ei*/ µ *RPC ---(22)
As a result, we have just identified our max CPC formula based on marginal
analysis and elasticity of our EC for a sub group of a keyword portfolio:
max CPC = s * ei/ µ * RPC ---(23)
Our new scaling factor SFi for a subgroup is following:
SFi = s*ei/ µ ---(24)
After plugging in the steady state of optimal RPC results from the learning
algorithm, we end up with the final formula for max CPC at any given time t
for keywords in subgroup i:
max CPC_t = s* ei/ µ * [aRPC_t * % + aRPC_0 * (1-%)] ---(25)
This formula employs subgroup level elasticity (ei), marginal return (µ) and
keyword level conversion rate (RPC) at time t to identify the optimal bidding
amount for a keyword. Efficiency is reached overtime through this bidding
management tool giventhe current status of the keyword portfolio set up of
CR/LP, QS and competition.
A few best practices based on this theoretical framework are provided here:
1. We get lower marginal return as we get more budget. Naturally, we tend to
increase spending cross the board. However, the right thing to do is to look at the
elasticity of different subgroup of our keyword portfolio and scale up accordingly.
2. Head keywords may not have higher elasticity. Tail keywords may not have lower
elasticity. If we use volume as a dimension for segmenting keyword portfolio, we
still need overlay elasticity on top of volume as an additional dimension. The
common practice of scaling up head keywords is not necessarily optimal.
Normally, we get more clicks while efficiency suffers.
3. A longer time window (t) is more beneficial than a shorter time window for
estimating the conversion rate. The learning algorithm needs enough data to learn
the optimal steady state. However, longer time window hides the seasonal or
spiking impact of certain keywords. One solution is to decay the RPC again with
a short window like recent 30 days’ data to include recency management in the
4. Simple test/control could be used to isolate or identify the impact of various PPC
campaign management factors. Improving CR/LP could shift the EC upwards and
result in efficiency gains. In addition, keyword addition and keyword pruning
could lead to the same shift and hence efficiency gains. This framework is not
only easy to use but also a powerful PPC management tool. Fig. 3 demonstrates
the results of one experiment. Efficiency gain is easily demonstrated as the
upward shifting of the EC. On the contrary, efficiency loss could be easily
identified as well if there was a downward shifting of the curve as the result of the
5. Positioning management is simplified after adapting this value based method.
There is essentially no need to manage the dynamics of the position of a keyword.
If a keyword is valued high enough to be on the first page of SERP, this
methodology will ensure max CPC is sufficiently high enough. Pruning of
keywords out of a portfolio is also simplified because the max CPC will be driven
down to be lower than the minimum bid for first page SERP or even zero.
However, we need balance the model results with minimum bid to ensure we get
enough data. At least, we need be mindful of the minimum bid while evaluating
the outcome of the model.
6. SF management is mostly an art in practice. There are many candidate factors for
grouping of keywords such as time on site, category, volume of clicks and
conversions, # of tokens in keywords, semantics of the keywords, etc. As long as
we are constantly identifying factors where differentiation in elasticity and margin
exist, we could leverage the learning ability of the algorithm to reach optimal.
Fig. 3. Improvement in CR/LP is illustrated as the shifting upwards of the
R. Dorfman and P. O. Steiner: Optimal Advertising and Optimal. Quality. The
American Economic Review 44 (1954), 826--836.
y = 30.909x0.5375
R² = 0.8297
y = 7.7191x0.6855
R² = 0.8312
Power (Efficiency Pre)
Power (Efficiency Post)