nnn

1. Targeting Bandits:
An Analysis of Feature Selection
Michael John Burton, Ph.D.
Ohio University
burtonm@ohio.edu
October 23, 2019
0 Introduction
Strategists in politics, business, finance, security, and elsewhere face an explore-
exploit dilemma. In political campaigns, every dollar spent ranking prospective
supporters is a dollar taken from contacting those same prospects; in counter-
terrorism, every minute spent ranking terror threats is a minute taken from
neutralizing those threats. Strategists seeking new markets or designing efficient
portfolios face similar kinds of trade-offs.
When 1) objects can be ranked in descending order of value even as 2) the act
of ranking diminishes the number of objects from which value can be realized,
we have a targeting problem.
Targeting problems resemble problems in reinforcement learning and in neo-
classical production theory. This paper will XXXX.
A manufacturer must allocate scarce resources to capital and labor, where
investments in capital increase the productivity of labor. A simplified version
of a textbook production function can be written
y = kα β
(1)
where y is the output from pairing capital input k with labor input as modified
by shape parameters α, β. If a budget b constrains expenditures such that
= b − k, then the manufacturer knows
y = kα
(b − k)β
(2)
and should therefore, under a constant b, find the optimal input k so as to
receive the maximal output y.
Targeting problems are similar to production problems in that a strategist,
like a manufacturer, faces a trade-off between investments and spending, but
they differ in the effect that investments on returns from spending. The pro-
duction function in Equation 2 has investments capital equipment multiply the
1

2. slope of the labor function by a constant. We will see below that investments
in algorithmic targeting alter the convexity of the coverage function.
In bandit problems, a gambler must allocate scarce resources to any of a
number of slot machines and compute the expected their expected values in
hopes of settling on the one with the highest reward. A simplified function can
be written
yi =
p0 if lever 0 is pulled
p1 if lever 1 is pulled
(3)
where p0 and p1 are rewards associated with each of two machines expressed as
the probability of receiving a reward of 1. A sequence of pulls can be written
as a vector y = y0 . . . yb−1 , and
y =
i
yi ∈ y (4)
or equivalently
y = 0p0 + 1p1 (5)
where 0, 1 are the number of times each lever is pulled and y is the reward.
Finally, under a budget b,
y = 0p0 + (b − 0)p1. (6)
Targeting problems are similar to bandit problems in that the strategist, like
the like gambler, faces a trade-off between exploration and exploitation. There
may be no direct analogy to k, except perhaps in the regret suffered from pulls
of the lower-paying lever, but clearly the gambler is gaining knowledge that can
lead to the right policy – that is, repeated pulling of the higher-paying lever.
In a targeting problem, the cost is exploration is assessed separately from
the cost of exploitation. Targeting precedes contact and has no independent
reward. Moreover, in a targeting problem we will assume that we are limited
in the number of times we can pull the lever of any particular machine. Given
a large but finite number of machines a strategist will want to arrange those
machines in descending order of p and then play each machine until the number
of plays is exhausted before moving to the next. However, if the act of arranging
the machines carries a cost in itself, then, under a budget constraint, a strategist
must decide how much of the scarce resources should be invested in targeting
before funds are spent on contact.
This paper will proceed in steps. Section 1 XXXX Section 4 briefly discusses
extensions and implications of the campaign problem.
1 The Targeting Problem
Each machine is good for one pull. Each has a number of hidden markings. The
pit boss is willing to state the expected rewards of aggregations, but the boss
has rules: We may see a list of markings and order a set of markings from that
2

3. list. The boss will examine the machines and note markings on the machine
that correspond to the markings on the list. Machines are then grouped into
segments of the whole population of machines such that, for all listed markings,
every machine in each segment is identical to all other machines in that segment.
The boss will then reveal the expected reward for the machines in each segment.
But there is a fee for his service: For every marking added to the list, a
strategist the strategist loses a machine.
2 The Setting
Growth of partitions.XXX
If an argument is presumed, then a function can be abbreviated. For exam-
ple, if D is known and beyond control we can write G(q, ) and F(q, b).
Recalling that k is the length of q and that = b − k, we can transform G
without loss to F(D, q, b). When budgets and populations aren, and therefore
what remains is to determine the features of interest, especially as data on
features is costly.
Let T
(k)
r, be a data set consisting of objects D†
[: , :] when D†
is generated
by r[: k].
Let T
(k)
r, be a target set and T
(k+1)
r, be an updated target set.
2.1 Objects and Expectation
Let X be that matrix wherein m > 0 rows stand for as many objects and n > 0
columns stand for as many features. For simplicity, we will assume that every
feature value for every object is binary.
Let a vector y be an m-length vector such that each yi ∈ y represents a
reward associated with object i. We will begin with an assumption that all
rewards are binary.
Let v be the sum of the m rewards in vector y. The expected reward E[y]
for every object i is therefore v/m.
Let z be a vector comprised of m occurrences of E[y]:
z = E[y]0, . . . , E[y]m−1
or for convenience we many choose to say
z = z0, . . . , zm−1
where z ← E[y].
Let D ← (X, y, z). We will call D the data set. We will assume throughout
that, for every object i, the features in xi, reward yi, and expectation zi are
bound to that object even as the order of these objects may change.
Let the mapping m, n → X, vcty be a random process.
3

4. 2.2 Ranking
A vector 0 . . . n − 1 will be used to reference the columns of X and by
extension the features of object i. This vector is the index vector.
Let r be a permutation of the index vector. This vector is an ordering of
features.
Let k ∈ [[0, n].
A vector r[: k] is a feature vector. Note that many feature vectors may be
produced by more than one ordering of features.
A procedure rank takes D, vctr, k to return revised versith of D
For any r[: k] we may define a set of segment definitions. An exhaustive set
of binary values will have 2k
definitions for any r. The definitions are numbered
as s, . . . , 2k
− 1].
Let Ds be a partition of D such that all objects that fit definition s, with z
updated to reflect the expectation associated with the objects in this segment.
The procedure rank sorts the partitions in descending order of expectation
to return an updated data set k
rD, which holds k
rX, k
ry, and k
rz.
2.3 Optimization
Let be an integer in [0, m]. This value will serve as an operating point for any
data set k
rD. The truncated data set k
rD[: , :] will be the target set.
Let the expectation of v from k
rD given be the sum of expected rewards
from the top objects in the updated data set. If k
rD, r, k are presumed, we get
k
rz, from which we may determine:
E[t] =
i
zi ∈k
r z[: ].
We can therefore define a function G such that
G : D, r, k, → E[k
rt]
that uses rank to transform D into k
rz and with to create E[t] as k
rz[: ].
Suppose we are constrained by a budget b such that = b − k, leaving the
rest of the G intact.
Let F be equivalent to the function G under a budget constraint:
F(D, r, k, b) ≡ G(D, r, k, b − k).
If D, b are presumed, we may abbreviate F as
F : q, k → E[t].
Suppose F(r, k) is the function we want to maximize.
Let (r, k)∗
refers to a feature vector r[: k] that maximizes F(r, k):
(r, k)∗
= argmin
r,k
F(r).
4

5. where r ∈ R, k ∈ min b, m, n. There may be more than one such pair, but only
one is necessary for a solution.
The targeting problem consists of the search for an optimal pair (r, k)∗
so
as to achieve a maximal expectation E[t]∗
.
3 Analysis
3.1 Properties
The function G(D, r, k, ) records the results of ranking procedure described in
subsection 2.2 and leads to E[t].
Theorem 3.1. Presuming (D, r, k), it will be true that G( ) has a minimum
of 0, a maximum of v, and is non-decreasing.
Proof. Because k
rz[: ] is empty, G( = 0) → 0. Because, k
rz[: ] sums to v,
G( = m) → v. Letting g be the difference quotient g( ) = G( + 1) − G( ),
g( ) > 0 for ∈ [0, m − 1] because each zi is in the range [0, 1] (see Subsection
2.1).
Presuming (D, r), some useful facts about G(k, ) are evident.
Let i(h)
denote an object’s position i before ranking and its position h after
incrementing k to k + 1.
Let t be the result of ranking with k and t be the result of ranking with
k + 1.
We may notice that, 1) for some levels of , it will also be true that increasing
k will increase the expected reward, but 2) it is not a rule that increasing k
necessarily increases that expectation.
Want pertinent information....
One way to measure the information gain is to assess the reduction in total
uncertainty associated with y. Presuming v, the log of the number of pos-
sible permutations, and thus the quantity of information inhering in y, will
be log2
m
v . What we care about, however, is information that allows for the
maximization of 1-valued objects in the target set. Letting y0 ← y[: y] and
y0 ← y[y :], we can say that the pertinent information residing in the updated
data set would be log2
m0
v0
· m1
v1
, making the information gain the before-
and-after difference.
However, what we want is pertinent information. This is measured by
marginal benefit of increasing k at the expense of . XXX Lose the differ-
ence quotient; must gain enough efficiency in k + 1, − 1 to overcome the loss
of the difference quotient.
Suppose we have i(h<i)
, meaning that the object moves toward the beginning
of the order, and suppose that the object in question, among all objects that
have moved, this object has moved closest to the beginning. Suppose is held
constant. Obviously, 1) there would have been an object in position h before
incrementing k which was displaced and moved down to a position greater than
5

6. h, and 2) the reward expected from the displacing object must be greater than
that of the displaced object. If = h + 1, then t must be greater than t – this
because the sum of all expected rewards for objects i < h has not changed while
the reward expected from the object at position h will have increased.
3.2 Expectation across Values of k
A vector q is comprised of a subset of column indexes in the range [0, n − 1].
Let r be a permutation of that range. Because k is the size of q, we can say
q ← r[: k], where r[: k] is equivalent to [r0, . . . , rk−1].
Let G be an expansion of G that makes k explicit:
G(D, r, k, ) ≡ G(D, r[: k], ).
Theorem 3.2. For any D, r, k, it will be true that G(k + 1) ≥ G(k + 1).
Proof. Suppose a minimal case: The introduction of a new feature to the ranking
procedure results in the expectation associated with one object increasing and
that of another object decreasing. If the difference is
moving to a higher position in the data set. For this to happen, other objects
must move downward, including the object that had been in the position now
held by the object that moved up.
For the swap to be valid under the ranking procedure, the updated data set
must have the objects ordered such that associated expectations are sorted in
descending order. The object sent to the higher ranking position must have an
expectation greater than the object it is displacing – otherwise, the updated
data set is not well ordered, and the swap was therefore not valid.
Two objects in the same segment share the same expectation by definition.
If the introduction of a feature splits that segment, then either the expectations
of any two members of the former segment will either remain equal or increase.
If they remain equal, then the is no effect. If, however, a difference arises, then
either they will change position or they will not. Again, if no change in position
occurs, there is no effect. If, however, the position changes, then, by the terms of
the ranking procedure, the object offering the higher expectation will be ranked
higher in the order. To move into its new position, it must displace an object
offering a lower expectation. The cumulative for ≥ the new position must be
greater. between rge lowest and highest swap.
Theorem 3.3. For any D, r, b, it will be true that F(k) is nondecreasing.
Proof. Let D(k)
r be the data set generated by ranking D according to the feature
vector r[: k].
Let z
(k)
r be the reward vector associated with D(k)
r .
We will call D(k)
r [: , :] the original target set, as it is constituted by the
objects offering the highest expectation.
We will call D(k+1)
r [: , :] the updated target set.
6

7. If an object was in D(k)
r [: , :] but does not appear in D(k+1)
r [: , :], we will
say that it has been swapped out of the target set. Likewise, if an object was not
in D(k)
r [: , :] but appears in D(k+1)
r [: , :], we will say that it has been swapped
in.
Suppose a minimalist case: The introduction of feature r[k + 1] splits only
a single segment.
If this split prompts an increased expectation for one object, then it must
simultaneously prompt a decreased expectation in at least one other object,
because the sum of expectations must equal v. Any object swapped out of the
target set has an expectation less than the object that was swapping it.
Given D(k)
r and D(k+1)
r , only object i present in updated target set was not
originally in the target set, and only object ¬i is not present in the updated
target set but was originally in the target set. The two objects have been
swapped.
Assuming that D(k)
r and D(k+1)
r were properly ranked, the only way for a
such a swap to occur is for the expectation associated with i to increase and the
expectation associated with ¬i to decrease.
For example, it may be the case that both objects were in the same segment
under D(k)
r , but when feature r[k + 1] was introduced, they were placed in two
different segments producing two different expected rewards. Because only two
objects are involved, and because the sum of all expectations must equal v, we
know that the expectation associated with i increased to the same extent that
the
By the terms of the ranking procedure, a swap would move the higher ex-
pectation object i to a higher position in D(k+1)
r than the lower expectation
object ¬i.
Let E[v
(k)
r ] = i zi ∈ z
(k)
r [: ].
Suppose a minimal case: The data set produced by the ranking procedure
using r[k + 1]
In the transition from the target set T
(k)
r, to the updated target set T
(k+1)
r, ,
only one object i was originally in the target set was left out of the updated
target set and only one object ¬i was left out of the original target set was
included in the updated target set. We will call this movement a swap.
Under the terms of the ranking procedure, the only way to prompt a single
swap is for zi to increase and z¬i to decrease with the updating that occurred
thanks to the inclusion of rk in the feature vector q. Further, a swap will not
occur unless the expectation z for i
(k+1)
r, is greater than the expectation for
¬i
(k)
r, .
Let z
(k)
r, ¬i
(k)
r, be the vector z associated with the target set with ¬i left out.
Let z
(k+1)
r, i
(k+1)
r, be the vector z associated with the updated target set
with i left out.
We note that z
(k)
r, ¬i
(k)
r, is equivalent to z
(k+1)
r, ir.
Ranking leaves the higher of the two expectations in the updated target set.
If has not changed, then the sizes of T
(k)
r, and T
(k+1)
r, are equal, as are the
7

8. sizes of z
(k)
r, and z
(k+1)
r, . However, since
i
(k+1)
r, > i
(k)
r,
¬i
(k)
r, > ¬i
(k+1)
r,
¬i
(k)
r, > i
(k)
r,
i
(k+1)
r, > ¬i
(k+1)
r,
We have a function F(D, q, b), which uses the ranking procedure described
in subsection 2.2 to find E[v].
Recall that a vector q is a subset of column indexes in a range [0, n − 1].
Let r be a permutation of that range. Because k is the size of q, we can say
q ← r[: k], where r[: k] is equivalent to [r0, . . . , rk−1].
Let F be a trivial expansion of F that makes k explicit and allows for its
manipulation:
F(D, r, k, b) ≡ F(D, r[: k], b).
i
(k+1)
r, > i
(k)
r,
¬i
(k)
r, > i
(k)
r,
i
(k+1)
r, > ¬i
(k+1)
r,
¬i
(k)
r, > ¬i
(k+1)
r,
the aggregate expectation E[v] will increase by sum of the changes in the
expectations of the object that moved into the target set and the one that
moved out of it.
If more than two objects are swapped, then every swapped object that is
placed in the updated updated target set offer an expectation greater than
every swapped object left out of the updated target set. Otherwise, the terms
of the ranking procedure have been violated. The the ranking procedure was
followed, the aggregate expectation E[v] will increase by sum of the changes in
the expectations of all objects that moved into the target set and all those that
moved out of it.
We restrict our view to interesting cases. Therefore, we will suppose b ∈
[1, m − 1].
Theorem 3.4. The function H(k, b − k) for with respect to k when b = m − 1
can be increasing, decreasing, single-peaked, or multi-peaked depending on the
amount of information available in D and in which that information is made
available by r.
8

9. Proof. demonstration with appendix
Theorem 3.5. The function H(k, b − k) for with respect to k when b = m − 1
can be increasing, decreasing, single-peaked, or multi-peaked depending on rate
at which information is made available by r.
Proof. demonstration with appendix
4 Illustration
Information Value Theory is typically employed to assess the payoff received
from decreasing uncertainty in a general sense. The present research goes to a
specific application: Costly target ranking. Where every dollar spent on ranking
targets is a dollar lost from acting on those targets, a tactician must find an
optimal allocation of scarce resources to exploration (in the form of ranking)
and exploitation (in the form of acting).
A vendor has matrix X showing residents of a certain voting district as rows
and their respective features are columns. The matrix is available to strategists
on a pay-per-feature basis. All feature columns are sold at the the same price.
A strategist who can afford k features will submit an order specifying q[k] and
then receive Xq,k.
Suppose some residents will vote whether or not the strategist makes contact;
some will not vote even if contacted; some will vote if and only if contacted.
The latter category of voters have a potential reward of 1, which is recorded
in the vector yq,k. However, yq,k is hidden; only pq,k can be revealed to the
strategist.
Suppose the strategist has some kind of magic survey method to reveal pq,k
at no cost and without problems introduced by sampling, overfitting, and so
forth. Let pq,k be an exact representation of expected value.
With these facts in hand, the strategist can find pq,k, and zq,k.
Suppose the strategist has sufficient funds to contact residents. The strate-
gist can expect to realize v = zq,k[ ] new votes.
9