Data Mining : What is a good lift ?

… ever modeled the lift of a targeting model ?

In a previous post “Data mining for marketing campagns : interpretation of lift” I discussed the factors that influence de lift of a targeting model. Apart from the quality of the model, the lift is theoretically also influenced by
– the natural return = normal percentage of buyers among your customers during a specific period
– the size of your selection in % of the customer base

As a reaction to my post, Tim Manss, in his post I’ll show you mine if you show me yours… proposed to exchange lift figures in order to be able to have something of a benchmark to check the quality of targeting models.
It is indeed not easy to get these figures, because everybody wants to keep his or her secrets … well, secret.

So I decided to give away at least some info about the lift of my targeting models by calculating a model predicting their lift.

Here is what I did :

I took the lift figures of my models (a handful of dozens of them) together with the natural return and 4 different selection sizes : 10%, 5% 1% and 0.5%
And with this simple dataset I calculated a linear regression (I actually used the logarithms of these data).

What turned out ?

– There was of cause a lot of noise : R-squared = 0.45 which means that more than half of the variance is unexplained noise. Which also means that different targets have different predictability.
– the natural return showed no statistical significant meaning
– so the only relevant predictor is the selection size.

Here is the equation and the corresponding chart (lift=ordinate, selection size=axis)

ln(lift) = 3.06291 – 0.4829 * ln(selection_size)

Lift (vertical) as a function of the selection size (horizontal)

So, I showed mine… what about yours ? 🙂

Other posts you might enjoy reading :
data mining with decision trees : what they never tell you
The top-10 data mining mistakes
Good enough / data quality
Data mining for marketing campaigns : interpretation of lift
Are you a good data miner ?

Two interesting articles of Gregory Piatetsky-Shapiro (KDnuggets) on lift modeling :
Measuring lift quality in database marketing
Estimating campaign benefits and modeling lift