Online marketing research

IBM Journal of Research and Development, Sep-Nov 2004 by Agrawal, A, Basak, J, Jain, V, Kothari, R, Et al

Figure 1 provides an overview of the chain of events. The overall flow begins with the specification of an objective on the part of the merchant-for example, "determine the demand as a function of price," or "find the effect on the sales of Brand B caused by a discount on Brand A," and so on. The marketing and response variables are identified, and a data-gathering activity is initiated. We call each such data-gathering activity an "experiment"; the deployment of an OMR experiment may utilize other subsystems of a commerce server. The OMR experiment then changes the marketing variable for selected visitors to the Web site and assigns each selected visitor to a group to create matched control and experimental groups (explained in greater detail in Section 3). The visitors (users) can be selected on the basis of their clickstream (navigation pattern), along with the user's historical transactions (if the user is logged in) and other information. Nonetheless, the use of the clickstream provides a way of selecting a user even when the user is simply browsing as an anonymous user without actually logging in.

The response of each selected user is measured either explicitly or implicitly. The experiment can execute for a prespecified time period, or an information-theoretic criterion can be used to terminate the experiment when the gain in information from additional data collection falls below a certain threshold.

For example, the control group may not see anything related to the product, and there may be three matched groups which are offered e-coupons with a discount value of 5%, 10%, or 20%. The differential response of the groups then provides a basis on which to extrapolate the demand at various price points.

3. Foundations of systematic online marketing research

Several innovative features required to make OMR systematic, rapid, and cost-effective are described in the sections that follow.

Matched control and experimental groups

A significant part of OMR is driven by observing the change in response resulting from a change in the marketing variable (price, bundling of products, and so on). This requires that the possible effect of any variable other than the changing marketing variable be removed. OMR achieves this through the use of matched control and experimental groups. To illustrate how this increases the accuracy of the inferences made, suppose that a merchant wishes to ascertain the demand at various price points. Say, multiple groups of customers are formed on the basis of random selection, and each group is offered the product at a certain price. The difference in the overall response of the various groups cannot be attributed entirely to the change in price (unless the sample size of each group is large). This is because differences in population characteristics also contribute to the difference in the response of the groups.

Using the above strategy ensures that the groups are "matched" in the sense that for each user in a group, a similar user exists in each of the other groups. A marketing variable can thus be changed between the groups, and the effect of the change can be measured directly from the difference in responses of the groups (the groups are similar to each other, with the single exception that they are exposed to a different marketing variable). Clearly, it is more difficult to do this matching with the more traditional forms of marketing research, in which implicit information such as clickstream is not readily available.