Optimal design of the online auction channel: Analytical, empirical, and computational insights

Decision Sciences, Fall 2002 by Bapna, Ravi, Goes, Paulo, Gupta, Alok, Karuga, Gilbert

Trace Driven Validation

To validate more rigorously our analytical results against our simulation model, we adopted a trace-driven validation technique, as proposed by Kleijnen et al. (1996, 1998). The idea is to compare results of two temporal streams, in our case a theoretical stream of auction revenue values and a simulated stream. We are able to do this because our analysis captures the temporal dimensions of the auctions we tracked. They varied for each auction along the dimensions of lot size, price magnitude, and number of participating bidders. The trace driven validation approach tests if the two streams have identical means and variances. While details of the approach are beyond the scope of this paper, we present the main methodology below.

There is overwhelming support for the equivalence of means for all revenue comparisons. There is also overwhelming support for the equivalence of variances for the comparisons involving theoretical model with empirical observed valuations. The support goes down (R2 goes up in the fourth column of Table 6) for the comparisons of the theoretical model with assumed uniform valuations. This is not surprising because under the uniform assumption we have more loss of information since individual valuations are not used and therefore more variability. As Kleijnen et al. (1996, 1998) point out, this is the case when the two distributions do not have equivalent variances. In other words, though the central tendencies are equivalent, the distributions themselves are not. Kleijnen et al. (1998) suggest that when the results of second regression are nonconforming to the hypothesis of equal variance, individual tests or pairwise t-test should be used to further investigate the properties.

Therefore, at the individual auction level, we test whether the theoretical revenue was equivalent to the mean of the 31 simulation runs for that auction. We used a standard t-test of difference of means. Table 7 shows the number of individual auctions in which we failed to reject revenue equivalence (theoretical vs. simulated) at the 10% significance level, for the same comparisons above.

Table 7 indicates that in a large number of cases even with limited information (assuming uniform critical fractile valuations) we can at least predict the expected revenue. Overall, the results indicate that if the auctioneer has access to empirical consumer valuations, the theoretical model can be used with a high degree of accuracy to determine the optimal bid increment for the auction. If these valuations are not available, but the customer demand of the critical fractile can still be inferred through a uniform distribution, the bid increment determined by the theoretical model can still yield very good results. From Table 7, revenues generated by using such bid increments can be very accurate two-thirds of the time. From the results in Table 5, on the average, the revenues obtained by using these bid increments are off by less than 4%.

The overall significance of our findings from the exploratory data analysis and the trace driven validation is that there appears to be strong support for the use of the theoretically computed bid increment in the design of multi unit online auction. By choosing the analytically determined bid increment, auctioneers can expect to maximize their revenues. There is also an interesting cost-benefit tradeoff between the cost of acquiring the information necessary to plug into the model (the p estimate) and the corresponding revenue benefit.