Uncork the performance of caching appliances - Technology Tutorial - Tutorial

Communications News, May, 2001 by Edward Sharp

Lab environment. The upside is you control the test completely, and you can measure the actual performance of the box. The lab provides a controlled environment with no risk of impacting streaming operations, and the ability to vary parameters and troubleshoot performance-affecting variables. The downside is you cannot be sure that your simulation represents the real world, and you may have difficulty simulating a large number of users. Setting up a test involving the 15,000 users supportable by a single streaming server obviously is not feasible. If you choose lab testing, you need to build a suitable environment.

Broadcast-mode tests in a lab environment require a live stream as a content source, a switch to connect the edge server, the edge server, and a means of simulating the required number of users. Load simulators may be available from streaming format vendors. They vary in flexibility in terms of usage patterns that simulate real-life loads (for example, each user requesting a unique on-demand file on a unique playing schedule), and in the number of users a given instance of the load simulator can simulate.

An instance may be limited to a few hundred users. If you are testing an edge server with a 10,000-stream capability, you will need a large number of simulator servers, but they would be far more manageable than 10,000 users.

What if you simply cannot set up the number of simulators you need to reflect your user population? You can test the streaming server at less than full capacity, measuring the known internal bottleneck (such as CPU load), then extrapolate those results to estimate the server's total streaming capacity.

There are pitfalls here, however. Extrapolation will not reveal unknowns that may crop up when the server is pushed to its limits. In addition, without full-load testing, predicting which bottleneck (for instance, disk I/O, the CPU, or network interface cards) will cap the total performance of the server may be difficult.

TRIANGULATING RESULTS

When your results have been recorded and analyzed, you should compare them with information from other industry sources. Third-party testing houses can supply equipment performance data that will raise your confidence level in your test records. The server vendor should be willing to supply reference contacts to other customers who are willing to share performance data on the equipment you are testing.

Companies whose primary business success depends on high quality of service, and who plan to deploy many servers, should definitely do their own testing internally. This may seem complex and time-consuming, but the alternatives are vendor-provided specifications, word-of-mouth or independent lab testing.

If you choose not to test your server technology, remember that your deployment and user patterns may have a large impact on the performance you achieve. Review the variables you expect with your vendor to estimate server performance. Understanding these issues before purchasing streaming servers can go a long way toward ensuring that your servers meet your expectations.