Another Reason You Can’t Rely on IOMeter, FIO, or Vdbench For Big Decisions

By Peter Murray, Principal Systems Engineer

Storage testing is all about making it as real as possible, and real applications send I/O in sub-second, or what are called “micro” bursts.  Shareware tools like IOMeter and FIO were designed to test disk drives. In the days of disk drives, sub-second, or “micro” bursts were much less of a concern. With higher-speed and higher-capacity flash arrays (and NVMe arrays are not far away), bursting behavior can have a larger impact on storage arrays and therefore applications. Micro bursts cause queues to fill more and stresses array CPUs more highly than legacy all-spinning disk arrays. Enterprises who want to ensure good performance should test with micro bursts.

The creators of IOMeter and FIO have not enabled micro burst behavior. The creator of Vdbench did include the capability for micro bursts. However, the limitations of many OSes, when running under control of hypervisors, limit queue depth and therefore spread out packet transmission, especially when high numbers of threads are executing in multiple VMs. This means that even when bursts are enabled, they may not occur during testing because bursts may be defeated by the OS or hypervisor.  You don’t know.

Real initiators (clients) do not behave in this way. They burst with nearly every transmission. We learned this last year, after one vendor challenged us by saying LDX Enterprise is unrealistic. We reviewed hundreds of applications monitored by VirtualWisdom and found micro bursts in all of the data. After demonstrating our findings to array vendors and at the SNIA Storage Developers Conference, they agreed that our implementation is more realistic. After our SDC presentation, no one challenged us. In fact, several vendors thanked us for exposing the important of bursts.

Please be sure you get an honest evaluation when you test storage arrays. Micro bursts are an important element of your evaluation.

To learn more, see this excellent whitepaper by Craig Foster of Virtual Instruments.