While there have been plenty of articles and buzz of late about the power of big data and even how WAN optimization will benefit big data, there’s been surprisingly little about what exactly constitutes a “Big Data” WAN optimizer. Yes, extreme scalability and resiliency are critical, but perhaps equally important is the use of off-the-shelf hardware.

However, we get ahead of ourselves. While there’s little consensus on how big is big data, most industry observers seem to agree that these databases are radically larger than what’s currently used within the organization, extending into the terabytes — and even petabytes — of information. Moving this volume of data across the WAN poses significant challenges because of the bandwidth of the wire as well as the latency between the sites, explains Everett Dolgner, Silver Peak’s director of storage and replication product manager.blog

Scaling to the necessary wire speeds is obviously going to be challenge for many WAN optimizers. As heretical as it might sound, though, even with those performance challenges, proprietary hardware architectures have no role in the data center-class WAN optimizer. Yes, I know, there’s this conception that specialized hardware is all but synonymous with superior performance and all, but that’s a carryover from the early days of our industry. The processing power and I/O performance of today’s servers coupled with well-designed software is more than sufficient to power the data center. And since these are standard PC architectures, organizations continue to ride the PC’s price-performance curve.

Silver Peak’s NX-10K, for example, is the highest capacity WAN optimizer in the industry – 2.5 Gbps of encrypted, optimized WAN traffic with support for a leading 256,000 IP flows. With so many flows, the 10K is able to consolidate thousands of network clients or, in more exotic cases, big data updates from tens of thousands of sensors and actuators in the field. Yet open up the NX-10K or any of our appliances and you won’t find custom ASICs (Application-Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), or any other purpose-built silicon from Silver Peak. Just standard, server hardware tuned for WAN optimization.

Besides cost savings, insisting on standard server hardware sets the stage for the next step in data center-class WAN optimization: virtualization. The same code base that runs on the commercial severs can be made to run on standard hypervisors. Our Virtual Acceleration Open Architecture (VXOA), which drives the NX-10K, for example, is also the basis of our VRX-8, the highest capacity, virtualized WAN optimizer on the market.

Virtual appliances have other benefits to offer beyond the lower costs of commercial iron. If you’re organization is like most, virtualization is probably a strategic direction in the data center. This probably means that processing cycles for running a virtual appliance might be copious, good luck getting more rack space to deploy a physical appliance. Virtual appliances can also be easily moved between networks and sites through software. Not so with physical appliances.

Custom hardware might have played a valuable role in the early days of networking, but not today. The power of today’s commercial server iron and the adoption of virtualized appliances make big iron a bad choice for big data.