The Skinny on Fat Pipes (Part 1)

If you’re like most enterprises, you face critical issues dealing with the growing demand for high-capacity WAN connections. These networks  are critical for disaster recovery, server centralization and many other initiatives that increase the amount of data going over WANs.

Known as “fat pipes” — high-capacity WAN connections that support 45 Mbps or higher throughput — these networks provide have unique performance, scalability, and configuration challenges that affect application delivery. More importantly, they affect end-users’ experiences, which is why it is imperative to take these issues into consideration when deploying WAN optimization solutions.

Finding the Fat

Typically, fat pipes are deployed between large data centers, disaster recovery facilities, regional hubs, and in large branch offices with skilled remote employees. Because of this, they seamlessly need to handle:

  • Varying bandwidth
  • Large data volumes
  • Diverse applications
  • Complex configurations
  • Security issues

To complicate matters, the size of WAN links has also changed over the years, becoming higher capacity as bandwidth becomes cheaper.  I’ve always said that “today’s DS-3 is tomorrow’s OC-3”, which appears true as the number of high-capacity links in enterprises has grown steadily over the years.

But don’t fear! WAN optimization techniques have evolved to keep pace with these changes.  One such area, which we address here, is WAN deduplication.

Unduplicated Deduplication

In recent years, traditional compression, QoS and TCP acceleration techniques have been augmented with disk-based data reduction (e.g., WAN deduplication) in order to improve the throughput of ever-increasing and ever more congested WAN bandwidth.

Basically, disk-based data reduction incorporates dedicated local drives on WAN optimization appliances in each location, such as the data center or disaster recovery facility. These appliances monitor all communications in real time, “fingerprinting” data sets and storing a single instance of each piece of information locally for future reference. So, when duplicate data is sent between communicating devices, the appliances detect these duplications and send a reference (or “mirror image”) across the WAN rather than the actual data — speeding the transfer. The changed data is delivered from the local data store on the far-end appliance.

However, disk-based data reduction on high-capacity WAN connections comes with its own set of unique challenges because it involves many parallel reads and writes to disk at high speeds. For enterprises, this means they need a WAN optimization solution that provides an efficient fingerprinting algorithm and significant hardware processing power to complete these processes — and complete them seamlessly. It also means a significant increase in storage capacity on the WAN optimization appliances to handle the large volumes of data and finding an efficient way to index and store data in order to access and retrieve it over time.

Exercising the Fat Pipes

In addition to these challenges, high-capacity WAN connections must support real-time traffic, such as data replication, SQL, voice, video, etc. While most applications run over TCP, many applications also use UDP, so scalable data reduction solutions must be application-agnostic, regardless of transport protocol. Real-time traffic and applications, such as data replication, are adversely affected not only by latency, but also when latency fluctuates, so the fingerprinting process should not add more than a few milliseconds of latency and deliver consistent (i.e., predictable) latency throughout the course of operations.

Finally, different WAN deduplication solutions provide different levels of accuracy. WAN optimizers that utilize packet-oriented architectures, like Silver Peak, can detect variable length redundant patterns down to byte-level resolution. As an added bonus, these solutions also incur low latency and deliver high data reduction results. In contrast, WAN optimizers that conduct pattern-matching based on a fixed block size cannot attain the granularity that packet-oriented solutions can when matching patterns.

Stay tuned for Part 2 of “The Skinny on Fat Pipes” series next week to see how you can eliminate, or at least mitigate, other WAN issues, such as latency and packet loss.