Rock-em Sock-em robots as still life in the studio on black. Lit by a snooted li

MPLS or SD-WAN: A Systems Engineering Analysis

Southernitman, a self-identified IT manager on Reddit, recently posted his dilemma on choosing between MPLS or an SD-WAN solution, commonly referred to as WAN Virtualization. Here’s how I thought he and anyone else considering MPLS should analyze the technology.


Southernitman said on the Reddit forum that he was looking at upgrading the connectivity to his five offices. The offices are currently connected via “standard IPsec VPN [virtual private network] connections…running on business class broadband,” he said. “One office will have managed fiber soon and our PRI at headquarters will be retired in May. I run VoIP traffic across these VPN connections and haven’t encountered many issues.”

He was quoted an MPLS connection at a cost point that he knew would not be approved. “I was contacted by a Level 3 salesman who had been working with the previous IT Manager on building out a MPLS network between our 5 offices. I looked into some old emails and the original quote was for ~$5300/mo. There’s no way this is going to fly with our CFO…. “

He wanted to know “Do any of you have experience with this tech and is it a truly viable alternative budget wise to a MPLS setup when looking at reliability and continuity?”

My analysis of MPLS can be condensed down into four key areas:

– Reliable packet delivery and availability
– Class of Service / Quality of Service (COS/QOS)
– Network security through isolation
– Carrier managed site-to-site connectivity

Let’s take each of those at a time.

Reliable Packet Delivery and Availability

One of the most important pieces and biggest selling points for MPLS is the reliable delivery of packets and the ability to provide COS/QOS for your most important traffic. Reliable delivery of packets generally relates to packet loss and is particularly important when you want real-time protocols, such as VoIP, video conferencing and virtual desktops, to function without degradation in quality.

MPLS networks typically deliver reliable delivery of packets even though the MPLS networks themselves are a shared infrastructure. This is because special labels (hence the name Multiprotocol Label Switching (MPLS) are placed on your packets virtually isolate them from other customers using the same infrastructure. Nevertheless, since an MPLS network is a shared medium, you are at times competing with others for shared resources, such as bandwidth and router processing, resulting in dropped packets during times of congestion. But while an MPLS network might see .1% to 1 % of packets dropped, the Internet can experience periods of packet loss of 1% or more packets. MPLS providers also have a service level agreement (SLA) in the contract around the percentage of packets they may drop in a given period. As an unmanaged network, Internet connections come with no such SLA and therefore provide no guarantee about how much packet loss you may experience in a given period.

On top of the guarantee about not dropping too many packets, MPLS providers also offer COS/QOS buckets that you can put your most critical traffic into so that it has a higher priority for delivery and therefore a lower probability of being dropped. If you were to compare this to an Internet solution there is no guarantee of packet delivery and no built in QOS for your most critical traffic.

The SD-WAN Value

Speed lightsSD-WAN improves packet delivery across the Internet and any network on a number of levels. Before you can offer a reliable way for QOS to work, you need to make sure that the network isn’t experiencing packet loss. After all, what good is it if I guarantee a packet is sent out first from my network if it’s just dropped somewhere on the Internet?

The first thing to do is avoid packet loss altogether, many customers who chose to run their WAN over the Internet as opposed to MPLS will chose to order two Internet links from two different providers, this is an easy way to maintain 99.99% availability in case of link failure, and it also gives you two different paths onto the Internet.

From our perspective we measure per-packet loss across each of the links, if we see a link is having issues with packet loss, we can simply route traffic down the link with less loss. If however, loss is still present on both links we have another tool, Forward Error Correction (FEC), which can address the problem.

You can think of FEC as RAID for packets. As we send packets out, we actually have the ability to inject parity packets into the mix, if packets are lost in transmit we can simply rebuild the lost packet(s) at the receiving site which preserves the delivery of packets and the quality of a connection. Applying both of these techniques allows us to then apply QOS to prioritize which packets are most critical and have the highest priority for delivery, and we know that they will be reliably delivered because FEC is providing consistent delivery of all packets regardless of loss on the underlying WAN link.


From a security perspective MPLS is considered secure even though it’s a shared medium like the Internet because of unique labels appended to the packets. Only MPLS nodes looking at that tag can read the packets contained within the MPLS path. The Internet, of course, has no such mechanism opening the opportunity for the types of attacks and security holes we’ve all heard about. Traffic traversing the Internet needs to be secured with a protocol such as IPSec and firewalls should be used at gateway points into your network.

The SD-WAN Value

MPLS security has of course really been called into question over the past year with carriers clearly sharing data with the government, so the question around security might be better phrased as who do you want your data to be secure from? Data sent across an MPLS link is not typically encrypted unless you choose to do it yourself.

Security across the Internet can be easily addressed with an SD-WAN, or for that matter most devices Southernitman uses today, by establishing VPNs across the Internet. In fact, given that we now know MPLS service providers clearly share data with governments, VPNs are required regardless. It’s true that there is an additional level of security provided by MPLS, but for most organizations VPN security will be sufficient.

Site-to-Site Connectivity

Enterprise-class, site-to-site connectivity has been a strength for MPLS in large part because of simplicity. When I want to open a new office with MPLS, I only need to call the provider with the details for the new MPLS connection for the new location. The provider handles the line delivery, parameter configuration and more, delivering what’s been seen in the past a secure, enterprise-ready connection. By contrast, the Internet puts the onus of the configuration and deployment on IT.

The SD-WAN Value

From an Internet perspective, configuration is a bit more complex, but with SD-WAN it becomes simpler and far quicker than with MPLS services. You probably already have an Internet connection in the office, but if not you call up an Internet provider to provision a new line.

Unlike a managed, MPLS service, IT now needs to take over and configure the devices. But with SD-WAN this process is not only faster, the enterprise can configure the line to meet its needs. We’ve already discussed the limitations of relying on the traffic separation of MPLS for security purposes. Provisioning an MPLS line probably doesn’t take into consideration the appropriate security measures. Those are extra.

Not so with SD-WAN where security is implicit. You place a network device at the remote site and device (or software) can connect to your existing VPN infrastructure. You then configure the other sites to talk to it. Depending on the solution use this can be well automated, or a manual process.

But What About…

Aside from those four features there aren’t many other advantages MPLS has over broadband Internet. The Internet, though, has numerous advantages including global availability and less expensive bandwidth. Internet access can be a tenth of comparable MPLS costs, particularly for high-speed connections. Internet lines can be brought online much quicker as MPLS can take months to order and have delivered.

A couple of pieces of advice if you choose to go with Internet and an SD-WAN. Make sure to purchase a symmetric Internet connection, for example, 50 Mbps up / 50 Mbps down, this can help to greatly simplify the setup of QOS and will just be easier in general. For reliability make sure to order two lines from two different Internet providers. If you are going to run any real-time protocols – such as voice, video or VDI – make sure that whatever solution you go with can avoid and fix packet loss and make sure VPN and encryption are built-in. All of these things will simplify your deployment and help to make for a successful project.

You can see the original Reddit post here.

Image credit: Ariel Waldman (flickr) / CC-BY

  • MessagingAdmin

    The gap between the costs of MPLS and bare Internet are closing, so while the 10x figure you mention is indicative of older contracts, renegotiating at renewal should find costs that are more in line (perhaps 1.5x – 2x) with Internet costs, particularly in Ethernet handoff situations.
    The other situation to deal with is whether you use one provider or many on the Internet. The “no guarantees” situation becomes very bad when you have more than one provider in the mix. The finger pointing between different ISP’s when there are performance issues makes true issue resolution nearly impossible. This aspect can be minimized by sourcing from one provider, however.

    • Anthony Harrison

      In my experience, in order to provide the intelligent traffic routing and application prioritization, these SD-WAN OTT platforms generate a wealth of real-time and historical network performance data of not only the link performance but end-to-end network paths as well. That makes it so easy to see under performing service providers and provide the hard data to back-up your trouble ticket in the network vendor mix across the WAN Nationally and internationally. I’m totally empowered to choose the best priced access with dual access local provider diversity on a per location basis without a loss of control or quality since the WAN virtualization platform provides centralized management and monitoring across all locations in real-time.

  • netmagdave

    Southernitman’s experiences are his own so I can’t speak to
    them, but they align w/ what we’ve seen in the industry. In a survey of MPLS
    costs, Telegeography, which tracks MPLS pricing, noted that a 10 Mbps port in New York on average runs $608 a month. By contrast, FIOS can be
    purchased for ~$160 per month for $150 Mbps, a per megabit difference of ~60X. With
    that said, MPLS costs do vary between providers and regions. Buyers should always
    use their own data when conducting this kind of research.