Anthony Caputo

4G LTE Load Balancing and Failover

Blog Post created by Anthony Caputo on Jan 29, 2016

There are a few devices on the market that can provide network failover and load balancing multiple Wide Area Network (WAN) connections. The theory is, if you lose one of multiple WAN connections, the router seamlessly failover to a secondary and/or third WAN connection to transmit and receive crucial data. It can also load balance bandwidth from all the connections, which if using 4G LTE can provide relief from extensive public usage and saturation.

load balancing road.jpg

I've researched and configured a few devices. I've been primarily focusing my efforts on the Cradlepoint IBR1400 and the IBR600, along with the Pepwave MAX 700 at two locations. In order to configure these devices properly, you need at least two WAN connections.

 

My first location was an isolated cottage, with some perseverance, invoked AT&T to install a Uverse Internet connection for $40 per month. The infrastructure limited the bandwidth capabilities to up to 12Mbps download (excellent for Netflix), and only 1Mbps upload (poor for video surveillance video). In may not appear to be such a monumental task to get wired nowadays, but this isolated cottage was on a small secluded island, in the middle of a river. It was well worth the effort as wired broadband connections are sold as infrastructure and not by bandwidth or data usage such as the cellular carriers. The data usage costs for uploading security video (or download video) can be astronomical.  Even wireless 4G LTE service providers, with an unlimited data plan does not include uploading a continuous bandwidth intensive stream of synchronous security video. Experience has proven that at peak capacity, because it's a shared public network, bandwidth will be throttled, reducing quality and performance.

 

This cottage provided free reign to temporarily install anything, on any pole or structure without city ordinances, codes or permits and privacy issues, as it was secluded and used infrequently. I've installed four cameras to test this new failover and load balancing test case. Power was a challenge as usual, but nothing an upgraded circuit breaker and some trenching couldn't handle.

 

I decided to install various cameras for comparison. My analog Pelco Spectra IV PTZ workhorse, connected to a digital video encoder; a Hikvision 1080p PTZ; a Hikvision Darkfighter 1080p PTZ, and a fixed 3MP Longse IP camera, with IR. All together, these four cameras, to get optimal quality require a minimum of 10Mbps upload through a WAN connection to view remotely. I had to trim down the frame rates substantially and got it down to 5Mbps. So, now how do I get a 5Mbps elephant through a ATT Uverse 1Mbps fire hose?

 

The Milestone XProtect server received the full 15fps at maximum resolution through the LAN, the Pelco using MPEG4 and the Hikvision and LongSe using H.264 without any issues. The video stream on the Pelco Spectra IV was set to the maximum analog 4CIF resolution and performed as expected. Sharp, smooth and exceptionally well at low light. By comparison, next to the Hikvision HD PTZ 1080p megapixel video stream, the image quality was found wanting. The Hikvision Darkfighter image was crisper, with exponentially far more details in the area-of-coverage, including color correction, leaves, grass, and even wood grain and the best low light capabilities without IR I've seen to date. However, the pan-tilt-zoom on the Pelco out-shined the clunky Hikvision. This probably has something to do with the TCP commands for PTZ (versus the UDP streaming for video) and the processing power behind streaming only 4CIF versus 1080p.

 

The 3MP LongSe camera was powered by a PoE injector (all the equipment is in climate controlled LCOM enclosures). I have another one of these cameras powered using a PoE Injector at the other location, with impressive image quality, although with a maximum frame rate of 15fps. Of course, I have Comcast/Xfinity internet at that location, with 20Mbps upload bandwidth.

 

The math identifies its connectivity issues are not switch or power related, but the limited ATT Unverse 1Mbps upload speed, which barely gave me a single frame per second for the cameras, and erratic PTZ controls. A 3MP camera, at full resolution, using H.264 30% compression requires about 900Kbps for a single frame. A 1Mbps upload pipe doesn't leave much room for anything other than a signal frame from the 3MP camera.

 

Obviously, seemed like a perfect location for some load balancing and failover testing using a 4G LTE network or two. Started with the Cradlepoint AER2100 Multi-WAN Router, using swappable USB modems, with exceptional horse-power, but it's 8" x 10" x 2" size wouldn't fit into the enclosure. The smaller COR IBR600 was the right size, but included an integrated modem, limiting flexibility. The Cradlepoint MBR1400 provided a smaller form factor, and swappable USB modems ports. However, unlike the AER2100 or IBR600, the MBR1400 wasn't a hardened device with extended temperature specifications, so I had to upgrade the enclosure for extended temperatures.

 

CP.jpg

Cradlepoint MBR1400


Would’ve like to have used the Pepwave MAX700, but I grew too fond of it in my home lab for testing platforms and devices. The Pepwave MAX700 includes two separate wired WAN inputs and up to four USB modems inputs.

 

pepwave.jpg

Pepwave MAX700

Below is the configuration using the ATT Uverse as the primary WAN connection, an ATT 4G LTE  and a Sprint 3G/4G connection (limited bandwidth in the region) for failover and load balancing. The ATT 4G LTE and the Sprint 3G/4G are configured as “on-demand,” which limits data usage to only when needed.

 

Cradlepoint configured with three load balancing and failover WAN connections

 

The WAN connection bandwidth demand jumps when streaming from my Milestone video management server to view the camera remotely, especially when viewing all four cameras at once. Now, thanks to the load balancing, I’m able to get some crisp imagery with smoother control of the PTZ cameras.

Data usage example chart


Incidentally, when I tested the failover feature, by unplugging the ATT Uverse wired connection, it took me a while to realize that it worked. It was that seamless.

 

Remote Live View of cameras using Milestone Xprotect Web Client

Outcomes