Traffic Squeezer - Documentation Compression Introduction

Data Processing (Payload Handling Mechanism):
Basically the L2 layer treats its whole of its payload as an application payload. Where-as an L3/L4 suite (such as TCP/IP) treats the payload space after the L4 layer as the application payload. So, there can be a mechanism where in which we can compress this traffic within the corporate edge device and route the same to its destination link and decompress the same over there before delivering to the remote corporate LAN. In the current project the Option:2 is implemented as the solution to ensure better performance.

This is illustrated in the below picture:

Compression/Decompression Engine - (Traffic Squeezer-prealpha)
To achieve maximum compression ratio, Traffic Squeezer-prealpha proposes a design in which the compression of the network payload have been compressed with more than one loss-less compression algorithm. As one can understand this sort of dual compression will have its own drawbacks but as a proof of concept its been implemented in TS-prealpha. One can really understand how well very rarely dual compression works well having both the compression algorithms different from one another.

If the whole compression and the packet processing is done in the hardware core, then the packet processing latency will be close to wire-speeds. Rather than per-packet/per-frame compression, based on the deployment scenario, the Traffic Squeezer proposes also a possible mechanism in which set of incoming packets/frames are coalesced and do compression of the same.

As a proof of concept the current prototype's compression/decompression engine does per-frame compression for now. But always its kept in mind that Traffic Squeezer is objective is to make a more general purpose WAN Acceleration framework and suggest possible WAN acceleration methods. And also if possible a software implementation of each mechanism.

Since the current design includes couple of compression methods done in sequence, the design of this specific packet processing engine can be called as a multi-stage cascaded Compression/Decompression Engine. This is illustrated in the picture below:

The network payload comprises highly unpredictable/random data, so a lossless compression algorithm like RLE is chosen. But in some cases if the data contains any repeating patterns then such patterns are well compressed in a LZ compression algorithm. So, after compressing with RLE algorithm, the data have been sent into LZ compression engine. To decompress, first the data is decompressed with the LZ77 engine and then fed to RLE engine to get the original uncompressed source data.

If in case of any small sized frames in the current prototype version will send processed (compressed) frame if its size is lesser than the actual frame or else it sends the un-processed (actual) frame itself.

Loss-Less Compression Algorithms
To perform loss-less data compression, RLE and LZ77 compression/decompression algorithms have been used. Traffic Squeezer uses the existing open-source project BCL (Basic Compression Library Version-1.1.1, Author: Marcus Geelnard) http://bcl.sourceforge.net have been used with little changes in the APIs so that the same is more suitable to use for compression real-time dynamic network payload.

Traffic Compression Techniques

Packet/Frame Compression:
The corporate network/LAN within a specific site will have good bandwidth. But when these sites(LAN) are interlinked through the WAN links, its bandwidth is very much limited. So the same efficiency is lost when the network spans to remote geographical locations. This bottle neck is fairly reduced with the compression boxes deployed between each of these sites.

Since the payload is compressed, the user will get more effective use of the less available bandwidth. So, with the same link speeds the users will get a better perfomance. The figure below explains the overall picture of the packet flow in a compression enabled network:

RLE (Run Length Encoding)
RLE is the simplest possible lossless compression method. Nevertheless it serves a purpose, even in state of the art compression (it is used in JPEG compression, for instance). The basic principle is to identify sequences of equal bytes, and replace them with the byte in question and a repetition count.

LZ77 - (Abraham Lempel, Jakob Ziv, Year-1977)
The LZ77 compression scheme is a substitutional compression scheme proposed by Abraham Lempel and Jakob Ziv in 1977. It is very simple in its design, and uses no fancy bit level compression. The principle of the LZ77 compression algorithm is to store repeated occurrences of strings as references to previous occurrences of the same string. The point is that the reference consumes less space than the string itself, provided that the string is long enough (in this implementation, the string has to be at least 4 bytes long, since the minimum coded reference is 3 bytes long). Also note that the term "string" refers to any kind of byte sequence (it does not have to be an ASCII string, for instance). The coder uses a brute force approach to finding string matches in the history buffer. The complexity is somewhere between O(n^2) and O(n^3), depending on the input data. The upside of LZ77 is that the decompression is very fast, and the compression ratio is often very good.