With today’s post I am starting a repeat series of “from the trenches” reports. Over the years, I have come across numerous examples of so-called “aha-moments” that I think are worth sharing.

When analyzing corporate messaging systems you can observe many consistent trends, but every now and then, you will notice a peculiar anomaly. Those are the times when in the middle of a presentation a person in the back of the room stands up and proclaims: “I told you that something strange was going on, but nobody would listen to me.” However, these stories do not just make good anecdotes they make you think and wonder if something like this might be happening in your corporate messaging environment as well, potentially costing your company unnecessary time and money.

The need for more bandwidth
A few years back a large financial services company hired us to perform a detailed analysis of their networking traffic. At the time, the European based bank had completed a number of mergers with smaller firms in South America. Consequently, their corporate email environment had grown substantially and transatlantic network traffic had multiplied to a point where the company’s private network link was consistently overloaded. The network admin staff naturally looked at the traffic packets and ports and concluded that email traffic was to blame (sounds familiar?) for using-up most of the bandwidth.

Email being an essential business tool, of course, triggered the procurement process of “bigger pipes” with an initial cost estimate of about $2MM in extra telecom expenses annually. That’s when somebody (smart) at the firm suggested to take a deeper look at the actual data causing the traffic.

Our analysis of the banks entire messaging environment very quickly made a fundamental discovery. The email traffic causing the network overload was actually mostly messaging traffic between branches and subsidiaries of the bank in South America, all routed through the bank’s main data center in Europe, traversing the transatlantic link twice. The explanation for this seemingly crazy configuration was the fact that as the bank acquired other companies over time, a direct network connection to headquarters was quite logical. However, as the number of physical locations in that particular geography grew that initial network topology with distant hub and spoke routing needed some revision.

The solution to the perceived bandwidth problem was actually quite simple in the end. Most of the South American branches already had some rudimentary interconnectivity. For the branches that were not connected, the cost of provisioning the appropriate services was marginal compared to the proposed overseas upgrade. After looking at the actual email traffic patterns the email administrators together with the network staff revised the mail routing topology resulting in the cancellation of the expensive network upgrade.

What are the lessons learned?

• Corporate messaging systems are constantly changing (user behavior, mergers, relocations, consolidation, etc)
• Port based traffic analysis alone does not provide sufficient insight into the root cause
• Ongoing traffic monitoring and trending should be part of the corporate IT strategy

Technorati Tags: email traffic, email routing, network topology, email traffic patterns, network cost

If you are new here, you may want to subscribe to my RSS feed. An RSS Subscription will deliver new Blog posts automatically to your computer.
Thanks for visiting!