[Discuss] Network Traffic Visualization

[bill at horne.net (Bill Horne)]

On 2/1/2012 3:40 PM, Daniel C. wrote:
> As we discussed in another thread, I'm taking a class on data
> visualization and for my final project I plan to write a piece of
> software that will create a visualization of network traffic.  I have
> a few questions that will help me get started:
>
> - What problems do you have that a visualization tool could help
> solve?  Are they well defined ("I need to keep an eye on http requests
> in case we get Slashdotted"), or is it more nebulous ("I just want to
> see what's going on in my network")?

Usage spikes, especially for non-standard ports. "Well defined" isn't 
well defined, sad to say: managers tend to brush off anything that 
causes employees to deviate from the script, so it's important to have a 
clear, concise guide for them to follow.

>
> - What do you need to see in order to solve the problem(s)?

The information that will allow the troubleshooter to silence an 
infected or compromised machine quickly. Phone numbers aren't useful: 
users resist any order from someone they don't know. What the guy in the 
hot seat needs is the addressing, control, and security information that 
will allow him/her to drop any given machine out of the network. If your 
network isn't equipped with addressable/programmable Ethernet switches, 
then your system needs to show the lowest-level router that has 
MAC-address filtering capability, or the firewall that can embargo a 
certain port or IP, and (failing that), the closet and jack number that 
the tech can run to.

If you figure out how to keep this information up-to-date without 
needing to have a full-time data librarian, please tell me.

>
> - Do you have any preference for how you see it?  For example, will
> you have a single monitor (or projector screen, etc.) just for this
> data, or will it be a single window that you check on occasionally,
> but is otherwise minimized?
It depends on the personnel that are assigned to the monitoring job. If 
they are marginally trained, then you'll need specific pop-ups that 
can't be dismissed or minimized, and which demand specific action and 
supply all the information needed to accomplish the task. More 
experienced staff members will prefer separate windows that have 
color-coded borders for severity, but which can be "put on hold" while 
key players are paged or in transit.

>
> - What workflows are currently in place to tackle the problems that
> could be improved by having access to a visualization tool?

By "workflows", I assume you mean "applications". They run the gamut 
from having "tail" applications on system logs, to email alerts for 
high-water warnings. The short answer is "all of them": a visualization 
tool is mostly useful to create *RELATIVE* indicators that show unusual 
traffic flows, but is also very useful to alert employees to exceptional 
events such as foreign DNS access, web traffic to non-standard proxies, 
IRC traffic, shotgun emails, etc.

> Thank you all in advance!  Hopefully I'll be able to produce something
> genuinely useful.

When you do, copyright it before you tell anyone or show it to anybody. 
Don't ask me how I know.

Bill

-- 
Bill Horne
339-364-8487