Trhoughput of Flowscan..

Date view Thread view Subject view Author view

Subject: Trhoughput of Flowscan..
From: Alex (alex@nac.net)
Date: Sun Mar 11 2001 - 11:01:16 CST 

After a couple of weeks tinkering with Flowscan, I figured I'd share what
I found.

Today, we're only netflow'ing one router, a 7507 thats moving an aggregate
of about 150 mb/s. The overall can be seen at:

http://netflow.nac.net/flowscan/io_services_bits.png

We first tried to run this on a somewhat already busy dual p3-700 with 2
gigs of RAM. The machine couldn't keep up; took about 400 seconds for each
5 minutes of data.

So, I ordered me up a new p3-933 with 256 megs, 40 gig UDMA IDE, and put
fbsd 4.2 on it. This one can process the flows in about 150 seconds
initially.

Then, we turned on SubnetIO. It increased to about 160 seconds.

Then, I thought of something whack. We have three main assignments
(207.99/17, 209.123/16, and 64.21/17). All of the assignments are stored
in a dynamically updated SQL table. Why not create our_subnets.boulder
that had all of the assignments, thus creating a TopN flow report for
every customers netblock? Well, I did (has anyone else done this?)

netflow:/usr/local/flows/bin# cat our_subnets.boulder | grep SUBNET | wc -l
    1386

With this, the processing time has grown significantly:

2001/03/11 11:47:20 working on file flows.20010311_11:46:58-0500...

2001/03/11 11:50:47 flowscan-1.020 SubNetIO: Cflow::find took 207
wallclock secs (184.94 usr +  0.51 sys = 185.45 CPU) for 43032055 flow
file bytes, flow hit ratio: 675772/782401

2001/03/11 11:51:56 flowscan-1.020 SubNetIO: report took 69 wallclock secs
(24.95 usr +  0.77 sys = 25.72 CPU)


The only thing odd we're seeing is:


Mar 11 01:15:58 netflow cflowdmux[7356]: [E]
semop(327680,0xbfbff078,1) failed to release buffer lock: Resource
temporarily unavailable {CflowdPacketQueue.cc:438}

Anyone see this?




--
Help        mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe flowscan" in message body
Archive     http://net.doit.wisc.edu/~plonka/list/flowscan/archive/

Date view Thread view Subject view Author view

This archive was generated by hypermail 2b25 : Sun Mar 11 2001 - 10:59:38 CST