Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was "Re: RRD files generated by FlowScan have no data")
From: Ahsan Khan (ahsank@one.net.pk)
Date: Sun Oct 01 2000 - 18:19:48 CDT
Dear Todd,
Thanks a lot for your guidance, I have almost done it and i can now
see my Graphs.
I have one question, ie. I have bandwidth in Kb not Mb and my Graphs are
showing it in MB, Is there any patch that i can reduce the Scale of
Calculation ,??
With Regards
Ahsan Khan
Sr. System Admin
Internet Division (OneNet)
Sun Communication Pvt. Ltd.
Pakistan
http://www.one.net.pk
----- Original Message -----
From: "Todd Caine" <todd_caine@eli.net>
To: "Ahsan Khan" <ahsank@one.net.pk>
Sent: Friday, September 29, 2000 3:42 AM
Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was "Re: RRD files
generated by FlowScan have no data")
> Good. So you now have flow files being created!
>
> Now, in your FLOWDIR you should see a couple of different types of files:
>
> 10.0.0.1.flows.0
> 10.0.0.1.flows.1
> 10.0.0.1.flows.2
> 10.0.0.1.flows.3
> 10.0.0.1.flows.4
> 10.0.0.1.flows.5
> 10.0.0.1.flows.6
> 10.0.0.1.flows.7
> 10.0.0.1.flows.8
> 10.0.0.1.flows.9
> flows.20000928_15:24:50-0700 <---- These are what flowscan will
use
> flows.current
>
> If you don't see flows.current then you aren't using the version of cflowd
> mentioned on the Flowscan INSTALL file. If you are using the correct
version of
> cflowd, and have also applied both patches mentioned in FlowScans INSTALL
file,
> then you probably aren't running cflowd with the proper flags. Cflowd
should be
> ran like:
>
> cflowd -s 300 -O 0 -m /path/to/cflowd.conf
>
> Running cflowd like this should create flow files every five minutes.
These
> five minute files are what you want flowscan to look for. I would suggest
> changing the FlowFileGlob directive found in you flowscan.cf file back to
> /path/of/flow/files/flows.*:*[0-9]. You do not want to match the
> ip.address.flow.# files. This should get you processing the correct
files. It
> is very important that you are using the correct version of cflowd and
have
> applied the patches mention in the INSTALL file for FlowScan. This should
do
> the trick as long as you have flowscan.cf configured correctly, along with
> CampusIO.cf if that's the reporting module you are using.
>
> I wish you were in the US, so that I could ask you for a beer sometime.
;)
>
> Cheers,
> Todd Caine
>
>
>
>
> Ahsan Khan wrote:
>
> > Dear Todd,
> >
> > Thanks for the detail help, i am also confused that what could
be
> > wrong, i am here sending you my cflowd.conf file and other things to see
> > what could be wrong.
> >
> > OPTIONS {
> > LOGFACILITY: local6
> > TCPCOLLECTPORT: 2056
> > TABLESOCKFILE: /usr/local/arts/etc/cflowdtable.socket
> > FLOWDIR: /u/cflowd/cflowd
> > FLOWFILELEN: 1000000
> > NUMFLOWFILES: 10
> > MINLOGMISSED: 300
> > }
> >
> > CISCOEXPORTER {
> > HOST: 209.58.78.1 # IP address of Cisco sending
data.
> > ADDRESSES: { 207.45.215.202 }
> > CFDATAPORT: 2055 # Port on which to listen for
data.
> > SNMPCOMM: 'public'
> > COLLECT: { protocol, ifmatrix, portmatrix, netmatrix, nexthop,
tos,
> > flows
> > }
> > }
> > COLLECTOR {
> > HOST: 209.58.78.10 # IP address of host running cfdcollect
> > ADDRESSES: { 127.0.0.1, 209.58.78.10 } # other addresses of host
> > AUTH: none
> > }
> >
> > The next step i have done is that i gave flowdir
/usr/local/arts/data/flows
> > and after that i have got the flow files now. which have the name
started
> > from my router ethernet ip address,
> >
> > But still after running flowscan there is no processing and rrd files.
even
> > i have changed the option to
> > /usr/local/arts/data/flows/flows.*:*[0-9]
> >
> > The Output of my syslog when i started cflowd and cflowdmux with
> > ../etc/cflowd option is
> >
> > Sep 29 03:20:08 mail cflowd[11475]: [I] cflowd (version cflowd-2-1-a9)
> > started.
> > Sep 29 03:20:08 mail cflowd[11475]: [I] got semaphore: id 0
> > Sep 29 03:20:08 mail cflowd[11475]: [I] attached to 1052672 byte packet
> > queue at
> > 0x40366000
> > Sep 29 03:20:23 mail cflowdmux[11505]: [I] cflowdmux (version
cflowd-2-1-a9)
> > sta
> > rted.
> > Sep 29 03:20:23 mail cflowdmux[11505]: [I] created 1052672 byte packet
queue
> > shm
> > em segment {CflowdPacketQueue.cc:247}
> > Sep 29 03:20:23 mail cflowdmux[11505]: [I] attached to 1052672 byte
packet
> > queue
> > at 0x4017c000
> > Sep 29 03:20:23 mail cflowdmux[11505]: [I] created semaphore: id 0
> > Sep 29 03:20:23 mail cflowdmux[11505]: [I] set UDP recv queue to 261040
> > bytes fo
> > r fd 4 (port 2055)
> >
> > Ok after an hour while i am writing you this email i have done some
> > modification in and found that the basic fault is in
> > FlowFileGlob option so i just modified it to
> >
> > FlowFileGlob 209.58.78.1.flows.*
> >
> > and run the flowscan in the dir where my flow files started to creat.
> >
> > but after running flowscan i have one .rrd file in my graph dir and
after
> > that the following message comes up
> >
> > [1] 10969
> > [root@mail cflowd]# Loading "/u/flows/bin/Napster_subnets.boulder" ...
> > Loading "/u/flows/bin/local_nets.boulder.209" ...
> > Loading "/u/flows/bin/local_nets.boulder.216" ...
> > Loading "/u/flows/bin/local_nets.boulder.207" ...
> > Loading "/u/flows/bin/local_nets.boulder.203" ...
> > 2000/09/29 03:13:02 working on file 209.58.78.1.flows.0...
> > 2000/09/29 03:13:02 %CampusIO::FTPSession -> 0
> > 2000/09/29 03:13:02 %CampusIO::FTPSession -> 0
> > 2000/09/29 03:13:02 %CampusIO::NapServer -> 0 %CampusIO::NapUser -> 0
> > 2000/09/29 03:13:02 %CampusIO::NapServer -> 0 %CampusIO::NapUser -> 0
> >
> > [root@mail cflowd]# 209.58.78.1.flows.0: Invalid index in flow data
file: 0!
> > Ve
> > rsion 5 flow-export is required with *all* data being saved using the
> > COLLECT fi
> > eld of the CISCOEXPORTER stanza(s)!
> > 2000/09/29 03:13:19 flowscan-1.017 CampusIO: Cflow::find took 17
wallclock
> > secs
> > (15.49 usr + 0.14 sys = 15.63 CPU) for 1000000 flow file bytes, flow
hit
> > ratio:
> > 11384/15768
> >
> > [1]+ Segmentation fault (core dumped) /u/flows/bin/flowscan
> >
> > What i believe now is there is something wromg with my FlowFileGlob
> > options but as i am not that much expert in perl i can not determin what
it
> > should be ,
> >
> > so as a good student i have analyze all the case and have th out put and
i
> > believe i am near to fix it can you help me in it. I will wait for your
> > answer in this regard.
> >
> > I really appriciate your help.
> >
> > With Regards
> > Ahsan Khan
> > Sr. System Admin
> > Internet Division (OneNet)
> > Sun Communication Pvt. Ltd.
> > Pakistan
> > http://www.one.net.pk
> >
> > ----- Original Message -----
> > From: "Todd Caine" <todd_caine@eli.net>
> > To: "Ahsan Khan" <ahsank@one.net.pk>
> > Cc: <netflow@doit.wisc.edu>
> > Sent: Friday, September 29, 2000 1:42 AM
> > Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was "Re: RRD
files
> > generated by FlowScan have no data")
> >
> > > Ahsan,
> > >
> > > What I was trying to demonstrate is that the source address for the
> > flow-export
> > > packets must match the IP address that you have configured in you
> > cflowd.conf
> > > file. We use a Loopback address so that the flow export isn't
dependent
> > on an
> > > individual interfaces IP address. I'm not necessarily suggesting that
you
> > use a
> > > Loopback address. If you want to, you need to do the following:
> > >
> > > conf term
> > > int Loopback0
> > > ip address your.ip.address.here your.subnet.mask.here
> > > ctrl-z
> > > ctrl-z
> > >
> > > I'm not sure why you are not getting raw flow files. The output of
your
> > show ip
> > > flow export command seemed to be good. So you are sending the udp
> > packets.
> > > Make sure that you don't have a firewall or access-lists blocking the
udp
> > > traffic from your router to the host that cflowd is running on.
> > >
> > > The steps that I would take to solve this is:
> > >
> > > 1. Check to make sure cflowdmux and cflowd are running by doing a
'ps -ef
> > |
> > > grep cflowd' if you see them both then continue, otherwise let us know
and
> > we
> > > can trouble shoot that part.
> > >
> > > 2. Setup your /etc/syslog.conf file for local6.debug so that you can
see
> > what
> > > is happening on start up. So now stop cflowdmux and cflowd and then
start
> > > cflowdmux first followed by cflowd. Now check the log file. You
should
> > get
> > > some messages. If the log file is blank, you should resolve your
syslog
> > > problems first( you can use 'logger -p local6.debug "Testing"' to make
> > sure
> > > syslog is working properly for the local6 facility )
> > >
> > > 3. Check to see if you have flow files. The files will be located in
the
> > > FLOWDIR directory which is configured in your cflowd.conf's OPTIONS
> > stanza.
> > > Normally, this will be /usr/local/arts/data/flows. Make sure that the
> > > /usr/local/arts/data/flows directory structure exists, if not create
them.
> > If
> > > you do not have any flow files continue.
> > >
> > > 4. Make sure that cfdcollect created the semaphore and that it still
> > exists.
> > > You can verify this with the 'ipcs -a' command. The output of this
> > command
> > > should show something similar to the following:
> > >
> > > IPC status from <running system> as of Thu Sep 28 13:16:13 PDT 2000
> > > Message Queue facility inactive.
> > > T ID KEY MODE OWNER GROUP CREATOR
CGROUP
> > > Shared Memory:
> > > m 1300 0x9c0 --rw-r--r-- netflow vegemite
> > > netflow vegemite
> > > T ID KEY MODE OWNER GROUP CREATOR
CGROUP
> > > Semaphores:
> > > s 0 0x9c0 --ra-ra-ra- netflow vegemite
> > > netflow vegemite
> > >
> > > Notice that I have an entry for shared memory and a semaphore to
access
> > the same
> > > shared memory segment. If you don't have output simular to this, let
us
> > know.
> > > If you do then cflowdmux seems to be working correctly.
> > >
> > > 5. If all is well thus far, do a 'netstat -an' and you should see an
> > entry such
> > > under UDP like:
> > > *.2055 IDLE
> > >
> > > If you don't see this your flow-export datagrams aren't reaching your
> > server. I
> > > could go on and on, but try these steps and let us know what happens.
> > >
> > > Hope this helps.
> > > Todd Caine
> > >
> > >
> > > Ahsan Khan wrote:
> > >
> > > > would you please let me know how to configure loopback0 in Cisco or
do
> > you
> > > > mean that i need to do nothing else in router, if so then what will
be
> > my
> > > > loopback address.??
> > > >
> > > > With Regards
> > > > Ahsan Khan
> > > > Sr. System Admin
> > > > Internet Division (OneNet)
> > > > Sun Communication Pvt. Ltd.
> > > > Pakistan
> > > > http://www.one.net.pk
> > > >
> > > > ----- Original Message -----
> > > > From: "Todd Caine" <todd_caine@eli.net>
> > > > To: "Ahsan Khan" <ahsank@one.net.pk>
> > > > Sent: Thursday, September 28, 2000 10:17 PM
> > > > Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was "Re:
RRD
> > files
> > > > generated by FlowScan have no data")
> > > >
> > > > > For my setup I also have in global configuration:
> > > > >
> > > > > ip flow-export source Loopback0
> > > > >
> > > > > Then in my cflowd.conf file I have the Loopback0 address of the
router
> > > > sending
> > > > > flow-export datagrams.
> > > > >
> > > > > -todd
> > > > >
> > > > > Ahsan Khan wrote:
> > > > >
> > > > > > I have already sent you the Command output and here is my Router
> > config
> > > > > >
> > > > > > Global
> > > > > > ip flow-cache timeout active 1
> > > > > > serial interface
> > > > > > ip route-cache flow
> > > > > > Global
> > > > > > ip flow-export version 5 peer-as
> > > > > > ip flow-export destination 209.58.78.10 2055
> > > > > >
> > > > > > With Regards
> > > > > > Ahsan Khan
> > > > > > Sr. System Admin
> > > > > > Internet Division (OneNet)
> > > > > > Sun Communication Pvt. Ltd.
> > > > > > Pakistan
> > > > > > http://www.one.net.pk
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > From: "Todd Caine" <todd_caine@eli.net>
> > > > > > To: "Ahsan Khan" <ahsank@one.net.pk>
> > > > > > Cc: <flowscan@net.doit.wisc.edu>
> > > > > > Sent: Thursday, September 28, 2000 4:30 AM
> > > > > > Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was
"Re:
> > RRD
> > > > files
> > > > > > generated by FlowScan have no data")
> > > > > >
> > > > > > > Log into the router that you are trying to get flow data from
and
> > send
> > > > us the
> > > > > > > output of a 'show ip flow export' command. Accompany this
with
> > your
> > > > flow-export
> > > > > > > IOS configuration.
> > > > > > >
> > > > > > > tOdd
> > > > > > >
> > > > > > >
> > > > > > > Ahsan Khan wrote:
> > > > > > >
> > > > > > > > I am not getting any flow files, When i run cflowd it just
> > create my
> > > > router
> > > > > > > > Ethernet ip address named dir and there i have the files
name
> > > > arts.2000 etc,,
> > > > > > > >
> > > > > > > >Can you tell me how to create flow files .??
> > > > > > > >
> > > > > > > > With Regards
> > > > > > > > Ahsan Khan
> > > > > > > > Sr. System Admin
> > > > > > > > Internet Division (OneNet)
> > > > > > > > Sun Communication Pvt. Ltd.
> > > > > > > > Pakistan
> > > > > > > > http://www.one.net.pk
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > > From: "Todd Caine" <todd_caine@eli.net>
> > > > > > > > To: "Ahsan Khan" <ahsank@one.net.pk>;
> > <flowscan@net.doit.wisc.edu>
> > > > > > > > Sent: Thursday, September 28, 2000 3:23 AM
> > > > > > > > Subject: Re: NetFlow v5 required for FlowScan-1.00[234] (was
> > "Re: RRD
> > > > > > > > files generated by FlowScan have no data")
> > > > > > > >
> > > > > > > > > If your flow files are located in /var/local/flows then
try
> > the
> > > > following:
> > > > > > > > >
> > > > > > > > > cd /var/local/flows
> > > > > > > > > nohup /var/local/flows/bin/flowscan &
> > > > > > > > >
> > > > > > > > > Hopefully you will see a message similar to:
> > > > > > > > >
> > > > > > > > > Working of file flows.2000........
> > > > > > > > >
> > > > > > > > > I can't seem to get flowscan to do anything unless I am in
the
> > same
> > > > > > > > > directory as the raw flow files when I start flowscan.
Let me
> > know
> > > > if this
> > > > > > > > > works for you.
> > > > > > > > >
> > > > > > > > > -Todd
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Ahsan Khan wrote:
> > > > > > > > >
> > > > > > > > > > Dear David,
> > > > > > > > > >
> > > > > > > > > > I am once again emailing you about my Problem,
> > > > > > > > > >
> > > > > > > > > > My problem is that after installing each &
everything
> > when i
> > > > start
> > > > > > > > > > Flowscan its start fine but there is no .rrd files in
graph
> > dir. i
> > > >
> > > > > > > > > > have once again cross check all my installation and
> > configurations
> > > > but
> > > > > > > > > > found nothing.
> > > > > > > > > >
> > > > > > > > > > can i get any help .??
> > > > > > > > > > here is the output of my flowscan startup.
> > > > > > > > > >
> > > > > > > > > > [root@mail bin]# ./flowscan
> > > > > > > > > > Loading "Napster_subnets.boulder" ...
> > > > > > > > > > Loading "local_nets.boulder" ...
> > > > > > > > > > sleep 300...
> > > > > > > > > >
> > > > > > > > > > With Regards
> > > > > > > > > > Ahsan Khan
> > > > > > > > > > Sr. System Admin
> > > > > > > > > > Internet Division (OneNet)
> > > > > > > > > > Sun Communication Pvt. Ltd.
> > > > > > > > > > Pakistan
> > > > > > > > > > http://www.one.net.pk
> > > > > > > > > >
> > > > > > > > > > ----- Original Message -----
> > > > > > > > > > From: "Dave Plonka" <plonka@doit.wisc.edu>
> > > > > > > > > > To: <flowscan@net.doit.wisc.edu>
> > > > > > > > > > Cc: "Krishna Raman" <krishna@netwhistle.com>
> > > > > > > > > > Sent: Thursday, September 28, 2000 12:09 AM
> > > > > > > > > > Subject: NetFlow v5 required for FlowScan-1.00[234] (was
> > "Re:
> > > > RRD
> > > > > > files
> > > > > > > > > > generated by FlowScan have no data")
> > > > > > > > > >
> > > > > > > > > > > On Wed, Sep 27, 2000 at 11:01:01AM -0400, Krishna
Raman
> > wrote:
> > > > > > > > > > > > Thanks for the information.
> > > > > > > > > > > >
> > > > > > > > > > > > This is for David Plonka
> > > > > > > > > > > > --------------------------
> > > > > > > > > > > > I think it will be useful to state in the flowscan
> > > > > > > > > > > > web page that it does not support Version 1. May be
> > > > > > > > > > > > I overlooked the configuration piece of it, but it
> > > > > > > > > > > > is not very evident that it DOES NOT support version
1.
> > > > > > > > > > >
> > > > > > > > > > > You are correct. In the INSTALL doc I only *suggest*
that
> > you
> > > > > > export
> > > > > > > > > > > from your Cisco like this:
> > > > > > > > > > >
> > > > > > > > > > > ip flow-export version 5 peer-as
> > > > > > > > > > >
> > > > > > > > > > > I'll add a note about requiring v5 in the "Hardware
> > > > Requirements"
> > > > > > > > > > > section I guess.
> > > > > > > > > > >
> > > > > > > > > > > Technically, it is the current Cflow perl module used
by
> > > > FlowScan
> > > > > > that
> > > > > > > > > > > requires NetFlow version 5. When I added version 5
> > support to
> > > > > > > > Cflow.pm
> > > > > > > > > > > I removed version 1 support because I couldn't imagine
> > anyone
> > > > > > wanting
> > > > > > > > > > > to use it once version 5 was available. I realize it
may
> > > > require
> > > > > > an
> > > > > > > > > > > IOS upgrade to get v5, but many of us find that we
can't
> > help
> > > > but
> > > > > > stay
> > > > > > > > > > > fairly current with IOS because of other features we
need.
> > > > > > > > > > >
> > > > > > > > > > > > Ofcourse, FlowScan is a great utility though.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Dave
> > > > > > > > > > >
> > >
>
> --
> --------------------------------------------------------------
>
> <!-- Todd Caine - tcaine@eli.net
> Software Engineer
> Electric Lightwave, Inc.
> 4400 NE 77th Avenue
> Vancouver, WA 98662
> Direct Dial: (360) 816-4344 //-->
>
> --------------------------------------------------------------
>
>
--
Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe flowscan" in message body
Archive http://net.doit.wisc.edu/~plonka/list/flowscan/archive/
This archive was generated by hypermail 2b25 : Sun Oct 01 2000 - 18:26:58 CDT