Subject: Re: [locker, gzip'ping and] Purging old flows files
From: Dave Plonka (plonka@doit.wisc.edu)
Date: Mon Nov 13 2000 - 11:15:53 CST
On Mon, Nov 13, 2000 at 04:56:47PM -0000, Martinez, Samson wrote:
> I am preparing to setup the automatic removal of old flow files from the
> @PREFIX@/saved directory. However, I'm a bit confused about the example
> crontab showing the compressing and removal of these files, specifically
> this entry:
>
> # gzip the saved flow files:
> 2,7,12,17,22,27,32,37,42,47,52,57 * * * * test -d @prefix@/saved && cd
> @prefix@/saved && @prefix@/bin/locker -ne .gzip_lock "@KSH_PATH@ -c
> '@LS_PATH@ flows.[0-9]!(*.gz) 2>/dev/null | @XARGS_PATH@ -n1 @GZIP_PATH@'"
> #
>
> What is the ".gzip_lock" entry specify?
It is a file on which the locker utility does file-locking, to be sure
that you only have one command running at a time. That is its sole
purpose in life.
Try out locker on its own to learn how it works, e.g.:
$ touch /tmp/.lock
$ locker -e /tmp/.lock sleep 60 & # get an exclusive lock, and run sleep
$ locker -ne /tmp/.lock date # non-blocking, exclusive lock
flock failed: Resource temporarily unavailable
$ locker -e /tmp/.lock date # blocking, exclusive lock
With locker, as a nice side-effect, you can say:
$ fuser .gzip_lock
- or -
$ lsof .gzip_lock
To see if any process is running in that directory, currently gzip'ping
the files.
> Is this a file that is supposed to
> exist somewhere?
Yes, you must create it yourself, as described in the FLowScan INSTALL
doc:
http://net.doit.wisc.edu/~plonka/FlowScan/INSTALL.html#Preserving_Old_Flow_Files
preferable prior to scheduling the cron job do this:
$ cd saved
$ touch .gzip_lock
Note that you will get an email message from the cron job saying
"Resource Temporarily Unavailable" or something to that effect,
whenever the previous gzip job is still running.
If it helps, here's the two cron jobs I run under Linux to gzip raw
flow files and remove old ones after `n' hours (e.g. 336):
#
# gzip the saved flow files:
2,7,12,17,22,27,32,37,42,47,52,57 * * * * test -d /var/local/flows/saved && cd /var/local/flows/saved && /var/local/flows/bin/locker -ne .gzip_lock "/usr/bin/ksh -c '/bin/ls flows.[0-9]!(*.gz) 2>/dev/null | /usr/bin/xargs -n1 --no-run-if-empty /bin/gzip'"
#
# Purge the flow files:
# find(1) -mtime +1 was insufficient - I want to delete them as soon as they're
# `n' hours old:
0 * * * * /usr/bin/find /var/flows/saved -type f -name 'flows.*' -print |/usr/bin/perl -e '$now = time; $seconds = 336*60*60; while (<>) { chomp; (@_ = stat $_) && ($now - $_[9] > $seconds) && print $_, "\n" }' |/usr/bin/xargs /bin/rm -f
Dave
--
plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI
--
Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body
Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
"unsubscribe flowscan" in message body
Archive http://net.doit.wisc.edu/~plonka/list/flowscan/archive/
This archive was generated by hypermail 2b25 : Mon Nov 13 2000 - 11:18:53 CST