| |
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [FW-1] Too large log file
FWIW, this was what we did on our Solaris system to handle heavy days. Might
not be useful for Win systems.
Our logs got too large over a 24-hour period and had to be switched more
often. Wanting to be able to still generate reports for the full day, we set
up a script that could be run multiple times each day under cron. Each time
it runs, the log is switched and the resulting file named with the date and
time. When the hour reaches 23, the logs and exports for the day are
concatenated into single large files for viewing and the day's report
processing uses the large files. We have always been doing the switch for
each day at 23:55, but of course the exact time is arbitrary. With this
script we can also do switches at, for example 05:55, 11:55, 17:55, and
still get only one set of daily reports. Or every hour if I had to (no,
thanks, please). I'm confident that this script can be improved, made more
efficient, less clumsy; OTOH it works just fine as is...
The pertinent parts are below.
#!/bin/sh
DATETIME=`date +%d%b%Y,%T`
DATE=`echo $DATETIME | cut -f1 -d","`
HOUR=`echo $DATETIME | cut -f2 -d"," | cut -c1-2`
#Start a new log
#Put this command first in case something else in this script fails...
echo " "
echo Starting switching fw-1 log file at `date`
$BINDIR/fw logswitch $LOGDIR/${DATETIME}
echo Finished switching fw-1 log file at `date`
echo " "
echo Starting ip log export at `date`
$BINDIR/fw logexport -n -i $LOGDIR/${DATETIME}.log -o
$LOGDIR/${DATETIME}.export.ip \
>> /dev/null 2>&1
echo Finished ip log export at `date`
#Check the hour. If 23, then concatenate all export files for today into one
# comprehensive export file, concatenate all fw-1 log files into one
similar
# comprehensive log file, process the daily reports, then compress all work
# files and non-comprehensive log and export files, otherwise exit.
echo " "
if [ X$HOUR != X23 ]; then
echo HOUR not 23, exiting.
exit 0
fi
echo HOUR is 23, generating reports.
#Concatenate all export files for the day into comprehensive export file
echo " "
echo Starting export file concatenation at `date`
cat `ls -1 $LOGDIR/${DATE}*export.ip` > $LOGDIR/${DATE}.export.ip
echo Finished export file concatenation at `date`
#Concatenate all fw-1 log files for the day into single log file
#This is to allow a day's logs to be viewed in one process. No other
#processing is done with the concatenated log file. If the log file is
#very large, >200MB, viewing may fail, in which case individual log files
may
#be viewed.
echo " "
echo Starting fw-1 log file concatenation at `date`
cat `ls -1 $LOGDIR/${DATE},??:??:??.log` > $LOGDIR/${DATE}.log
echo Finished fw-1 log file concatenation at `date`
#<snip> (other files saved and dated, several reports run)
echo " "
echo Starting file compression at `date`
ls -1 $LOGDIR/${DATE}*
/usr/local/bin/gzip `ls -1 $LOGDIR/${DATE}* | egrep -v $LOGDIR/${DATE}.log`
echo " "
echo Finished file compression at `date`
ls -1 $LOGDIR/${DATE}*
echo " "
echo Run completed at `date`
#End of file
Chuck Sterling
Magic is REAL, unless declared INTEGER
> ----------
> From: tolits[SMTP:[email protected]]
> Reply To: Mailing list for discussion of Firewall-1
> Sent: Monday, December 03, 2001 10:33 PM
> To: [email protected]
> Subject: Re: [FW-1] Too large log file
>
> im new to this thing, what's "/dev/null 2 &1" for?
>
> Gerard MANNIG wrote:
>
> A 12:00 29/11/01 -0000, Leon Noble écrit:
> >Hi Billy,
> >
> >This is not a solution to your immediate problem, but just a
> suggestion to
> >help organise your logs a bit better.
> >
> >
> >cron or at
> >
> >$FWDIR/bin/fw logswitch > /dev/null 2>&1 (Replace $FWDIR
> with it's
> >appropriate value.)
> >
> >every night at midnight. This will help organise your log files.
> >
> >
> >best regards
> >
> >Leon.
> >
> >-----Original Message-----
> >From: Mailing list for discussion of Firewall-1
> >[ mailto:[email protected]]On Behalf Of
> Billy
> >Chan
> >Sent: 29 November 2001 09:45
> >To: [email protected]
> >Subject: [FW-1] Too large log file
> >
> >
> >Dear all,
> >
> >I try to do a "fw logswitch" since the log file size is very large
> (about
> >300M), but it fail, it display "Log switch failed".
> >
> >Then, I run "fwstop" and try to rename the log file, but it display
> " The
> >Process cannot access the file because it is being used by another
> process."
> >while I run the command " ren old.log new.log".
> >
> >Is there any other process still running after "fwstop" ?
> >How to rename the file name?
> >
> >Best Regards,
> >Billy
> >
> >===============================================
> >To unsubscribe from this mailing list,
> >please see the instructions at
> > http://www.checkpoint.com/services/mailing.html
> >===============================================
> >
> >===============================================
> >To unsubscribe from this mailing list,
> >please see the instructions at
> > http://www.checkpoint.com/services/mailing.html
> >===============================================
> >
> >
>
> # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
> REPONSE AUTOMATIQUE - AUTOMATIC RESPONSE
> # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
>
> Monsieur, Madame,
>
> J'ai remarque que vous utilisiez Aoutelouque comme client mail. Ce
> logiciel
> est un des moyens privilegies de propagation de virus que son
> createur,
> Microsoft, monopole de fait, n'a toujours pas ameliore tout comme il
> a
> cautionne par son passeisme la creation d'environ 45.000 virus ( ete
> 2000).
>
> Si vous souhaitez continuer a l'utiliser, je vous serais
> reconnaissant de
> bien vouloir au moins me rayer de votre carnet d'adresses.
>
> -_-
>
> Hi :
>
> I noticed that you run Outlook as your mail client. This application
> is one
> of the best ways to spread computer viruses and alike that its
> creator,
> Microsoft, de facto monopoly, has still not improved as well as it
> has
> welcome by its neutrality the creation of about 45,000 viruses ( by
> July
> 2000) . If you are willing to keep on using it, I should be pleased
> to be
> wiped out from your Outlook agenda.
>
> --
> Virus LoveLetter et autres virus "Aoutelouqueurs" : "Les 2.6
> milliards de
> cout estime; ca n'est pas une perte, c'est une taxe a
> l'incompetence. La
> prochaine fois, ce sera une amende a la connerie."
> --
> Virus LoveLetter and other look alike viruses : "The estimated 2.6
> billion
> it cost us this time, that's not a financial loss,
> it's a tax on incompetence. The next time will add penalties for
> stupidity."
>
> --
> Manifeste anti-Microsoft
> http://perso.wanadoo.fr/sevillane/microsoft.html
> http://realites-virus.org (Utilisateurs IE s'abstenir )
>
> ===============================================
> To unsubscribe from this mailing list,
> please see the instructions at
> http://www.checkpoint.com/services/mailing.html
> ===============================================
>
> --
> Lito A. Lampitoc
> PLDT Foundation http://www.codewan.com.ph
> --
> "If you think you're good, you're not."
>
>
===============================================
To unsubscribe from this mailing list,
please see the instructions at
http://www.checkpoint.com/services/mailing.html
===============================================
|
|
