Index: [Article Count Order] [Thread]

Date:  Mon, 12 Jul 2004 09:28:33 +0100
From:  Gustavo Silva <pseudo (at mark) zbronx.com>
Subject:  [coba-e:00888] Re: AW:  Re: AW:  Re: logrotation / split-logs /	awstats
To:  coba-e (at mark) open.cobaltqube.org
Message-Id:  <1089620913.3325.27.camel (at mark) PSEUDOSTATION.localdomain>
In-Reply-To:  <000001c46692$5297b970$120011ac (at mark) lanofglory.net>
References:  <000001c46692$5297b970$120011ac (at mark) lanofglory.net>
X-Mail-Count: 00888

Hi markus:

I didn't had auth_pam.so, so i downloaded it and compiled it (httpd2
version) without any compile errors.

Ran zz_awstats-run.pl so it could password protect the awstats
directories again, checked the .htacces config, but when trying to
access it trought the browser it gives me exactly the same eror.
AUTHORIZATION REQUIRED.


This is way to weird... why isn't he understanding the .htaccess?
:D damn....

I'll keep on trying...


Regrads,

gustavo
On Sat, 2004-07-10 at 16:26, MvA - OpenQube wrote:
> Hi Gustavo
> 
> I'ts a nice solution! It works. But now my review :-) We must difference
> between "Log Rotating" and "Splitting the Apache Log".
> 
> You have RH9, right? The analog is part of the BQ-Install (in case of a
> fedora install). Because the split_logs is usable for other logfiles: mail &
> ftp. 
> 
> First of all, we need a logrotation, because there many other logfiles to
> rotate (email, messages, ftp, ...). So please let the system rotate
> (Reanimate your /etc/cron.daily/logrotate ;-)). I tested your script on my
> fedora, it works nice. So i changed the "/etc/logrotate.d/httpd":
> 
> /var/log/httpd/access {
>         missingok
>         notifempty
>         sharedscripts
>         postrotate
>         /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null ||
> true
>         /usr/local/sbin/split_logs_webaccess
>         endscript
>         daily
> }
> 
> Your script is located in /usr/local/sbin/split_logs_webaccess (not in
> /etc/cron.daily/, because i hate to create cron orders with the filenames
> (the awstats has two zz's --> running after logrotate)) and called after the
> access-logfile of the apache has new rotated.
> 
> Thats it. The result is viewable in a few days. Shuld we create a
> BQ-System-00001-1.pkg ? :)
> 
> ===========================
> .htaccess-Problem: I had installed following components: 
> 
> 1-) check, if the /etc/httpd/modules/mod_auth_pam.so exists. If not, compile
> it: 
> 	http://pam.sourceforge.net/mod_auth_pam/
> 	(To compile, you need the httpd-devel package)
> 
> 2-) Add fallowing line to the httpd.conf 
> 	LoadModule auth_pam_module modules/mod_auth_pam.so 
>  	
> 3-) Edit /etc/pam.d/httpd so it looks like this:
> 
> 	#%PAM-1.0
> 	auth       requisite    /lib/security/pam_nologin.so
> 	auth       requisite    /lib/security/pam_shells.so
> 	auth       required     /lib/security/pam_unix.so shadow nullok
> 	auth       required     /lib/security/pam_cce.so
> 	account    required     /lib/security/pam_unix.so
> 
> 4-) tail the /var/log/httpd/error for more info's. 
> 5-) Let the .htaaccess-file original looking like this: 
> 
> 	#Access file
> 	order allow,deny
> 	allow from all
> 	require group site18
> 	Authname WebStats
> 	Authtype Basic
> 	AuthAuthoritative off
> 
> 6-) Try and error, hehe :) At my FC-System, i can't login... and i had no
> errors :)
> 
> ===========================
> 
> Markus
> 
> 
> -----Urspr端ngliche Nachricht-----
> Von: Gustavo Silva [mailto:pseudo (at mark) zbronx.com] 
> Gesendet: Samstag, 10. Juli 2004 07:11
> An: coba-e (at mark) open.cobaltqube.org
> Betreff: [coba-e:00883] Re: AW: Re: logrotation / split-logs / awstats
> 
> 
> Hi,
> 
> I found a dirty way to make it happen (STILL VERY BUGGY), take a look and
> try ir for yourself:
> 
> 
> 
> - first of all i completly ignored logrotate (for now, I #'ed out logrotate
> command in /etc/cron.daily/logrotate), since it misses analog, and from what
> i've read analog is really slow i tried another way:
> 
> 
> 
> - next i addapted a perl script i found to do log splitting and i altered 2
> lines to make it suitable to our install:
> 
> 
> 
> - the script needs Getopt::Long from CPAN
> 
> 
> 
> ---- here is the code --------
> 
> #!/usr/bin/perl -w
> #
> # Script to split an Apache Extended Combined access log into individual #
> logs based on the virtual host logged in the second field. # # Written by
> Jonathon Vance # Copyright (c) 2002 Quantified Systems, Inc.
> 
> ###modifications by######pseudo (at mark) zbronx.com 10-07-2004 for BlueQuartz
> 
> use strict;
> no strict 'refs';
> use Getopt::Long;
> 
> my $dir='/home/sites';
> my $log='/var/log/httpd/access_log';
> 
> # Verify that the directory exists and is writeable
> if ((! -e $dir) && (! -d $dir) && (! -w $dir)) {
>         die "$dir does not exist or is not writeable.";
> }
> 
> # Check that the log file exists and is readable
> if ((! -e $log) && (! -r $log)) {
>         die "$log does not exist or is not readable. Exiting."; } # Open the
> logfile as readonly open(LOG, "<$log") || die "Problem opening log file:
> $!";
> 
> my ($line, @parts, $vhost, $out);
> my %files = ();
> 
> # Loop through each line and write it to the appropriate log file
> while(<LOG>) {
>         # Get the virtual host from the hit (make sure it exists)
>         $line = $_;
>         @parts = split(/ /, $line, 3);
> 
> #1########pseudo (at mark) zbronx.com 10-07-2004 - log field 0 instead of 1
>         $vhost = $parts[0];
> 
>         if (!$vhost) { next; }
> 
>         # Open the output file if not already open
> 
> #2########pseudo (at mark) zbronx.com 10-07-2004 - the path using $vhost pointer
>         $out = "$dir/$vhost/logs/web.log";
>         if (!$files{$out}) {
>           $files{$out} = uc($vhost);
>           open($files{$out}, ">>$out") || warn "Unable to open $out";
>         }
> 
>         # Print the hit to the file
>         select($files{$out});
>         print $line;
>         select(STDOUT);
> }
> 
> # Close the open files and exit
> my $file;
> foreach $file (values(%files)) {
>         close($file) || warn "Problems closing $file";
> }
> 
> ---- here is the code --------
> 
> 
> 
> - copy this script to /etc/cron.daily
> 
> 
> 
> - first run it to give it a try
>   it creates all possible logs found in access_log
> 
> 
> 
> - i've installed the MvA supplied AWStats package
> ftp://ftp2.mva.ch/BlueQuartz-FC1/BlueQuartz-FC1-Awstats-5.9-3.pkg
> 
> 
> 
> 
> - first run /etc/cron.daily/zz_awstats-run.pl
> still buggy, it passwords protect awstats directory incorrectly i'll try a
> rewrite
> 
> 
> 
> - deleted the .htaccess from the /awstats i wanted to try 
> 
> 
> 
> - checked www.site.com/awstats
> and voil叩, at least it worked with me
> 
> 
> 
> BUT, 
> - still needs to do some sort of backup of access_log
> - still needs more tests to see if the records are being respected
> - still need your opinion
> 
> 
> regards,
> 
> gustavo
> 
> 
> 
> 
> 
> 
> 
> 
> On Sat, 2004-07-10 at 03:42, MvA - OpenQube wrote:
> > but the logfiles are not splitted :-( i think, there is a problem with 
> > split_logs :-( an there is also no logfile to download at the 
> > webinterface for "web".  what could be the problem..? if the splitting 
> > works (it shuld be), we have both a solution for webalizer and 
> > awstats...
> >         -----Urspr端ngliche Nachricht-----
> >         Von: Scott McClain [mailto:smcclain (at mark) mcsgeek.com] 
> >         Gesendet: Freitag, 9. Juli 2004 17:28
> >         An: coba-e (at mark) open.cobaltqube.org
> >         Betreff: [coba-e:00875] Re: logrotation / split-logs / awstats
> >         
> >         
> >         
> >         I am getting an error.
> >         
> >         /tmp/logrotate.OsIjVo: line 5: /tmp/.logrotate_apache_access:
> >         No such file or directory
> >         
> >         I tested with
> >         
> >         /usr/sbin/logrotate /etc/logrotate.conf v
> >         
> >          
> >         
> >         if I touch /tmp/.logrotate_apache_access first then no error.
> >         
> >          
> >         
> >         -----Original Message-----
> >         From: MvA - OpenQube [mailto:openqube (at mark) mva.ch] 
> >         Sent: Friday, July 09, 2004 5:22 AM
> >         To: coba-e (at mark) open.cobaltqube.org
> >         Subject: [coba-e:00873] logrotation / split-logs / awstats
> >         
> >          
> >         
> >         hi again,
> >         
> >         
> >          
> >         
> >         
> >         sorry, for bugging this mainiglist again. a few things to say
> >         about the log-file-rotation: first of all, i changed some
> >         settings of "/etc/logrotate.d/httpd",  after i deleted the
> >         apache-rotate in "/etc/logrotate.d/". new
> >         /etc/logrotate.d/httpd:
> >         
> >         
> >          
> >         
> >         
> >         /var/log/httpd/access {
> >                 missingok
> >                 notifempty
> >                 sharedscripts
> >                 postrotate
> >                 /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2>
> >         /dev/null || true
> >                 /usr/local/sbin/split_logs web <
> >         /tmp/.logrotate_apache_access
> >                 rm -f /tmp/.logrotate_apache_access >/dev/null 2>&1
> >                 endscript
> >                 daily
> >         }
> >         
> >         
> >         the main-change is the rotate from weekly to daily. i also
> >         added the two lines for splitting the logs to the user's
> >         directory: logs/web.log. i created a additional file in
> >         "/etc/logrotate.d/" named httpd-error contains:
> >         
> >         
> >          
> >         
> >         
> >         /var/log/httpd/error {
> >                 missingok
> >                 sharedscripts
> >                 postrotate
> >                 /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2>
> >         /dev/null || true
> >                 endscript
> >         
> >         
> >                 weekly
> >         }
> >         
> >         
> >         the rotate is working (tested with: /usr/sbin/logrotate
> >         /etc/logrotate.conf -v). on systems like fedora, i had to
> >         change the name of the apache-logfile from access_log to
> >         access and error_log to error, and the log-settigs in
> >         /etc/httpd/conf/httpd.conf. thats my first-step. now my
> >         problem: i think, the splitting of access-log doesn't work.
> >         has anyone some hints and ideas for checking the split_log
> >         script?
> >         
> >         
> >          
> >         
> >         
> >         after the rotation, i created the awstats-package for install
> >         on
> >         BQ-FC1: 
> > ftp://ftp2.mva.ch/BlueQuartz-FC1/BlueQuartz-FC1-Awstats-5.9-3.pkg (its a
> beta, the .htaccess does not work at this time.... :-( but this file can be
> deleted), the cron.daily-script is working.
> >         
> >         
> >          
> >         
> >         
> >         cya,
> >         
> >         
> >         markus
> >         
> >         
> 
> 
> 
>