Index: [Article Count Order] [Thread]

Date:  Thu, 23 Feb 2006 20:46:21 +0800
From:  patricko <patricko (at mark) staff.singnet.com.sg>
Subject:  [coba-e:04154] Bug fixes: Please use version 0.02a only ignore 0.01a
To:  coba-e (at mark) bluequartz.org
Message-Id:  <20060223204621.1996ca15 (at mark) localhost.localdomain>
In-Reply-To:  <20060221213731.2e11f783 (at mark) localhost.localdomain>
References:  <057601c62666$b9a473c0$02002c0a (at mark) WebHoster>	<Pine.LNX.4.44.0602010050050.19226-100000 (at mark) staff.singnet.com.sg>	<20060221202810.455b1cfd (at mark) localhost.localdomain>	<20060221213326.79e10fc5 (at mark) localhost.localdomain>	<20060221213731.2e11f783 (at mark) localhost.localdomain>
X-Mail-Count: 04154


Hi,

Silly mistake.
Causing Group quota not to sync.

Have fixed that


#!/usr/bin/perl -w -I/usr/sausalito/perl -I/usr/sausalito/handlers/base/vsite
#
#######  meta-verify, version 0.02a
#
#  
#  system(OS) & database(codb aka meta) verification and integrity check utility
#  for the Cobalt 550 / Bluequartz  product family
#
#  Patrick Ong Tiong Heng (patricko (at mark) staff.singnet.com.sg)
#
#
#
#######  OPERATION
#
#  meta-verify -u -q | -s -q | -a | [-v] [-h] 
# 
#    -u   user
#    -s   virtual site
#
#    -q   quota
#    -a   all | users + virtual sites 
#
#    -v   verbose 
#    -h   usage help
#
#
#
#######  DESCRIPTION
#
#
#  Actually, this script should call system-verify as it attempts to detect and 
#  repair coherency problems between the state saved in the database and 
#  the system configuration state. In order to let cobalt users feel "at home",
#  I decided to retain the name: meta-verify.
#
#  - (KIV) Verify and repair virtual sites.
#  - (KIV) Verify and repair user accounts.
#  - (KIV )Maintain intergrity of the RaQ system configuration files.
#  - (Partial - Only Quota) Ensure coherency between System and Meta/quota backend.
#
#
#######  INSTRUCTION
#
# For fixing corrupted User,Group Quotas, boot server to Single mode.
# And run:
#
# 1)
#  quotacheck /home
#  quotacheck -g /home
#
# 2)
# Reboot.
#
# 3)
# Run: metaverify -a and wait (process will printout set Quotas without errors)
# Sync, sync, sync. And reboot again.
#
# 4)
# Run: repquota /home (integrity checking)
# make sure SITExx-logs, admin accounts DO NOT HAVE quota.
# if there are some entries, remove them manually via edquota
#
# 
#######  SUPPORTED SYSTEMS
#
#  Cobalt 550, Bluequartz
#
#
#######  CHANGELOG
#
#
# version 0.02a (Feb 23 2006)
#   - Fixed typo, Group quota not able to sync from Meta to Disk
#   - Added INSTRUCTION section
#
# version 0.01a (Feb 21 2006)
#   - initial creation alpha
#
#
#######

BEGIN {
    require CCE;
    require Quota;
    
    use Getopt::Std;

    use vars qw($TITLE $VERSION $AUTHOR $COPYRIGHT);

    $TITLE       = "meta-verify";
    $VERSION     = "0.01a";
    $AUTHOR      = "Patrick Ong Tiong Heng (patricko\ (at mark) staff.singnet.com.sg)";
    $COPYRIGHT   = "bluequartz.org";
}


use CCE;
use strict;
use Quota;

my $path = "/home";
my $dev;
my $uid = "";
my $uqs = "";
my $uqh = "";
my $siteUID = "";
my $userUID = "";

use vars qw($opt_u $opt_s $opt_q $opt_a $opt_h $Verbose);





# command-line arguments, verbose level
getopts("usqavh");
    
# usage help
&usage if ($opt_h);

if ($opt_a) {
   &meta2system();
}
elsif ($opt_u) {
   print "KIV\n";
}
elsif ($opt_s) {
   print "KIV\n";
}
elsif ($opt_q) {
   print "KIV\n";
}
elsif ($Verbose) {
   print "KIV\n";
}
else {
    &usage;
}
print "\n" unless ($Verbose);
exit 0;

1;


sub usage ()
{

    printf("\n");
    printf(" usage: %s [options]\n", $TITLE);
    printf("\n");
    printf("   -u   user\n");
    printf("   -s   virtual site\n");
    printf("\n");
    printf("   -q   quota\n");
    printf("   -a   all \| users + virtual sites\n");
    printf("\n");
    printf("   -v   verbose\n");
    printf("   -h   help\n");
    printf("\n");
    exit 1;
}



sub meta2system()
{

 my $cce = new CCE;
 $cce->connectuds();
 my @vsites;
 my @vsitelist;
 my @userlist;
 my $hostname=`hostname`;
 chomp($hostname);

 $dev = Quota::getqcarg($path) || die "$path: $!\n";


 ### Find Domain records

 @vsitelist = $cce->findx('Vsite');
 foreach my $vsite (@vsitelist) { 
  my @userlist;
  my $siteNum;
  my $userNum;
  my $fqdn;
  my $domain;
  my $vsitequota;
  my $vsitequotaExtra;
  my $userquota;
  my $userquotaExtra;
  my ($ok, $object, $old, $new) = $cce->get($vsite);
  my ($vDiskok, $vDiskobject, $vDiskold, $vDisknew) = $cce->get($vsite, "Disk");

  $siteNum="${%{$object}}{'name'}";
  $domain="${%{$object}}{'domain'}";
  $fqdn="${%{$object}}{'fqdn'}";
  $vsitequota="${%{$vDiskobject}}{'quota'}";
  $vsitequota*=1024;
  $vsitequotaExtra=$vsitequota;
  $vsitequotaExtra+=1024;

  $siteUID = `quota -g $siteNum | head -1 | awk -F \"gid \" '{print \$2}' | awk -F \"): \" '{print \$1}'` ;
  chomp($siteUID);
  $uid = $siteUID;
  $uqs = $vsitequota;
  $uqh = $vsitequotaExtra;

  unless(Quota::setqlim($dev, $uid, $uqs, $uqh, 0, 0, 0, 1)) {
       print "Quota set for $uid, $domain, $siteNum, QUOTA\: $vsitequota\n";
  }
  print  $fqdn." ".$siteUID." ".$siteNum." ".$vsitequota." ".$vsitequotaExtra."\n";


  ### Find User account

  @userlist = $cce->findx('User', {"site" => "$siteNum"} );

  foreach my $user (@userlist) { 
    my ($userOK, $userOBJ, $userOLD, $userNEW) = $cce->get($user);
    my ($userDiskOK, $userDiskOBJ, $userDiskOLD, $userDiskNEW) = $cce->get($user, "Disk");
    #print "${%{$userOBJ}}{name}"." (at mark) "."$domain\n";
    $userNum="${%{$userOBJ}}{'name'}";
    $userquota="${%{$userDiskOBJ}}{'quota'}";
    $userquota*=1024;
    $userquotaExtra=$userquota;
    $userquotaExtra+=1024;


    $userUID = `quota $userNum | head -1 | awk -F \"uid \" '{print \$2}' | awk -F \"): \" '{print \$1}'` ;
    chomp($userUID);
    $uid = $userUID;
    $uqs =  $userquota;
    $uqh = $userquotaExtra;

    unless(Quota::setqlim($dev, $uid, $uqs, $uqh, 0, 0)) {
           print "Quota set for $uid, $domain, $userNum, QUOTA\: $userquota\n";
    }
    print $fqdn." ".$userUID." ".$userNum." ".$userquota." ".$userquotaExtra."\n";
  }

 }

 $cce->bye('SUCCESS');
 #exit(0);
 exit(1);

}
























Cheers
patrick









On Tue, 21 Feb 2006 21:37:31 +0800
patricko <patricko (at mark) staff.singnet.com.sg> wrote:

> Hi,
> 
> Forgotten to add:
> 
> after meta-verify, run
> repquota /home          (integrity checking)
> 
> 
> 
> make sure SITExx-logs DO NOT HAVE quota.
> if there are some entries, remove them manually via edquota
> 
> 
> 
> Cheers
> patrick
> 
> 
> 
> 
> 
> On Tue, 21 Feb 2006 21:33:26 +0800
> patricko <patricko (at mark) staff.singnet.com.sg> wrote:
> 
> > Hi,
> > 
> > Bluequartz DB out-of-sync with System-OS quota
> > is a reality.(Personal experienced the below mentioned issue)
> > 
> > 
> > Its an OS issue relating to ext3. The inodes or the superblocks
> > are "dirty" causing "extra invisible users" or "grand total of all users quota NOT equal to domain quota" 
> > 
> > 
> > 
> > So we need to go a quota check as follow:
> > 
> > [Single mode]
> > quotacheck /home
> > quotacheck -g /home
> > 
> > 
> > 
> > Sometimes, the system quotas are corrupted.
> > Therefore, you need my meta-verify to SYNC 
> > the data back.
> > 
> > 
> > 
> > 
> > Cheers
> > patrick
> > 
> > 
> > 
> > 
> > 
> > 
> > On Tue, 21 Feb 2006 20:28:10 +0800
> > patricko <patricko (at mark) staff.singnet.com.sg> wrote:
> > 
> > > Hi Blues,
> > > 
> > > 
> > > I have written a meta-verify utility for Bluequartz.
> > > It is in still development. 
> > > ***But the basic CODB-meta -> system-OS is working*** 
> > > 
> > > 
> > > The logic for writing meta-verify is because OS and CODB-meta
> > > can be out-of-sync, especially system quota (when corrupted with quotacheck) with CODB.
> > > 
> > > 
> > > 
> > > ps: Gavin, you can use this script to sync BQ-DB with system quotas.
> > > 
> > > 
> > > 
> > > >>>>>>>>>>>>>>>>>>>>>>meta-verify>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > 
> > > #!/usr/bin/perl -w -I/usr/sausalito/perl -I/usr/sausalito/handlers/base/vsite
> > > #
> > > #######  meta-verify, version 0.01a
> > > #
> > > #  
> > > #  system(OS) & database(codb aka meta) verification and integrity
> > > check utility
> > > #  for the Cobalt 550 / Bluequartz  product family
> > > #
> > > #  Patrick Ong Tiong Heng (patricko (at mark) staff.singnet.com.sg)
> > > #
> > > #
> > > #
> > > #######  OPERATION
> > > #
> > > #  meta-verify -u -q | -s -q | -a | [-v] [-h] 
> > > # 
> > > #    -u   user
> > > #    -s   virtual site
> > > #
> > > #    -q   quota
> > > #    -a   all | users + virtual sites 
> > > #
> > > #    -v   verbose 
> > > #    -h   usage help
> > > #
> > > #
> > > #
> > > #######  DESCRIPTION
> > > #
> > > #
> > > #  Actually, this script should call system-verify as it attempts to
> > > detect and 
> > > #  repair coherency problems between the state saved in the database
> > > and 
> > > #  the system configuration state. In order to let cobalt users feel
> > > "at home",
> > > #  I decided to retain the name: meta-verify.
> > > #
> > > #  - (KIV) Verify and repair virtual sites.
> > > #  - (KIV) Verify and repair user accounts.
> > > #  - (KIV )Maintain intergrity of the RaQ system configuration files.
> > > #  - (Partial - Only Quota) Ensure coherency between System and
> > > Meta/quota backend.
> > > #
> > > #
> > > #######  SUPPORTED SYSTEMS
> > > #
> > > #  Cobalt 550, Bluequartz
> > > #
> > > #
> > > #######  CHANGELOG
> > > #
> > > #
> > > # version 0.01a (Feb 21 2006)
> > > #   - initial creation alpha
> > > #
> > > #
> > > #######
> > > 
> > > BEGIN {
> > >     require CCE;
> > >     require Quota;
> > >     
> > >     use Getopt::Std;
> > > 
> > >     use vars qw($TITLE $VERSION $AUTHOR $COPYRIGHT);
> > > 
> > >     $TITLE       = "meta-verify";
> > >     $VERSION     = "0.01a";
> > >     $AUTHOR      = "Patrick Ong Tiong Heng (patricko\ (at mark) staff.singnet.com.sg)";
> > >     $COPYRIGHT   = "bluequartz.org";
> > > }
> > > 
> > > 
> > > use CCE;
> > > use strict;
> > > use Quota;
> > > 
> > > my $path = "/home";
> > > my $dev;
> > > my $uid = "";
> > > my $uqs = "";
> > > my $uqh = "";
> > > my $siteUID = "";
> > > my $userUID = "";
> > > 
> > > use vars qw($opt_u $opt_s $opt_q $opt_a $opt_h $Verbose);
> > > 
> > > 
> > > 
> > > # command-line arguments, verbose level
> > > getopts("usqavh");
> > >     
> > > # usage help
> > > &usage if ($opt_h);
> > > 
> > > if ($opt_a) {
> > >    &meta2system();
> > > }
> > > elsif ($opt_u) {
> > >    print "KIV\n";
> > > }
> > > elsif ($opt_s) {
> > >    print "KIV\n";
> > > }
> > > elsif ($opt_q) {
> > >    print "KIV\n";
> > > }
> > > elsif ($Verbose) {
> > >    print "KIV\n";
> > > }
> > > else {
> > >     &usage;
> > > }
> > > print "\n" unless ($Verbose);
> > > exit 0;
> > > 
> > > 1;
> > > 
> > > 
> > > sub usage ()
> > > {
> > > 
> > >     printf("\n");
> > >     printf(" usage: %s [options]\n", $TITLE);
> > >     printf("\n");
> > >     printf("   -u   user\n");
> > >     printf("   -s   virtual site\n");
> > >     printf("\n");
> > >     printf("   -q   quota\n");
> > >     printf("   -a   all \| users + virtual sites\n");
> > >     printf("\n");
> > >     printf("   -v   verbose\n");
> > >     printf("   -h   help\n");
> > >     printf("\n");
> > >     exit 1;
> > > }
> > > 
> > > 
> > > 
> > > 
> > > sub meta2system()
> > > {
> > > 
> > >  my $cce = new CCE;
> > >  $cce->connectuds();
> > >  my @vsites;
> > >  my @vsitelist;
> > >  my @userlist;
> > >  my $hostname=`hostname`;
> > >  chomp($hostname);
> > > 
> > >  $dev = Quota::getqcarg($path) || die "$path: $!\n";
> > > 
> > > 
> > >  ### Find Domain records
> > > 
> > >  @vsitelist = $cce->findx('Vsite');
> > >  foreach my $vsite (@vsitelist) { 
> > >   my @userlist;
> > >   my $siteNum;
> > >   my $userNum;
> > >   my $fqdn;
> > >   my $domain;
> > >   my $vsitequota;
> > >   my $vsitequotaExtra;
> > >   my $userquota;
> > >   my $userquotaExtra;
> > >   my ($ok, $object, $old, $new) = $cce->get($vsite);
> > >   my ($vDiskok, $vDiskobject, $vDiskold, $vDisknew) = $cce->get($vsite, "Disk");
> > > 
> > >   $siteNum="${%{$object}}{'name'}";
> > >   $domain="${%{$object}}{'domain'}";
> > >   $fqdn="${%{$object}}{'fqdn'}";
> > >   $vsitequota="${%{$vDiskobject}}{'quota'}";
> > >   $vsitequota*=1024;
> > >   $vsitequotaExtra=$vsitequota;
> > >   $vsitequotaExtra+=1024;
> > > 
> > >   $siteUID = `quota -g $siteNum | head -1 | awk -F \"gid \" '{print \$2}' | awk -F \"): \" '{print \$1}'` ;
> > >   chomp($siteUID);
> > >   $uid = $siteUID;
> > >   $uqs = $vsitequota;
> > >   $uqh = $vsitequotaExtra;
> > > 
> > >   unless(Quota::setqlim($dev, $uid, $uqs, $uqh, 0, 0)) {
> > >        print "Quota set for $uid, $domain, $siteNum, QUOTA\: $vsitequota\n";
> > >   }
> > >   print  $fqdn." ".$siteUID." ".$siteNum." ".$vsitequota." ".$vsitequotaExtra."\n";
> > > 
> > > 
> > > 
> > > 
> > > 
> > >   ### Find User account
> > > 
> > >   @userlist = $cce->findx('User', {"site" => "$siteNum"} );
> > > 
> > >   foreach my $user (@userlist) { 
> > >     my ($userOK, $userOBJ, $userOLD, $userNEW) = $cce->get($user);
> > >     my ($userDiskOK, $userDiskOBJ, $userDiskOLD, $userDiskNEW) = $cce->get($user, "Disk");
> > >     #print "${%{$userOBJ}}{name}"." (at mark) "."$domain\n";
> > >     $userNum="${%{$userOBJ}}{'name'}";
> > >     $userquota="${%{$userDiskOBJ}}{'quota'}";
> > >     $userquota*=1024;
> > >     $userquotaExtra=$userquota;
> > >     $userquotaExtra+=1024;
> > > 
> > > 
> > >     $userUID = `quota $userNum | head -1 | awk -F \"uid \" '{print \$2}' | awk -F \"): \" '{print \$1}'` ;
> > >     chomp($userUID);
> > >     $uid = $userUID;
> > >     $uqs =  $userquota;
> > >     $uqh = $userquotaExtra;
> > > 
> > >     unless(Quota::setqlim($dev, $uid, $uqs, $uqh, 0, 0)) {
> > >            print "Quota set for $uid, $domain, $userNum, QUOTA\: $userquota\n";
> > >     }
> > >     print $fqdn." ".$userUID." ".$userNum." ".$userquota." ".$userquotaExtra."\n";
> > >   }
> > > 
> > >  }
> > > 
> > >  $cce->bye('SUCCESS');
> > >  #exit(0);
> > >  exit(1);
> > > 
> > > }
> > > 
> > > 
> > >   ### Find User account
> > > 
> > >   @userlist = $cce->findx('User', {"site" => "$siteNum"} );
> > > 
> > >   foreach my $user (@userlist) { 
> > >     my ($userOK, $userOBJ, $userOLD, $userNEW) = $cce->get($user);
> > >     my ($userDiskOK, $userDiskOBJ, $userDiskOLD, $userDiskNEW) = $cce->get($user, "Disk");
> > >     #print "${%{$userOBJ}}{name}"." (at mark) "."$domain\n";
> > >     $userNum="${%{$userOBJ}}{'name'}";
> > >     $userquota="${%{$userDiskOBJ}}{'quota'}";
> > >     $userquota*=1024;
> > >     $userquotaExtra=$userquota;
> > >     $userquotaExtra+=1024;
> > > 
> > > 
> > >     $userUID = `quota $userNum | head -1 | awk -F \"uid \" '{print \$2}' | awk -F \"): \" '{print \$1}'` ;
> > >     chomp($userUID);
> > >     $uid = $userUID;
> > >     $uqs =  $userquota;
> > >     $uqh = $userquotaExtra;
> > > 
> > >     unless(Quota::setqlim($dev, $uid, $uqs, $uqh, 0, 0)) {
> > >            print "Quota set for $uid, $domain, $userNum, QUOTA\: $userquota\n";
> > >     }
> > >     print $fqdn." ".$userUID." ".$userNum." ".$userquota." ".$userquotaExtra."\n";
> > >   }
> > > 
> > >  }
> > > 
> > >  $cce->bye('SUCCESS');
> > >  #exit(0);
> > >  exit(1);
> > > 
> > > }
> > > 
> > > 
> > > 
> > > >>>>>>>>>>>>>>>END>>>>>>>meta-verify>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > > 
> > > 
> > > Cheers
> > > patrick
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > On Wed, 1 Feb 2006 00:59:00 +0800 (SGT)
> > > patricko (at mark) staff.singnet.com.sg wrote:
> > > 
> > > > Some ideas,
> > > > 
> > > > use bluequartz system quota.
> > > > Run a script read out all system quotas and
> > > > re-populate via edquota.
> > > > 
> > > > 
> > > > or during maintenance night.
> > > > Export all domain out via cmuExport.
> > > > 
> > > > then
> > > > try destroy one domain and import it back.
> > > >  ~check: quota in tact? if so, continue with others
> > > > 
> > > > or else you need to do up another server and move all
> > > > domains over as OS level - quota are broken beyond repair
> > > > 
> > > > Cheers
> > > > patrick  
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > On Tue, 31 Jan 2006, Gavin Nelmes-Crocker wrote:
> > > > 
> > > > > Guys
> > > > > 
> > > > > I did something very bad
> > > > > 
> > > > > /sbin/quotacheck -c -u -g -a -v -f -m
> > > > > 
> > > > > The background to this was that I had users showing up using far more disk
> > > > > space that they really were, this used to happen on RaQ4's as well.
> > > > > 
> > > > > I started of with  /sbin/quotacheck -c -u -g -a -v
> > > > > 
> > > > > but it complained that I had to force it so I added -f then it complained
> > > > > that it could mount /home as read only but to force I could add -m and I
> > > > > did.
> > > > > 
> > > > > Great - the problem was sorted
> > > > > 
> > > > > DISASTER - I go back into the GUI and it looks correct except I see that
> > > > > everyone now has unlimited quota.  If I edit a user I see that their
> > > > > original quota amount is still there (presumably from the database) so is
> > > > > there a way to rewrite it all back to the system or do I have to go into
> > > > > each user change their quota and then save.
> > > > > 
> > > > > There are several hundred users on the box - so I really don't want to have
> > > > > to edit every single one if I can avoid it :-)
> > > > > 
> > > > > I think the mistake was -c I think I should have used  /sbin/quotacheck -u
> > > > > -g -a -v -f -m
> > > > > 
> > > > > Thanks
> > > > > 
> > > > > Gavin
> > > > > 
> > > > > 
> > > >