Solaris 11 Package Updates

Using a Solaris 11 Repository
Solaris 11 has moved to a package management system that is similar to yum, in that is pulls from a network and manages dependencies.
The command used is pkg - enter pkg --help for a full list of options.

root@solaris:~# pkg publisher
PUBLISHER                             TYPE     STATUS   URI
solaris                               origin   online   http://pkg.oracle.com/solaris/release/
root@solaris:~# pkgrepo info -s http://pkg.oracle.com/solaris/release/
PUBLISHER PACKAGES STATUS           UPDATED
solaris   4292     online           2011-11-09T15:42:00.183118Z
root@solaris:~#
root@solaris:~# pkg update
No updates available for this image.
root@solaris:~#

Before you install a package you can dry-run it:

root@solaris:~# pkg install -nv gcc-3
           Packages to install:         2
     Estimated space available:   9.67 GB
Estimated space to be consumed: 219.43 MB
       Create boot environment:        No
Create backup boot environment:        No
            Services to change:         1
          Rebuild boot archive:        No

Changed packages:
solaris
  developer/gcc-3
    None -> 3.4.3,5.11-0.175.0.0.0.2.537:20111019T093736Z
  developer/gnu-binutils
    None -> 2.19,5.11-0.175.0.0.0.2.537:20111019T095434Z
Services:
  restart_fmri:
    svc:/application/texinfo-update:default
root@solaris:~#

The you go ahead and install the package:

 root@solaris:~# pkg install -nv gcc-3
           Packages to install:         2
     Estimated space available:   9.67 GB
Estimated space to be consumed: 219.43 MB
       Create boot environment:        No
Create backup boot environment:        No
            Services to change:         1
          Rebuild boot archive:        No

Changed packages:
solaris
  developer/gcc-3
    None -> 3.4.3,5.11-0.175.0.0.0.2.537:20111019T093736Z
  developer/gnu-binutils
    None -> 2.19,5.11-0.175.0.0.0.2.537:20111019T095434Z
Services:
  restart_fmri:
    svc:/application/texinfo-update:default
root@solaris:~#

 To list the packages that you have installed:

timj@solaris11vm2:~$ pkg list | head
NAME (PUBLISHER)                                  VERSION                    IFO
SUNWcs                                            0.5.11-0.170               i-r
archiver/gnu-tar                                  1.26-0.175.0.0.0.2.537     i--
audio/audio-utilities                             0.5.11-0.175.0.0.0.2.1     i--
codec/flac                                        1.2.1-0.175.0.0.0.0.0      i--
codec/libtheora                                   1.1.1-0.175.0.0.0.0.0      i--
codec/ogg-vorbis                                  2.30.0-0.175.0.0.0.0.0     i--
codec/speex                                       1.2-0.175.0.0.0.0.0        i--
communication/im/pidgin                           2.10.0-0.175.0.0.0.0.0     i--
compress/bzip2                                    1.0.6-0.175.0.0.0.2.537    i--
timj@solaris11vm2:~$

To list the details of a package:

timj@solaris11vm2:~$ pkg info xcalc
          Name: x11/xcalc
       Summary: xcalc - scientific calculator for X
   Description: xcalc is a scientific calculator desktop accessory that can
                emulate a TI-30 or an HP-10C.
      Category: Applications/Accessories
         State: Installed
     Publisher: solaris
       Version: 1.0.4.1
 Build Release: 5.11
        Branch: 0.175.0.0.0.0.1215
Packaging Date: September 27, 2011 12:58:35 PM
          Size: 104.32 kB
          FMRI: pkg://solaris/x11/xcalc@1.0.4.1,5.11-0.175.0.0.0.0.1215:20110927T125835Z
timj@solaris11vm2:~$

Want to know where a file comes from?:

timj@solaris11vm1:~$ pkg search libresolv.so
INDEX      ACTION VALUE                        PACKAGE
basename   link   lib/sparcv9/libresolv.so     pkg:/system/library@0.5.11-0.175.0.0.0.2.1
basename   link   usr/lib/sparcv9/libresolv.so pkg:/system/library@0.5.11-0.175.0.0.0.2.1
basename   link   lib/libresolv.so             pkg:/system/library@0.5.11-0.175.0.0.0.2.1
basename   link   usr/lib/amd64/libresolv.so   pkg:/system/library@0.5.11-0.175.0.0.0.2.1
basename   link   lib/amd64/libresolv.so       pkg:/system/library@0.5.11-0.175.0.0.0.2.1
basename   link   usr/lib/libresolv.so         pkg:/system/library@0.5.11-0.175.0.0.0.2.1
timj@solaris11vm1:~$

To update all packages:

root@solaris11vm1:~# pkg update
No updates available for this image.
root@solaris11vm1:~# 

What to do when things go wrong:

root@solaris11vm1:~# pkg verify -v   gcc-3
PACKAGE                                                                 STATUS
pkg://solaris/developer/gcc-3                                               OK
root@solaris11vm1:~# mkfile 1024 /usr/sfw/bin/gcc
root@solaris11vm1:~# pkg verify -v   gcc-3
PACKAGE                                                                 STATUS
pkg://solaris/developer/gcc-3                                            ERROR
        file: usr/sfw/bin/i386-pc-solaris2.11-gcc-3.4.3
                Mode: 01600 should be 0555
                Unexpected Exception: Request error: class file/memory mismatch
root@solaris11vm1:~# pkg fix   gcc-3
Verifying: pkg://solaris/developer/gcc-3                        ERROR        
        file: usr/sfw/bin/i386-pc-solaris2.11-gcc-3.4.3
                Mode: 01600 should be 0555
                Unexpected Exception: Request error: class file/memory mismatch
Created ZFS snapshot: 2011-12-07-15:27:53
Repairing: pkg://solaris/developer/gcc-3                   
                                                                             

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         1/1      0.1/0.1

PHASE                                        ACTIONS
Update Phase                                     3/3

PHASE                                          ITEMS
Image State Update Phase                         2/2
root@solaris11vm1:~# pkg verify -v   gcc-3
PACKAGE                                                                 STATUS
pkg://solaris/developer/gcc-3                                               OK
root@solaris11vm1:~#

Mail merge in PHP

Just for a laugh

The question was, how do you do a mail merge in PHP. OK, it was a question that no-one was asking, but I never let the deter me before.

Input Data $ cat inputdata Tim,Jinkerson, Ben,De Mora, Jes,Ferrier, tjinkers@TJINKERS-GB ~/merge $ cat template FAO: .f1 .f2 Dear .f1 Now is the winter of our discontent made glorious summer by this sun of York Many thanks Tim tjinkers@TJINKERS-GB ~/merge $

The Program

$ cat mymerge.php $filename = "template"; $mergef = "inputdata"; $fdm = fopen($mergef,"r"); $fnum=1; $mdata = fgets($fdm, 512); while (!feof($fdm)) {   list($f1, $f2, $f3, $f4, $f5) = explode(",", $mdata);   $fd = fopen($filename,"r");   $contents = fread ($fd, filesize($filename));   fclose ($fd);   $ofilename = "file.".$f1.".".$f2.".".$fnum;   $contents=str_replace(".f1", $f1, $contents);   $contents=str_replace(".f2", $f2, $contents);   $contents=str_replace(".f3", $f3, $contents);   $contents=str_replace(".f4", $f4, $contents);   $contents=str_replace(".f5", $f5, $contents);   $fdo = fopen($ofilename,"w");   fwrite ($fdo,$contents);   fclose ($fdo);   $fnum++;   $mdata = fgets($fdm, 512); } fclose ($mergef); ?>

tjinkers@TJINKERS-GB ~/merge
$

For each line of the input data, the program reads the template file as a single string, and then does a replace on the markers for the input date. It writes the results out to a file who's name is based on the input data.

$ php mymerge.php $ ls file.Ben.De Mora.2  file.Jes.Ferrier.3  file.Tim.Jinkerson.1  inputdata  mymerge.php  template tjinkers@TJINKERS-GB ~/merge $ cat file.Tim.Jinkerson.1 FAO: Tim Jinkerson Dear Tim Now is the winter of our discontent made glorious summer by this sun of York Many thanks Tim tjinkers@TJINKERS-GB ~/merge

$

Creating a Solaris 10 Sparse Zone

 First, create a directory for your zone to be built in.

mkdir -p /zones/myzone

 Create a zone config file.

create -b
set zonepath=/zones/myzone
set autoboot=true
add inherit-pkg-dir
set dir=/lib
end
add inherit-pkg-dir
set dir=/platform
end
add inherit-pkg-dir
set dir=/sbin
end
add inherit-pkg-dir
set dir=/usr
end
add net
set address=10.0.2.17/24
set physical=hme0
end

 zonecfg -z myzone -f sparsezone.cfg
 zoneadm -z myzone install

 zoneadm list -v

zlogin -C myzone 
zlogin myzone shutdown -y -i0 -g0

Creating a second database from the linux command line

The objective  here was to make a copy of an existing database and bring it up under a new instance. I could in theory have used a different version of the database but at the time I only had one available to me.

Create a new database:

su - root
mkdir /var/lib/postgresql/testinstance
chown postgres:postgres /var/lib/postgresql/testinstance
su - postgres
initdb -D /var/lib/postgresql/testinstance

Starting the Instance

pg_ctl start -D /var/lib/postgresql/testinstance -l /var/tmp/testinstance.log -o "-p 5434"

Check the status:

pg_ctl status -D /var/lib/postgresql/testinstance

Stopping the Instance

pg_ctl stop -D /var/lib/postgresql/testinstance

Create user on new database:

create role timj login password 'xxxxxx';
create database cddb;
alter role timj superuser;
alter role timj nosuperuser;
grant all on database cddb to timj;

Connecting to the database:

psql -d postgres -h localhost -p 5434 -U postgres

Dump the old database:

/usr/bin/pg_dump -h localhost -p 5432 -U timj -F p -v -f "/tmp/mybackup.pgsql" cddb

Restore to new database:

psql -h localhost -p 5434 -U timj -d cddb < /tmp/mybackup.pgsql

Dumping database in custom mode:

/usr/bin/pg_dump -h localhost -p 5432 -U timj -F c -v -f "/tmp/mybackup.pgsqlc" cddb

Use pg_restore if you have a non-ascii backup:

create role timj login password 'xxxxxx';
create database cddb;
grant all on database cddb to timj;
pg_restore -h localhost -p 5434 -U timj -d cddb /tmp/mybackup.pgsqlc

Point-In-Time Recover in Postgresql

The way you do point in time recovery in Postgresql is to restore a known backup, and then reapply WAL logs until you reach the time you wish to stop at. The database must be in Archive Log Mode (just like Oracle), and you must have all the logs to complete the recovery.

Putting the database in Archive Log Mode

Step 1:
Back up your database
pg_dump -h localhost -p 5432 -U timj -F c -v -f "D:\pgbackup\cddb_21Dec2009_2.backup" cddb

Now let's put the database in archive log mode:

Edit postgresql.conf:

fsync = on
synchronous_commit = on
wal_sync_method = fsync
full_page_writes = on
archive_mode = on
archive_command = 'copy "%p" "D:\\pglogs\\%f"'
archive_timeout = 3600 # switch logs every hour

And remember to bounce the database.

Taking a base backup

I now have to take a base backup of my system.
To do this I logged in as postgres and issued the following:
SELECT pg_start_backup('label2');

I backed the database up using a normal file system backup with the datbase up, and then issued:
SELECT pg_stop_backup();

Next - put in a transaction that we will want recovering:

insert into artist (name) values ('Vicky Beeching');

As this is my first time doing this, I forced a log switch to help with the recovery!

SELECT pg_switch_xlog();

The Disaster!

After my backup, I created a record in the artist table that should be there post recovery. The table has a time trigger on it, so I knew exactky the time I wanted to recover to. I then deleted Nick Drake's entry from the artist table - could there be a bigger mistake than that!

The Recovery

Step 1: Take the database down. This may seem like the obvious first step, but there is a gotchya. The default checkpoint time-out in Postgresql is 5 minutes. This means that if you are relying on recoving your database up to the last good transaction, and if that transaction was less than 5 minutes before hand, it may not have been written to a log file in the pg_xlog directory. Therefore, it may be worth while logging on as an admin user and issuing the command checkpoint; before you shut down the instance.

Step 2: Take a backup!
With the database down, copy off the data directory and any other tablespaces you have in use.

Step 3: Delete your database!
We are going to rebuild the database using the good copy we created in our initial backup, the archived WAL logs and the WAL logs from pg_xlog, so we need to get rid of everything in the existing database.

Step 4: Copy across your backed up database
Copy your good backup into the live data directory.

Step 5: Delete everything from pg_xlog
The files in the restored pg_xlog directory will be from before the backup, and so of no use to you.

Step 6: Copy across the content of your pg_xlog directory
From the backup you took after you discovered you needed to recover, copy the files to your live pg_xlog directory.

Step 7: Create a recovery.conf file
Something like this:

restore_command = 'copy "D:\\pglogs\\%f" "%p"' # Windows
recovery_target_time ('2009-12-22 15:34:56')

The copy command should be in the syntax of whatever OS you are using, very much like the entry in postgresql.conf

Step 8: Restart the database
You might want to restrict who can access while you are checking the recovery.
At the end of the recovery recovery.conf is renamed recovery.done

Step 9: Take a fresh baseline baclup!
It is really good practice to do this before your users start posting transactions, if at all possible.

Solaris 10 CPU Usage

psrinfo shows how many CPUs you have:

bash-3.00# psrinfo 0       on-line   since 08/24/2009 13:40:13 1       on-line   since 08/24/2009 13:40:18 2       on-line   since 08/24/2009 13:40:18 3       on-line   since 08/24/2009 13:40:18 4       on-line   since 08/24/2009 13:40:18 5       on-line   since 08/24/2009 13:40:18 6       on-line   since 08/24/2009 13:40:18 7       on-line   since 08/24/2009 13:40:18 8       on-line   since 08/24/2009 13:40:18

adding the –v option gives you details about those CPUs

bash-3.00# psrinfo -v | head Status of virtual processor 0 as of: 11/19/2009 03:50:06   on-line since 08/24/2009 13:40:13.   The sparcv9 processor operates at 1165 MHz,         and has a sparcv9 floating point processor. Status of virtual processor 1 as of: 11/19/2009 03:50:06   on-line since 08/24/2009 13:40:18.   The sparcv9 processor operates at 1165 MHz,         and has a sparcv9 floating point processor. Status of virtual processor 2 as of: 11/19/2009 03:50:06   on-line since 08/24/2009 13:40:18. bash-3.00#

“prstat -s cpu -n 10” is a bit like top, but it tells you what cpu you are running against

bash-3.00# prstat -s cpu -n 10 PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 3138 900233 895M 588M cpu16 1 0 141:24:49 1.7% mysqld/26 1005 root 19M 18M sleep 59 0 38:34:21 0.0% esd/1 944 daemon 3192K 2208K sleep 29 0 0:00:00 0.0% statd/1 542 root 1768K 256K sleep 29 0 0:00:00 0.0% efdaemon/1 8633 root 1712K 1168K sleep 1 0 0:00:16 0.0% utmpd/1 165 root 7568K 4920K sleep 29 0 0:00:00 0.0% devfsadm/8 940 root 2992K 2136K sleep 29 0 0:00:00 0.0% keyserv/3 147 root 7464K 4472K sleep 29 0 0:00:00 0.0% syseventd/15 9 root 11M 11M sleep 29 0 1:00:07 0.0% svc.configd/12 7 root 32M 31M sleep 29 0 0:04:45 0.0% svc.startd/13 Total: 129 processes, 541 lwps, load averages: 1.12, 1.14, 1.18 bash-3.00#

You can the use the mpstat command to see how each processor is doing:

bash-3.00# mpstat CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 80 0 566 264 53 50 0 2 9 0 203 2 2 0 96 1 1 0 193 15 2 25 0 0 1 0 57 0 0 0 99 2 0 0 48 8 0 15 0 0 0 0 12 0 0 0 100 3 0 0 8 3 0 5 0 0 0 0 4 0 0 0 100 4 15 0 72 8 0 15 0 0 2 0 33 0 0 0 99 5 0 0 4 3 0 4 0 0 0 0 1 0 0 0 100 6 0 0 5 7 1 10 0 0 0 0 0 0 0 0 100 7 0 0 5 6 0 9 0 0 0 0 0 0 0 0 100 8 64 0 369 22 0 47 0 2 7 0 185 3 1 0 96 9 3 0 73 3 0 6 0 0 1 0 22 0 0 0 100

“mpstat –p 10 10”  will refresh the screen every 10 seconds

Perl: Chose your Database Vendor on the command line!

I have created two identical databases. Well, when I say 'identical', one is MySQL and the other is PostgreSQL, but other than that they are pretty similar! They have the same schema and pretty much the same data. It occurred to me that the Perl DBI package hides the database details away from you so successfully that it would be possible to write a program that allowed you to specify the type of database you are accessing on the command line!

Firstly, my new version of the command line routine:

sub dbinfo() {
        use Term::ReadKey;
        my $helpinstructions = "Valid parameters are\n-h {host}\n" .
                               "-d {database}\n" . 
                               "-u {user}\n" .
                               "-p [password]\n" .
                               "-dms [mysql/pgsql]\n";
        my %mydbinfo = (        db => "",
                                host => "",
                                user => "",
                                passwd => "",
                                dms => ""
        );


        die $helpinstructions unless ($#ARGV > 5);

        for (my $i=0; $i <= $#ARGV; $i++) {
                $ARGV[$i]= "-?" if ($ARGV[$i] ne '-h' and
                                    $ARGV[$i] ne '-d' and
                                    $ARGV[$i] ne '-u' and
                                    $ARGV[$i] ne '-dms' and
                                    $ARGV[$i] ne '-p' );
                die $helpinstructions if ( $ARGV[$i] eq "-?" ) ;
                $mydbinfo{db} = $ARGV[++$i] if ( $ARGV[$i] eq "-d" ) ;
                $mydbinfo{host} = $ARGV[++$i] if ( $ARGV[$i] eq "-h" ) ;
                $mydbinfo{user} = $ARGV[++$i] if ( $ARGV[$i] eq "-u" ) ;
                $mydbinfo{dms} = $ARGV[++$i] if ( $ARGV[$i] eq "-dms" ) ;
                if ( $ARGV[$i] eq "-p" ) {
                        if ( $i ne $#ARGV) {
                                $mydbinfo{passwd} = $ARGV[++$i];
                        }
                        else {
                                print "\nPlease Enter passwd: ";
                                ReadMode 'noecho';
                                chomp($mydbinfo{passwd} = ReadLine);
                                ReadMode 'normal';
                                print "\n\n";
                        }
                }
        
        }
        die $helpinstructions unless ( $mydbinfo{dms} eq 'mysql' or
                                       $mydbinfo{dms} eq 'pgsql');
        $mydbinfo{dms} = 'PgPP' if ($mydbinfo{dms} eq 'pgsql');
        return %mydbinfo;
}
return true;

And then the program that calls it:

#!/usr/bin/perl
use strict;
use DBI();

require "D:\\Documents and Settings\\timj\\perl\\choosedbcommanddbsettings.pl" or
die "Can't Open DB Settings File";


my %mydbinfo = dbinfo();
my $dbconnect = "DBI:" . $mydbinfo{dms} . ":database=" . $mydbinfo{db} . ";host=" . $mydbinfo{host} ;

my $dbh1 = DBI->connect($dbconnect,$mydbinfo{user}, $mydbinfo{passwd}, { raiseError => 1, AutoCommit => 0 })
|| die "Database connection not made: $DBI::errstr";
my $sql = "SELECT artist.name, cd.title FROM artist, cdtable AS cd WHERE artist.id = cd.artid";
my $sth = $dbh1->prepare($sql);

$sth->execute();

my( $name, $title );
$sth->bind_columns( \$name, \$title );

print "Name\t\tTitle\n";
while( $sth->fetch() ) {
        print "$name\t$title\n";
}
$dbh1->disconnect() if($dbh1);

and here's the output:

D:\Documents and Settings\timj\perl>choosedbreport.pl -dms mysql -h localhost -u timj -d cddb -p

Please Enter passwd:

Name            Title
Phatfish        Guaranteed
Phatfish        15
Bethany Dillon  Waking Up
Bethany Dillon  Bethany Dillon
Bethany Dillon  Imagination
Nick Drake      Made To Love Magic
Nick Drake      Five Leaves Left
Nick Drake      Bryter Layter

D:\Documents and Settings\timj\perl>choosedbreport.pl -dms pgsql -h localhost -u timj -d cddb -p

Please Enter passwd:

Name            Title
Nick Drake      Made To Love Magic
Nick Drake      Five Leaves Left
Nick Drake      Bryter Layter
Bethany Dillon  Waking Up
Bethany Dillon  Bethany Dillon
Bethany Dillon  Imagination
Phatfish        Guaranteed
Phatfish        15
Portland        These Broken Hands

D:\Documents and Settings\timj\perl>

Maybe not very useful, but it's a good demonstration of the power of DBI();