Login or Register to Ask a Question and Join Our Community


AIX

AIX & TAR related stuff

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems AIX AIX & TAR related stuff
# 1  
Old 02-22-2013
AIX & TAR related stuff
This thread is about using tar & other compression utilities on AIX ;

1. Find out which version of tar you are using
thanks to bakunin

Code:
>what $(which tar)
/usr/bin/tar:
        61      1.14  src/bos/usr/ccs/lib/libc/__threads_init.c, libcthrd, bos53  0 7/11/00 12:04:14
        10  1.49.1.75  src/bos/usr/bin/tar/tar.c, cmdarch, bos53A, a2004_38A1 8/  31/04 07:07:20
        92      1.7  src/bos/usr/ccs/lib/libIN/eprintf.c, libIN, bos530 6/10/91   10:23:13

>lslpp -L |grep -i tar
  devices.scsi.tm.rte       5.3.0.10    C     F    SCSI Target Mode Software
  devices.ssa.tm.rte         5.3.0.0    C     F    Target Mode SSA Support
  tar                         1.14-2    C     R    A GNU file archiving program.

>tar --version
tar: Not a recognized flag: -
Usage: tar -{c|r|t|u|x} [ -BdDEFhilmopRUsvw ] [ -Number ] [ -f TarFile ]
           [ -b Blocks ] [ -S [ Feet ] | [ Feet@Density ] | [ Blocksb ] ]
           [ -L InputList ] [-X ExcludeFile] [ -N Blocks ] [ -C Directory ] File ...
Usage: tar {c|r|t|u|x} [ bBdDEfFhilLXmNopRsSUvw[0-9] ] ]
           [ Blocks ] [ TarFile ] [ InputList ] [ ExcludeFile ]
           [ [ Feet ] | [ Feet@Density ] | [ Blocksb ] ] [-C Directory ] File ...

> find / -name tar
/opt/freeware/bin/tar
/usr/bin/tar
/usr/linux/bin/tar


2. GNU Tar

why it is important to know which version of tar you are using ?

Code:
# tar cvf - file | gzip > file.tar.gz
tar: 0511-825 The file 'file' is too large.

AIX tar has a 2 GB limit to the file size. On UNIX tar there are some limits: avg. 12 GB of file archive (may vary depend of UNIX flavor), 6 level of directories (all UNIX, but not GNU tar), 32k directories on one level (this is a limitation of JFS and JFS2) Use tar from the Linux Toolbox CD - can get it alternatively here: https://www14.software.ibm.com/webap...G=tar&cp=UTF-8
or
Welcome to Bull AIX freeware site
ftp://ftp.software.ibm.com/aix/freeS...aix5.1.ppc.rpm

Procedure
Find the native AIX tar location.
Code:
#which tar
/usr/bin/tar

Move the native binary tar command:
Code:
#cd /usr/bin
#mv tar tar_

Install the GNU tar, which can be obtained from the AIX toolbox site: IBM AIX Toolbox download information
Set the install location to be something similar to /opt/freeware/bin/tar

Create a gnu tar soft link:
Code:
ln -s  /opt/freeware/bin/tar /usr/bin/tar

Validate that the tar command is gnu tar:
Code:
#tar --version



3. Popular Tar options

-c = create
-f = read to/from the named file (instead of the device /tape)
-t = list contents of .tar file
-r = append to a .tar file
-v = verbose (tells you everything its doing)
-x = extract contents of .tar file
-z = compress files


4. tar: 0511-169 A directory checksum error on media ; 804399360 not equal to 12401.

Solution : https://www.unix.com/aix/213511-solve...ror-media.html


5. How to find if particular file is present in the tar file ?

6. How to find the size of particular file in the tar file ?
Last edited by filosophizer; 02-23-2013 at 04:54 PM..
# 2  
Old 02-22-2013
5 and 6
supply the filename (including path) to tar
Code:
skrynesaver@busybox ~$ ls tmp/
1.txt  2.txt  lac_sum.pl  regex.pl  time_check.pl  tmp.dat  tmp.dat.bak  tmp.pl  tmp.tok
skrynesaver@busybox ~$ tar -cjvf tmp.tbz2 tmp
tmp/
tmp/1.txt
tmp/2.txt
tmp/lac_sum.pl
tmp/regex.pl
tmp/time_check.pl
tmp/tmp.dat
tmp/tmp.dat.bak
tmp/tmp.pl
tmp/tmp.tok
skrynesaver@busybox ~$ tar -tjvf tmp.tbz2 tmp/regex.pl
-rw-r--r-- skrynesaver users 374 2013-02-12 16:20 tmp/regex.pl
skrynesaver@busybox ~$

This User Gave Thanks to Skrynesaver For This Post:
filosophizer
# 3  
Old 02-22-2013
Quote:
Originally Posted by filosophizer
AIX tar has a 2 GB limit to the file size.
AIX adheres to the "USTAR" standard. The 2GB limit was lifted with AIX 5.1, as shockneck pointed out. AIX 5.1 was out of software service 2005 or so - too long ago, in any case, to still use it in any conclusions. In fact, quoting from the tar man page for AIX 6.1:

Quote:
The ustar header format allows for file sizes to be as large as 8 GB. Therefore, the tar command is enabled to archive files of up to 8 GB in size.
Further, you can easily work around some limitations by using <stdin>/<stdout> as input/output destinations:

Code:
tar -cf - some_large_dir > /path/to/outfile.tar
cat /path/to/outfile.tar | tar -xf -

Quote:
Originally Posted by filosophizer
Move the native binary tar command:
Code:
#cd /usr/bin
#mv tar tar_

IMHO this is not a good way to do it. You change the OS by doing so and that is most probably not what you want. First, you will run into troubles when you update or apply corrective services, because the install routines of these procedures will expect "tar" (the executable, not some spurious link) to be found at "/usr/bin/tar", not at "/usr/bin/tar_". Second, you mislead tools which may rely on a certain behavior of the command they are using. These tools may or may not fail simply because they expect behavior X but experience behavior Y instead.

You may install any tar replacement either in a different location (/opt/gnutar-version/bin/tar if you think of it as an application, /usr/local/bin/tar if you think it is an additional administrative tool) and set your PATH accordingly (just put the part where GNU-tar resides in front of /usr/bin) or you can install GNU-tar under a different name ("gtar" for instance) and define an alias for your scripts and/or shells. But you should NEVER, NEVER EVER replace system commands with home-brewn concoctions.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
filosophizer
# 4  
Old 02-23-2013
If it is just to archive files and directories on AIX - consider the backup file format (bff) and the backupbyname option of backup.

I try to stay away from tar for a couple of reasons (does not know about ACL or most special files, as examples) so instead I use the following:

Code:
$ cd /some/dir/needing/backup
$ find . | backup -if - | bzip2 >/my/backup/directory/backupName.bff.bz2

And when I want to unpack, examine, etc

$ bzip2 -dc /my/backup/directory/backupName.bff.bz2 | restore -Tqf -

Don't forget to set fsize to unlimited. To change the fsize default:
Code:
chuser fsize=-1 default

otherwise, to change ony for michael and root
Code:
chuser fsize=-1 root
chuser fsize=-1 michael

Sure, it is not tar - but it works quite well, also for copying files/directories.
Last edited by MichaelFelt; 02-24-2013 at 07:08 AM..
These 2 Users Gave Thanks to MichaelFelt For This Post:
filosophizer landog
# 5  
Old 02-23-2013
Thank you for contributing to the thread:

@MichaelFelt : tar is recommended by oracle for cold backup. it is widely used for this purpose ( backup ) otherwise what you have mentioned is very reasonable.

@bakunin : thank you for the great valuable tips


7. How to find out if you have any errors during creating a tar

Suppose you are making a tar file of a filesystem which has several directories and files ( 100GB )
you execute the command
# tar -cvf tarfile_name.tar /filesystem

this is will start creating the tar file and will take several minutes/hours

how can you find out , if you have any errors during creating the tar archive :
errors like => tar: 0511-825 The file 'file' is too large
or any other errors ?

8. Is there a way to verify if all the files have been included in the tar achieve ?


9. How to list the number of files in the tar file ?
# 6  
Old 02-23-2013
Quote:
Originally Posted by filosophizer
how can you find out , if you have any errors during creating the tar archive :
errors like => tar: 0511-825 The file 'file' is too large
or any other errors ?
After the tar has finished you can consult the exit code (=errorlevel). As always in Unix, "0" means no error, any other code means various sorts of error conditions. Check the man page for further information.

Quote:
Originally Posted by filosophizer
8. Is there a way to verify if all the files have been included in the tar achieve ?


9. How to list the number of files in the tar file ?
The answer to these questions is one: you can use

Code:
tar -t

to "test" a tarfile. With the "-v" option it will display a listing rather similar to a directory listing. Use this and a "ls -lR" as a reference to check if all files are included (if you haven't access to the error output tar produces in case of errors and/or the above mentioned exit code) and pipe the output in "wc -l" to count files.

Regarding Oracle recommending "tar": any command which backs up many files into one archive is fair game. Don't cling too much to what DB-manufacturers tell you. If their OS knowledge would be so hot stuff they wouldn't do database programming, after all, would they? ;-))

Seriously: MichaelFelt has it absolutely correct: "backup" is a full-blown replacement for whatever you ever used "tar" for. I use "tar" myself, mostly out of habit, but have to acknowledge that "backup" has no drawbacks at all (and possibly some advantages).

I hope this helps.

bakunin
Last edited by bakunin; 02-23-2013 at 05:11 PM..
This User Gave Thanks to bakunin For This Post:
filosophizer
# 7  
Old 02-24-2013
Quote:
tar is recommended by oracle for cold backup. it is widely used for this purpose ( backup )...
Nods. TAR is a generic solution and this recommendation has wide applicability. That does not imply that it is a best practice for any platform.

Tar will often be a good choice for saving - I use it frequently - and opensource projects are usually distributed as a tar ball, but TAR is not always so great for restoring. Not sure what the status is of AIX tar atm, but it used to be if you restored a file using tar all the zero bytes that are not on disk actually get written to the "tape" AND these zero bytes get restored. (so called sparse files getting backed up and restored as "full" files. There is a real danger that the cold backup causes some hot tempers when it does not fit where it is supposed to :winkSmilie). In other words, a file/database that fit before the restore does not fit the same file system afterwards. This is what got me started >10 years ago to use backup and restore.

Granted, using backup and restore is less portable than tar, but any platform dependent tool understands everything there is to understand about files living on their system. Think: Backup and Restore are the generic UNIX tools for system backups.
I do not know what Solaris, HPUX, "Linux" recommend for system backups. If it is tar they will have added features to make it able to backup files, directories and device definitions. However, even though it is a "tar" file, it is not really portable.

I diverge - getting back on track - I would recommend taking a look at the AIX savevg script, or for something simpler my "HOWTO: copy directories on AIX". It is not exactly what your are doing for your cold backups, but I hope it is easy to improve - to fit to your needs.

Any questions about either script - Post and I will do my best to answer them here!

Or - make some improvements and post them back! I know there are many new features in backup and restore (working with encryption, e.g.). Much room for improvement.
This User Gave Thanks to MichaelFelt For This Post:
filosophizer
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

GNU TAR vs NATIVE AIX TAR

Hello, Getting this very strange error, made tar/zip through gnu tar GNU Tar ( successful tar and zip without any errors ) /opt/freeware/bin/tar cvf - /oraapp| gzip > /backup/bkp_15_6_16_oraapp.tgz GNU unTar error root@test8:/>gunzip < /config1/bkp_15_6_16_oraapp.tgz |... (5 Replies)
Discussion started by: filosophizer
5 Replies

2. Shell Programming and Scripting

help with tar & zip only last months(say,Sep) files

Need to 1. archive all the files in a directory from the previous month into a tar/gz file, ignoring all already archived 'tar.gz' files 2. Check created .tar.gz file isnt corrupted and has all the required files in it. and then remove the original files. I am using a function to get the... (1 Reply)
Discussion started by: Prev
1 Replies

3. AIX

How to upgrade AIX Firmware & TL Maintenance Level in AIX

Steps to upgrade AIX TL ( technology Level ) / Maintenance Level in AIX ( including Firmware HMC VIOS ) This article or post covers upgrades for - Hardware Management Console ( HMC ) - Firmware ( also known as microcode ) - VIO ( Virtual I/O Server = PowerVM ) - AIX Version, Technology... (2 Replies)
Discussion started by: filosophizer
2 Replies

4. Solaris

Few things related to HBA port & fcinfo command

Hi Gurus I need to know few things related to HBA port & fcinfo command I have a server where there are 4 HBA ports cards are their. Out of 4 ports 2 are in use & 2 are not in use when I check it physicall. Now I want to know the command through which I can get information about all above... (6 Replies)
Discussion started by: girish.batra
6 Replies

5. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

6. Solaris

Tar & Tape drive without media

Hi all, I would like to know what would happen if the tape (media) is not placed on the drive and a tar command is executed to backup on the tape. My problem is that tar command hanged for multiple days instead of throwing the error, Is it valid behaviour? I was unable to test the... (4 Replies)
Discussion started by: jmsathish
4 Replies

7. UNIX for Dummies Questions & Answers

tar & Grep together

Hi, I've got dozens of tar's with two files in each one, live_access_log & live_error_log (one tar for each day, backups). The probelm is i need to match a pattern in all of the archive_access_log files and output the line to a seperate file (All_access.log). I.e. I need to get details... (21 Replies)
Discussion started by: tom123
21 Replies

8. UNIX for Dummies Questions & Answers

disaster recover w/tar & find

Hi, I am creating a disaster recovery plan for my Linux 7.2 machine. I have two backups from my current machine. One created using the command tar -cvpf /dev/st0 --exclude=/proc --directory / . and one created with the command find / /boot /home -mount -path '/proc' -prune -o -print |... (4 Replies)
Discussion started by: jeremiebarber
4 Replies

9. UNIX for Dummies Questions & Answers

/dev/hd* ?? & HTFS .... What is this stuff?

# grep "Jul 3" syslog.messages | more Jul 3 00:16:03 www3 CPU3: NOTICE: HTFS: No space on dev hd (1/42) Jul 3 00:16:08 www3 CPU3: NOTICE: HTFS: No space on dev hd (1/42) Jul 3 00:17:01 www3 CPU2: NOTICE: HTFS: No space on dev hd (1/42) Jul 3 00:17:06 www3 syslogd: /usr/adm/debug: No space... (1 Reply)
Discussion started by: Cameron
1 Replies

10. Shell Programming and Scripting

Find & tar execution problem

I'm trying to set up a stanard sh script that will find all the files that have been changed within the last day and then tar them up. I think the command line should be something like : find /home/bob -atime +0 -exec \ tar cvf /home/bob/files.tar {}\; Help please ... Thanx (3 Replies)
Discussion started by: ianf
3 Replies
Login or Register to Ask a Question