Cpio - input files (from list) are stored in different order inside cpio archive - why?
Due to budget constraints I have to reinvent an Enterprise backup system in a SPARC (sun4v) Solaris estate (10 & 11). (yep - reinvent wheel, fun but time consuming. Is this wise?! )
For each filesystem of interest, to try to capture a 'catalog' at the front of each cpio archive (for an easy scripted restore system that I will write later), I touch a file .${DIR}/.${OFILE}.fullscan that will hold a full filesystem scan. Then I run this to populate the first 'record' in the catalog file, so that the catalog file always sits at the head of the archive:
This populates our file list:
This then runs the actual cpio operation:
9 times out of 10 my ${OFILE}.fullscan appears in the first few files in the cpio archive. Occasionally it's a few files 'lower down' but always in the first 20. So good so far.
Today, on a Solaris 11.2 system I found the fullscan file over 1000 files into one of the cpio archives and another one over 6400 files into the archive. (In another they appeared at the end but I'm still checking that's not a 'code' issue!) Why?! Help!
I checked the text file content to make sure the top record was as expected for the examples that put the fullscan file much farther down the archive.
My worst case scenario is a 57M file filesystem (yep - source code repo) which generates a 9.5GB 'fullscan' file (over 11 hours) and due to other bits and pieces I need to do, I really, really don't want my catalogs appearing half way through that one. (This supersized filesystem backup will inevitably be broken up into smaller tasks but for now I'm just asking.)
Is this a multithreading effect? I don't believe awk or cut would reorder the list and so cpio would receive ${OFILE}.fullscan as the first argument. The file in question is being read but that wouldn't generate an exclusive lock to prevent access or anything like that from another read process. Note: for what it's worth these 2 anomalies occurred in /var of a guest Solaris zone that's visible from the global zone.
Can anyone think of a way to:
Assuming there isn't a trivial answer to this - debug this easily for an explanation? (Bear in mind it's intermittent.)
Workaround this? I want the catalog to always be quickly and easly accessed from (the top of?) many 100GB+ gzipped cpio archives! How can we persuade cpio to load files into the archive in exactly the order its file list is fed to it?
I think your problem is that find takes the order in the directory.
In a simple file system initialy the order in the directory defaults to the order in which files are created. But if files are deleted, a new file can take the position of a deleted entry.
Not to speak of more complex file systems that use a hash table.
What is your file system type?
Perhaps you do not need to reinvent the wheel. There is ufsdump/ufsrestore for the ufs file system, and certainly another method for a zfs file system.
Hi I'm new to the forum and looking for some help with cpio archive creation.
I have a bunch of directories that I need to make into a cpio archive.
The problem I'm having is that when I input the commands I get the cpio archive but it creates a duplicate of the archive inside the archive... (4 Replies)
I need a hint for reading manpage (I did rtfm really) of cpio to do this task as in the headline described. I want to put all files of a certain type, lets say all *.txt files or any other format. Spread in more than hundreds of subdirectories in one directory I would like to select them and just... (3 Replies)
So I ran into a strange problem the other day when using cpio. I'm not really sure this needed to be in an HPUX specific forum, but wasn't sure where else to put it. I'm running HPUX 11.11 PARISC.
I was using cpio to copy a directory and its entire contents from one server, we'll call it... (7 Replies)
I created a CPIO archive I wanted to add addition data to it but am having issues:
-rw-r--r-- 1 test test 629295104 2011-10-28 12:41 /home/test/Downloads/test.cpio
I tried:
sudo find /tmp -depth | cpio -oAO /home/test/Downloads/test.cpio
cpio: premature end of file
and (1 Reply)
Hello Every one,
I want to back up all passwd files to /xyz/passfiles.cpio
and Then restore them to /abc directory.
Here is what I wrote:
find / -name passwd | cpio -oc > /tmp/passwd.cpio
and to restore
cd abc
cpio -ium < /tmp/passwd.cpio
I can not find the files restored to /abc... (2 Replies)
Hi all,
I want to unpack some files .Files and their sizes are:
1. Linux9i_Disk1.cpio -- 500m
2. Linux9i_Disk2.cpio--- 600m
3.Linux9i_Disk3.cpio---- 250m
I used cpio -idmv Linux9i_Disk1.cpio command to unpack the files. But Its taking more time to unpack the files.What could be the... (2 Replies)
Hi,
Currently a backup script copies compressed files to tape using cpio command (on AIX 5.2). Recently we've had a compressed file which has gone over 2 GB in size resulting in an error while copying this file onto the tape using cpio.
Any suggestions on relevant workarounds would be much... (2 Replies)
Hi,
Currently a backup script copies compressed files to tape using cpio command (on AIX 5.2). Recently we've had a compressed file which has gone over 2 GB in size resulting in an error while copying this file onto the tape using cpio.
Any suggestions on relevant workarounds would be much... (0 Replies)
Hi,
I use following command to restore data from my cpio tape archive:
$cpio -icvd < /dev/rct0
But this'll restore all tape contents to the current path, what if I want only selected files from the tape, suppose I want
/home/compdir/home2/Rev83/data/PL/01/*.*
files to be restored... (8 Replies)