UNIX for Dummies Questions & Answers

Hello
I thought of different ways of integrity check for the backup and look for the fastest approach to start programming.
in all these approaches randomness is used.
I would appreciate if someone give more suggestions or correct me.

1- Machine Name Check We can check if the machines were backed up.
• Randomness should not be used in this approach.
Advantage:
- Easy to implement
- quick
Disadvantage:
- ???

2- Timestamp
Timestamp on the file and directories will be checked to be sure if they are the same. This approach needs to compare the timestamp in the file system with the timestamp of a specific file which is stored in the backup. Timestamp is stored in the file system table, so one can look and figure out what the last modified timestamp for a certain file is.
Advantage:
- Easy to implement
- Need not to restore any Data
Disadvantage:
- The content can not be checked

3- Size
Advantage:
- Need not to restore any Data
- Fast Approach
Disadvantage:
- The content can not be checked. two files can have a same size, but two different content

4- Checksum
Advantage:
??
Disadvantage:
- Files need to be restored
- Collision may happen

5- Comparing Byte by Byte
Byte-by-byte comparison of files means that the contents of two files to be compared, is read byte-by-byte (in parallel) and checked for equality.
The cmp utility compares two files of any type and writes the results to the standard output. By default, cmp is silent if the files are the same; if they differ, the byte and line number at which the first difference occurred is reported.
cmp –l file1 file2
Advantage:
- 100% accurate
- Fast as it does less calculation
Disadvantage:
- Expensive for the big files
- Files need to be restored

and I also have a question:
what are the advantages and disadvantages of doing randomness in comparing?

Thanks

You have to trust your backup to a certain extent. The data is likely to change during the actual backup process or during the verification. You really need to ask yourselves these questions first:-.... and probably a whole bunch of other things.

Media failure is less error prone these days, but in any case there should be an error on the return code you should check for. Consider dual tape backups and/or incremental versus everything every day.

It's too big an overall question to deal with just worrying about how to verify it. It may be that you have answered many of these for yourself already, but if you share what you are planning, you can get a consensus if it is a valid approach or suggestions of other things you need to consider.



Robin

I agree with rbatte1.

Firstly, what are you trying to achieve? Is this some kind of academic exercise?

The basic backup utilities like cpio, tar and ufsdump were written years ago by people who did their homework. Most of us use these utilities (or other more expensive backup suites) to implement our backups. If these utilities were 'faulty' they wouldn't have been released as part of the O/S otherwise there would be uproar.

The main considerations we take into account are:

1. Did the backup job exit in a non-zero state, ie, with an error, eg, backup job lost communication with the volume being backed up.

2. Was there a full system crash during the backup that stopped it?

3. Is the media holding the backup readable (eg, is the tape too old and knackered)? Verify this by listing the media, possibly to a file, and then counting number of lines in the file to tell you the number of files backed up. A script could notify you if the number is "out of range".

Even if you run a backup in single user mode some files will still change between backup and verify, eg, log files.

You don't need everything to be exact to the trillionth bytes to perform a full system restore and get the system on its feet.

So tell us all some more information about the need.

Actually I have nothing to do with backup as a software already does it. I have just to monitor the file systems and keep track if they are really backed up due to the high availibility cluster architecture which is used so it would be difficult to keep track if file systems are backed up ok.
as I created some big files and compared the two files with CRC, MD5 and cmp(Byte by Byte), I see Byte by Byte is a faster approach.

for the case that some files change between backup and verify I have thought of this: If timestamp of file > timestamp of backup: go to the next file

I think what you explained has more relation to backup and not monitoring file systems.

Well, not wishing to be pedantic, but the title of the thread is Integrity check for the backup so I'm sure we all assumed that you wanted help with the backup. You first post seems to confirm this.

So infuriating, I have no further comment.

Well I'm afraid that you've lost me too!!!

I don't understand what you are trying to do. The backup is already taken care of (by what software?) but you want to run some other check. To achieve what? Or catch what? But you also indicate that if the file(s) have changed since the backup cycle and verify cycle, then you're going to skip over them.

I definitely need more input to help me understand the requirement here.