cksum's and zip's CRC32 algorithm


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users cksum's and zip's CRC32 algorithm
# 1  
Old 07-13-2009
cksum's and zip's CRC32 algorithm

Hello!

For long I used cksum to find file duplicates in linux and darwin.
Now I want to make my own program that does all.
However I can't seem to find the correct algorithm.

zip and cksum claim to use the same algorithm, but the computated sums are not the same.
I've already written an algorithm that yelds the same result as zip, but what I want is the specifics of cksum's algorithm.

I already tried to find cksum's source but failed.

Any help is apreciated.
# 2  
Old 07-13-2009
Define 'I already tried to find cksum's source but failed'

you cannot find the source code - or your found it and could not get it to work?
# 3  
Old 07-13-2009
I tried doing some trial and error experimentation with my code and
CRC calculation
using a string with 6 characters, reversing or not the results and bit by bit computations
If is usefull I can post the source for my function.

Googled 'cksum source code' and saw about some 30 different links take where mainly to forum threads asking for the same thing and an old sum BSD code.
I also read it there should be somewhere in my linux dist (fedora 10) but I don't know where (package it comes in).
# 4  
Old 07-17-2009
In some unixes the algorithm is described in "man cksum".
For example:

1p:cksum - Linux Man Pages Manual Documentation for Linux / Solaris / UNIX / BSD
# 5  
Old 07-18-2009
Code:
/** Reverses bit order. MSB -> LSB and LSB -> MSB. */
unsigned int reverse(unsigned int x) {
    unsigned int ret = 0;
    for (int i=0; i<32; ++i) {
        if (x&0x1 != 0) {
            ret |= (0x80000000 >> i);
        }
        else {}
        x = (x >> 1);
    }
    return ret;
}

unsigned int crc32(unsigned char* message, unsigned int msgsize, unsigned int crc) {
    unsigned int i, j; // byte counter, bit counter
    unsigned int byte;
    unsigned int poly = 0x04C11DB7;
    i = 0;
    for (i=0; i<msgsize; ++i) {
        byte = message[i];       // Get next byte.
        byte = reverse(byte);    // 32-bit reversal.
        for (j=0; j<= 7; ++j) {  // Do eight times.
            if ((int)(crc ^ byte) < 0)
                crc = (crc << 1) ^ poly;
            else crc = crc << 1;
            byte = byte << 1;    // Ready next msg bit.
        }
    }
    return reverse(~crc);
}

This is the code I'm using. It reproduces the same results as zip's CRC32.
(try zipping a small file and then unzip -v file on the cmdline). The poly is the same as the one mentioned in the cksum man page.

methyl thanks for the link, it has some pseudo code new to me. I'll try to find the Sarwate article about table generation, maybe it will cast some light on what is done different in cksum.

btw the code was adapted from Software Implementations
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (1 Reply)
Discussion started by: b.saipriyanka
1 Replies

2. UNIX for Beginners Questions & Answers

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (2 Replies)
Discussion started by: b.saipriyanka
2 Replies

3. Shell Programming and Scripting

Zip Multiple files to One .zip file in AIX system

Hi I have a requirement in unix shell where I need to zip multiple files on server to one single .zip file. I dont see zip command in AIX and gzip command not doing completely what I want. One I do .zip file, I should be able to unzip in my local Computer. Here is example what I want... (9 Replies)
Discussion started by: RAMA PULI
9 Replies

4. Shell Programming and Scripting

List all files with prepended CRC32 (or other) hash code?

I would like to list all files in a directory tree but with a prepended digest hash code (like CRC32). CRC32 is not a MUST. If suitable another hash code can be used as well. In case of CRC32 the listing should look like 3765AC \usr\bin\spool 23CE99 \usr\bin\spool\list.h ... 11AA04... (3 Replies)
Discussion started by: pstein
3 Replies

5. UNIX for Dummies Questions & Answers

cksum does not give me crc32

Is cksum the right command to calculate the crc32 checksum value? I tried it for a number of files now and every time the results dont match. So there is nothing wrong with the file. Also, cksum gives me an all numerical value while crc32 is alpha numeric. What am I doing wrong? Thanks (9 Replies)
Discussion started by: utamav
9 Replies

6. AIX

ZIP multiple files and also specify size of zip file

I have to zip many pdf files and the size of zip file must not exceed 200 MB. When size is more than 200 MB then multiple zip files needs to be created. How we can achieve this in UNIX? I have tried ZIP utility but it takes a lot of time when we add individual pdfs by looping through a... (1 Reply)
Discussion started by: tom007
1 Replies

7. UNIX for Dummies Questions & Answers

unzip .zip file and list the files included in the .zip archive

Hello, I am trying to return the name of the resulting file from a .zip archive file using unix unzip command. unzip c07212007.cef7081.zip Archive: c07212007.cef7081.zip SecureZIP for z/OS by PKWARE inflating: CEP/CEM7080/PPVBILL/PASS/G0063V00 I used the following command to unzip in... (5 Replies)
Discussion started by: oracledev
5 Replies

8. Shell Programming and Scripting

using cksum

hi, I am trying to use the cksum feature in unix. when i make a call to it i get returned something along the lines of: 4603435 14 file3 how do i get the first part of this response only; i.e: 4603435 I'm trying to use at a way without the use of sed and creating temp... (4 Replies)
Discussion started by: leeRoberts2007
4 Replies

9. Programming

crc32 info

hello again, does anyone know where i can find some detailed info about the cyclic redundancy check? thx (2 Replies)
Discussion started by: crashnburn
2 Replies
Login or Register to Ask a Question