parse csv file, sha1 hash and output

FreddyG · #1 (**permalink**) 01-28-2009

I have a file, not really a csv, but containing delineated data just the same. Lets call that file "raw_data.txt". It contains data in the format of company name:fein number like this:

Code:

first company name:123456789
second company name:987654321

what i need to do is read this file, apply sha1 to the fein field and then output that into a new file, we'll call "hashed_data.txt". So hashed_data.txt should contain data like

Code:

first company name:f7c3bc1d808e04732adf679965ccc34ca7ae3441
second company name:bfe54caa6d483cc3887dce9d1b8eb91408f1ea7a

i have tried looping thru the file with awk like so

Code:

read FILE
while read line
do
        awk 'BEGIN{FS=OFS=":"}{print $1, $2}'
done < $FILE

but I'm stuck on how to run the second field thru sha1sum

danmero · #2 (**permalink**) 01-28-2009

bash solution.

Code:

IFS=":" && \
while read name fein
        do echo "$name:"$(sha1 -qs $fein)
done < file > newfile

Note: You should read the sha1sum man

FreddyG · #3 (**permalink**) 01-28-2009

thanks! checking it out now

rwuerth · #4 (**permalink**) 01-28-2009

if your echo supports '-n' have you tried to put that into your echo statement?

Code:

echo -n "$name:"$(sha1sum <<< $fein)

I'm not sure if it will or wont work, I just don't know if it's been tried.

FreddyG · #5 (**permalink**) 01-28-2009

i think the command I'm actually going for is sha1sum. When i modified the first snipit you posted to this it seemed to work

Code:

IFS=":" && \
while read name fein
        do echo "$name:"$(sha1sum <<< $fein)
done < raw_data.txt > hashed_data.txt

however, it's getting a newline feed in the hash as well

the accurate hash of
123456789
should be
f7c3bc1d808e04732adf679965ccc34ca7ae3441

Code:

echo -n 123456789 | sha1sum

will produce this hash

so need a way to trim the newline feed out of there.

Code:

tr -d '\n'

but where would it go ion the above snipit?

FreddyG · #6 (**permalink**) 01-28-2009

Quote:

Originally Posted by rwuerth

if your echo supports '-n' have you tried to put that into your echo statement?

Code:

echo -n "$name:"$(sha1sum <<< $fein)

I'm not sure if it will or wont work, I just don't know if it's been tried.

seems to just remove the new line feed between the lines, IE ouput is now

Code:

first company name:179c94cf45c6e383baf52621687305204cef16f9  -second company name:a1b42d633e975efc2f665bda21f94e419c1b6074  -

rwuerth · #7 (**permalink**) 01-28-2009

Okay take the -n out of the echo.

try this before the echo statement

Code:

fein=`echo $fein | tr -d /\n/`

Or as I think about it, just

Code:

fein=`echo -n $fein`

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Parse Logfile output variable	Ikon	Shell Programming and Scripting	11	12-12-2008 10:37 AM
To parse through the file and print output using awk or sed script	cdfd123	Shell Programming and Scripting	4	03-03-2008 12:07 PM
How to parse through a file and based on condition form another output file	sivasu.india	UNIX for Advanced & Expert Users	6	02-28-2008 04:59 AM
parse through one text file and output many	sophiadun	UNIX for Dummies Questions & Answers	14	02-20-2008 06:08 AM
Parse ping output	palm101	Shell Programming and Scripting	3	11-11-2007 12:16 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: