Shell Programming and Scripting

Okay - try something like this:

Code:

#!/bin/ksh 
# file: dedup.sh  (be sure to chmod +x dedup.sh )
# $1 = file name - first parameter
# usage: dedup.sh myfile > newfile
old_rec="-1"
            # sort the file
sort $1 -o $1
            # process the sorted file
while read rec
do

# check first five chars of the record
        # first time thru the loop
    if [ "$old_rec" = "-1" ]; then
        old_rec="$rec"
        old_key=`expr substr "$rec" 1 5`
        continue
    fi
        # check if record keys match
    rec_key=`expr substr "$rec" 1 5`
    if [ "$old_key" = "$rec_key" ]; then
        # they match so keep reading file, use latest record
        old_rec="$rec"
        continue
    fi
# we have a new key, print last record
    echo "$old_rec"
    # set up for new record
    old_rec="$rec"
    old_key=`expr substr "$rec" 1 5`
# read from $1 the filename 
done < $1

exit

This works on a dummy file with 20 records. Print the last of the duplicated keys. key is the first five characters of the record

Thanks Jim.
Hi, it's working!
I was trying to use 'sort -un 'command to pick the last duplicate.
But this command can only pick from the 3 occurence, it will not recoginize from the 4th occurence.


I ran this script and found out that it does not print the last record in the file. So I put a echo "$old_rec" command right outside the while statement. Now it's working.

But I do have a question about changing shell within script.
Can I do that?

My script was started with using '#/bin/sh'. I don't know what shell this is. I was trying to copy your script to mine. but it won't run well with this shell. so I add one line like this:
/usr/bin/ksh
to start your script. after processing, I return back to my shell by typing
/usr/bin/sh
It seems weird because I will be prompted with $ and I have to type exit and get out twice, but the result is good.

>>/usr/bin/ksh
>>to start your script. after processing, I return back to my >>shell by typing
>>/usr/bin/sh


By doing above 2 things u are starting 2 more new shells over your current shell ; U had to exit 2 times to retain the current shell.


To know the shell u are working on ...

grep "userid" /etc/passwd

it gives the shell u are working on unless it is changed by .profile.



If shell gives prblem some times run the script like this .


./script

Hope this helps.