Hello everybody,

I have a sorted text file. some of the lines appear twice or even more. is there an unix utility that removes the extra appearences?

Thanks,
Ido.

Try uniq command. Check man uniq for more help.

Thanks!
Ido.

You are welcome!

Since you are new to these forums, I would recommend you to read the RULES we practice here:

(9) Edit your posts if you see spelling or grammar errors (don't write in cyberchat or cyberpunk style). English only.

Regards,
Tayyab

uniq wont work coz it only removes the repeated line if they come one after the other

if the file is like the following

ab
hjk
jl as
ab

it wont work.

i know its not the ans u want, but i am working on it.

Its okay coz anyway I sort the file in advance...

Thanks,
Ido.

there is some minor defect with this code. i think the logic is ok. someone plz try n debug it. i am doing it in a hurry n hv to go now.

#!/bin/awk -f~
cat bc|while read line
do
x=`wc -l bc|awk '{print $1}'`
for ( k=( NR + 1 ) ; k<=x ; k++ )
{
if /$line/
then
sed 'k d' bc
fi
}
done



error:


awk: syntax error near line 2
awk: bailing out near line 2

You are defining the script as awk, but there is non-awk code in the script. Use #!/bin/sh or #!/bin/ksh or #!/bin/bash (depending on your shell).