Shell Programming and Scripting

Hi,

I find out one problem is...the main point is we must delete 2nd duplicated of word in txt file. For example

apple
orange
pink
green
orange
yellow
orange
red

output should be:
apple
orange
pink
green
yellow
orange
red

#For the above, check all words in txt file and delete the 2nd duplicated of word (and MUST not use sort to do, because of the order should not be changed - only delete the duplicated). Due to the file may be over 10000 words, it is very difficult for me to check..by manually.....

any small script can help?

What's wrong with the solution in this thread?

And your definition of duplicated is a bit confusing. To delete duplicate entries means that except the first entry, all other entries of that word/phrase will be deleted.

Another question. Is this any form of school work/home work?

This will remove only the second instance of duplicated lines....

Code:

awk '++a[$0]!=2' file1

count=0
cat {1} | while read line
do
count=`echo "${count} + 1" | bc`
cc=`head -${count} ${1} |grep -c ${line}`
if [ ${cc} -eq 2 ]
then
:
else
echo ${line}
fi
done

Would this work ? I have not tested it but it should work. save it as a file , give it execute permission and then use it as "script filename".

All the best. It's a simple logic but i think that would work for you.

Regds,

Kaps

LOOK WOKING...BUT can I specify any word to delete? such as orange..and it can delete the 2nd duplicated of "orange" only.

chechout this script

#! /usr/bin/bash
cat $1|sort -c
if [ $? -eq 1 ]; then
sort -u $1|tee $1
else
echo "Already sorted"
fi

let me know if you have any questions. $1 is your input filename

thank you for the script..however, I cannot sort the txt file..because of all the word in the text file of order..should not be change. You can see my exmaple at the top of the post. I only need to delete specify (such as orange) word of 2nd duplicated and nothing change with other...