Shell Programming and Scripting

Hi,

I have one situation. I have some 6-7 no. of files in one directory & I have to extract all the lines which exist in all these files. means I need to extract all common lines from all these files & put them in a separate file.

Please help. I know it could be done with the help of cut,sort & uniq commands. But it will take more time whenever the script is executed. I want some quick & shortcut method.

I am using ksh shell.

Code:

cat * | sort > /tmp/alllines
uniq -d /tmp/alllines > /tmp/repeatedlines

-mschwage

Sorry Sir,
My requirement is I want the lines which are present in all these 'n' files. Means every line which will come in output must be present in each of these 'n' files.

Please guide.

>comm_data.txt
for fname in 1.txt 2.txt 3.txt 4.txt
do
if [ ! -s comm_data.txt ]
then
sort $fname > comm_data.txt
else
sort $fname > tmp2
comm -12 comm_data.txt tmp2 > tmp3
mv tmp3 comm_data.txt
fi
done
rm -f tmp2

If the contents of ur files are in order you can use following

>comm_data.txt
for fname in 1.txt 2.txt 3.txt 4.txt
do
if [ ! -s comm_data.txt ]
then
cp $fname comm_data.txt
else
comm -12 comm_data.txt $fname > tmp3
mv tmp3 comm_data.txt
fi
done
cat comm_data.txt

If you have three files f1 f2 f3:

Code:

cat f1 f2 f3 | awk '{a[$0]++} END{for (i in a) if (a[i]==3) print i}'

Thanks ripat, it's perfectly working fine. But ,I have a little concern.
Actually, I won't be sure how many no. of files will get generated everytime the script runs. So, I will be storing the no. of files in a variable & in the code -

cat f1 f2 f3 | awk '{a[$0]++} END{for (i in a) if (a[i]==3) print i}'

I think I have to use the value in that variable in place of 3 here. I tried to replace 3 with variable here. But seems to be not working. I even attempted 'awk -v' option. But, in vain.

Can u please quickly help me out in this. The requirement is quite urgent.

Thanks in advance for ur help.

Code:

ls -1 | wc -l | read filecnt
awk -v filecnt=$filecnt '{a[$0]++} END{for (i in a) if (a[i]==filecnt) print i}'