Quote:
Originally Posted by dislusive
If I understand what you're trying to do correctly, here's a quick bash script.
Code:
#!/bin/bash
compareFile = "/path/to/file/to/compare.txt"
outputFile = "/path/to/outputFile.txt"
for filename in /some/dir/of/text/files/*.txt; do
numlines=`cat $filename | wc -l`
for i in `seq 1 $numlines`; do
current=`cat $filename | head -$i | tail -1`
grep -q "${current}" ${compareFile}
if [ $? != 0 ]; then
#doesn't exist, append to $outputFile
echo "${filename}:${current}" >> ${outputFile}
fi
done
done
|
As mentioned by OP, the files are in GB. I think there will be some performance lag. just a guess.
Also seq is not a standard command in some *nix OS. Therefore if you want to use loops that loop over a counter, a while loop can be used instead. eg while [ $num -le $numlines ]
