Shell Programming and Scripting

Please help,

file1.txt

Code:

1
2
3
4
5

file2.txt

Code:

3
4
5
6
7

All I need is this result.txt

Code:

1
2

This code below is working with only with few records, but the problem is when I run with a large file like the the file is 2million records the result is theysame with file1

Code:

1
2
3
4
5

Code:

awk 'NR==FNR{a[$0];next}!($0 in a)' file1.txt file2.txt | xargs echo > result.txt

Question:
Why is it happen that it works in few records but not in more than 2million records?
Is it the command set it to timeout?
What could be the solution on this?

Thank you so much.

If this is what you want, then you have to compare file2.txt with file1.txt

Also if you have only one field in each record, better use $1 instead of $0 to avoid blank spaces if any:

Code:

awk 'NR==FNR{a[$1];next}!($1 in a)' file2.txt file1.txt

Also note that awk does has a memory limitation, so it might throw an error when it exceeds the limit.

simply use comm (be sure that both files are sorted, otherwise comm produces wrong output):

Code:

comm -23 file1.txt file2.txt

Hi thanks for the reply but still theysame i try this command:

Code:

comm -2 -3 <(sort file1.txt) <(sort file2.txt) | xargs > result.txt

How to set limit timeout of this command? Or is their any way to solve my problem.

Thanks so much,

did you try this?

Code:

sort file1.txt > file1
sort file2.txt > file2
comm -23 file1 file2 > result

Hi doganaym,

Yes, but still theysame.

Thanks,

---------- Post updated at 06:48 PM ---------- Previous update was at 02:43 PM ----------

Anyone can help me on this?

Thank you so much,
Richie

Did this help ?

Code:

sort file1 file2 | uniq -u