The file1 is a huge file. I am trying to execute following command -
When I do that, I get the error after a minute- grep: Memory exhausted.
Strangely, when I do grep -f File2 File1, then it doesn't give that error but output it generates is wrong as it prints some output that is not in File2!
Please help with what is the better way to compare and print output?
I can't use diff. I need to find those entries in File2 that are present in File1. They may not be in the same order. Please check expected output I have provided and input files.
Thanks for your help.
---------- Post updated at 01:12 PM ---------- Previous update was at 01:11 PM ----------
Corona,
That is also true. Would you know any command that splits a file into multiple chunks?
Yes, 'split < File1' would produce files xaa, xab, ... in chunks of 1000 lines. The parameter '-l 10000' would make it split at 10000 lines. This could produce hundreds or thousands of files if File1 is enormous enough, be warned.
I have two files
I need to compare these two files and take the lines that are common in both the files and consider the line present in second file for my further processing
I have used "Awk" along with "FNR and NR" but that is not working
gawk -F= '
> FNR==NR {a=$1; next};
> ... (2 Replies)
I have 2 files with exactly the same information (with header and separated by ";") and what I would like to do is print (for both files!) the columns that are different and also print the "key" column that is equal in the 2 files For example, if
File1:
key1;aaa;bbb;ccc
key2;ddd;eee;fff... (4 Replies)
Inspired by the extremely short awk code from Ygor on this post I wanted to compare two files on only one field. I can't get it to work. Can anybody help on explaining the code and fix the code?
My code which does not work:
awk 'BEGIN{a=1};a!=1' file1.txt file2.txt >outfile.txt
file1.txt... (1 Reply)
While running script I am getting an error like
Few lines in data are not being processed.
After googling it I came to know that adding such line would give some memory to it
ini_set("memory_limit","64M");
my input file size is 1 GB.
Is that memory limit is based on RAM we have on... (1 Reply)
Any clues on how to get rid of this PHP error?
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /website/www/includes/functions_manpages.php on line 58
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71... (4 Replies)
Hi everyone
I have a dilemma and I'm hoping someone has an answer for me.
I have two files:
# cat masterfile
line3
line4
line5
line6
line7
# cat tempfile
line1
line2
line3
line4
I want to compare tempfile with masterfile. (3 Replies)
Hello, I am new to scripting and need some help. In looking at other posts on this forum, I came up with the following logic. I cannot figure out why I am getting names of files of the current directory in my echo output.
Scenario: message file has a line containing the version. Version.txt... (2 Replies)
Dear All,
I have executed a awk script in linux box which consists of 21 Million records.And i have two mapping files of 500 and 5200 records.To my surprise i found an error
awk: cmd. line:19: (FILENAME=/home/FILE FNR=21031272) fatal: Memory exhausted.
Is there any limitation for records... (3 Replies)