I am new to this forum and this is my first post.
My requirement is like to optimize the time taken to grep the file with 40000 lines.
There are two files FILEA(40000 lines) FILEB(40000 lines).
The requirement is like this, both the file will be in the format below
I will take the refno001 as a search key from FILEA and grep the FILEB. The FILEB will be having the updated value of refno001 as follows.
.
So i will use this record from FILEB in my process. The problem here is i am using below code to grep from FILEB.
But this is taking 20 minutes to complete (i.e 40000 x 40000) search for each and every reference no from FILEA to FILEB. This has to be minized as short as possible.
So kindly guide me if there is any way to reduce the process time to less than 5 mins with any other commands.
Note: my UNIX is AIX 6.0
Last edited by Don Cragun; 11-19-2015 at 02:13 AM..
Reason: Add CODE and ICODE tags.
Hi There...
I have an application which starts up many different processes under different names and I'm creating a script to tell me which processes are running (approx 30 different processes). To do this, I parse the results of a ps -u $USER.
My question is, will my script be faster if I run... (2 Replies)
Hi I have to grep for 2000 strings in a file one after the other.Say the file name is Snxx.out which has these strings.
I have to search for all the strings in the file Snxx.out one after the other.
What is the fastest way to do it ??
Note:The current grep process is taking lot of time per... (7 Replies)
111111111100000000001111111111
123232323200000010001114545454
232435424200000000001232131212
342354234301000000002323423443
232435424200000000001232131212
2390898994200000000001238908092
This is the record format.
From 11th position to 20th position in a record there are 0's occuring,and... (6 Replies)
Basically, my problem is to multiply my file to $c times. Is there a faster way to do this?
c=100
while ]; do
cat file1.txt
((c=$c-1))
done > file2.txt
I appreciate your help! (6 Replies)
Hi Experts,
I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is:
while read line
do
emp_name=`echo $line`
grep -e... (7 Replies)
we have 30 GB files on our filesystem which we need to copy daily to 25 location on the same machine (but different filesystem).
cp is taking 20 min to do the copy and we have 5 different thread doing the copy.
so in all its taking around 2 hr and we need to reduce it.
Is there any... (9 Replies)
Hi Gurus,
I have two big files. I need to compare the different. currently, I am using
sort file1 > file1_temp;
sort file2 > file2_tmp
diff file1_tmp file2_tmp
I can use command
grep -v -f file1 file2
just wondering which way is fast to compare two big files.
Thanks... (4 Replies)
I have a very big input file <inputFile1.txt> which has list of mobile no
inputFile1.txt
3434343
3434323
0970978
85233
... around 1 million records
i have another file as inputFile2.txt which has some log detail big file
inputFile2.txt
afjhjdhfkjdhfkd df h8983 3434343 | 3483 | myout1 |... (3 Replies)