UNIX for Dummies Questions & Answers

@RUDIC & cjcox:

Please find the input(FILEA & FILEB) and output format(LDFILE):
Note: The dummy values shown below ranges from length 1500 - 1600 characters approx in a line
FILEA:

Code:

value~0000~refno001~value1~value2~value3........~value65

40000 lines like above with reference no unique
FILEB:

Code:

value~0000~refno001~value1~updatedvalue2~updatedvalue3........~value65

40000 lines like abovewith reference no unique
OUTPUT format in file INFILE_TMP & LDFILE should be like below.

INFILE_TMP:

Code:

value~0000~refno001~value1~updatedvalue2~updatedvalue3........~value65

LDFILE:

Code:

0000~refno001~value1~updatedvalue2~updatedvalue3........~value65

Using reference no from FILEA I will grep the ref no in FILEB in returned line update some more values and then write them into the INFILE_TMP & LDFILE.

@RUDIC I have few questions in your suggestion

do you think we can read the file FILEB and load all of the single line shown above can be accommodated in single array element?
if we accommodate then in single array element it can be searched and retrieved using only refno?

@cjcox:

The lines in both FILEA & FILEB will be having tilde separated values of about 60 to 65 fields approx.
They can be empty fields, date filled values, numbers, character fields.
But only first three fields will be field with value(mandatory) in both files.
The 3rd field(refno) in both files are unique. so i am using ref no as key for grep.

Thanks.

Not a solution, but something to think about. Turn the problem on its head. Avoid loops. Use FILEA to create a one time pass file (filter) against FILEB. For example:

Code:

sed 's;^[^~][^~]*~\([^~][^~]*\)~.*;s/^\\([^~][^~]*\\)~\1~\\([^~][^~]*\\)~\\([^~][^~]*\\)/\\1~\1~\\2~good/;' FILEA >filter.sed
sed -f filter.sed FILEB

Just an idea that might inspire you...

Hi cjcox Could you please explain what this sed command will do?
I could not understand from the series of special characters

Sure... the sed outputs a sed script...

So find all "refs" in FILEA and turn them into sed search and replace statements and store in the filter.sed file and then run that file against FILEB.

There are no "special characters" really in that sed... you should be able to directly cut and paste it.

From this very thin data basis, I can't suggest any solution or even algorithm. It seems you could use FILEB immediatedly as INFILE_TMP, and LDFILE would be derived from FILEB by removing field 1, and that's it.
If this is not what you require, post way mire details.