As has been noted before, the sample files you provided in post #1 in this thread do not test any of your requirements. All of the lines in your second sample file have a field #2 value that matches a value found in your first sample file. And, all lines in your second sample file have exactly 64 pipe symbols (so there is never any need to remove any pipe symbols) to achieve your goal. Using your sample input files, your second sample input file is identical to the output you say you want.

You say that the code you have shown us in post #9 in this thread works until now. That means that something has changed recently and that it no longer does what you want it to do. What has changed? In what way does it fail to produce the output you want?

I note that the awk in your inner loop redirects its standard output to the same file to which the outer loop redirects its standard output. That would usually have the effect of throwing away everything written to that file except for the output produced by the last invocation of awk and the last invocation of sed.

Please give us two small sample input files that actually test the features you want your code to provide and also give us a sample output file that is the exact output you want from those sample input files.

I think I have a fairly simple awk script that does what you want, but with no way to test it, I'm not sure that I have understood your requirements. Also, it assumes that the IDs found in your first file can be found in the second field of your second input file (as shown in your sample input files in post #1 in this thread). Is this a valid assumption, or does the code you want need to look for those IDs in every field in your second input file?