Shell Programming and Scripting

The latter script does not work with all awk versions; some convert the search key to a number that overflows, even after a ~ operator!
Fix: cast to a string ($0 ~ t"").
But at this occasion I would add a full field match ("|"$0"|" ~ "[|]"t"[|]").
Also the given file1 has trailing spaces, therefore it is advisable to use
awk 'script' file1 FS="|" file2 rather than awk -F\| 'script' file1 file2, so file1 works with the default FS where $1 strips leading and trailing spaces.
Here is another all-in-awk solution that uses an array. Like the latter script it deletes field #61 - not the 61th delimiter.

Code:

#!/bin/bash
PATH=/usr/xpg4/bin:/bin:/usr/bin
awk '
function prtARR() {
  out=ARR[1]
  for (a=2; a<=nARR; a++) out=(out FS ARR[a])
  print out
}
function rmARR(num) {
  for (a=num; a<nARR; a++) ARR[a]=ARR[a+1]
  nARR--
}
NR==FNR {
  K[$1]; next
}
{
  nARR=split($0,ARR)
  if (nARR>65) rmARR(61)
  prtARR()
}
' file1 FS="|" file2

Until now this code works for me :

Code:

#!/bin/bash
PATH=/usr/xpg4/bin:/bin:/usr/bin

while read line
do

grep "$line" /tmp/BadTransactions/test_data_for_validation_script.txt

awk 'NR==FNR { K[$1]; next } ($2 in K)' /tmp/BadTransactions/TRANSACTIONS_DAILY_20180730.txt FS="|" /opt/NorkomC
onfigS2/inbox/TRANSACTIONS_DAILY_20180730.txt > /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_20180730.txt

sed '/\([^|]*[|]\)\{65\}/ s/|//61' /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_20180730.txt

done < /tmp/BadTransactions/TRANSACTIONS_DAILY_20180730.txt > /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_201807
30.txt

So until now if there are more than 64th pipes in each line , it delete the 61th pipe.

Now , i want to delete the 61th pipe in each line if the line has more than 64 pipes until the line reaches the 64 pipes in whole line

What i mean :

If a line has for example 67 pipes , it will delete the 61th pipe , then it will go again to the same line and now it will check that it has more than 64 pipes(which actually has 66 now ) and i t will delete the 61th pipe.

This will be continued until the pipes are more than 64.

Could you please suggest me any idea how to loop that ?

Thank you

------ Post updated at 07:27 PM ------

Until now this code works for me :

Code:

Code:

#!/bin/bash
PATH=/usr/xpg4/bin:/bin:/usr/bin

while read line
do

grep "$line" /tmp/BadTransactions/test_data_for_validation_script.txt

awk 'NR==FNR { K[$1]; next } ($2 in K)' /tmp/BadTransactions/TRANSACTIONS_DAILY_20180730.txt FS="|" /opt/NorkomC
onfigS2/inbox/TRANSACTIONS_DAILY_20180730.txt > /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_20180730.txt

sed '/\([^|]*[|]\)\{65\}/ s/|//61' /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_20180730.txt

done < /tmp/BadTransactions/TRANSACTIONS_DAILY_20180730.txt > /tmp/BadTransactions/TRANSACTIONS_DAILY_NEW_201807
30.txt

So until now if there are more than 64th pipes in each line , it delete the 61th pipe.

Now , i want to delete the 61th pipe in each line if the line has more than 64 pipes until the line reaches the 64 pipes in whole line

What i mean :

If a line has for example 67 pipes , it will delete the 61th pipe , then it will go again to the same line and now it will check that it has more than 64 pipes(which actually has 66 now ) and i t will delete the 61th pipe.

This will be continued until the pipes are more than 64.

Could you please suggest me any idea how to loop that ?

Thank you

As has been noted before, the sample files you provided in post #1 in this thread do not test any of your requirements. All of the lines in your second sample file have a field #2 value that matches a value found in your first sample file. And, all lines in your second sample file have exactly 64 pipe symbols (so there is never any need to remove any pipe symbols) to achieve your goal. Using your sample input files, your second sample input file is identical to the output you say you want.

You say that the code you have shown us in post #9 in this thread works until now. That means that something has changed recently and that it no longer does what you want it to do. What has changed? In what way does it fail to produce the output you want?

I note that the awk in your inner loop redirects its standard output to the same file to which the outer loop redirects its standard output. That would usually have the effect of throwing away everything written to that file except for the output produced by the last invocation of awk and the last invocation of sed.

Please give us two small sample input files that actually test the features you want your code to provide and also give us a sample output file that is the exact output you want from those sample input files.

I think I have a fairly simple awk script that does what you want, but with no way to test it, I'm not sure that I have understood your requirements. Also, it assumes that the IDs found in your first file can be found in the second field of your second input file (as shown in your sample input files in post #1 in this thread). Is this a valid assumption, or does the code you want need to look for those IDs in every field in your second input file?

If you want to stick with your sed script, you can augment it with a loop:

Code:

sed -e ':Loop' -e '/\([^|]*[|]\)\{65\}/ s/|//61; tLoop'

The t branches to Loop if there was a successful substitution.
Alternatively you can do an unconditional branch if you put it in a { } block. The / / provides the condition for the whole block.

Code:

sed -e ':Loop' -e '/\([^|]*[|]\)\{65\}/{' -e 's/|//61; bLoop' -e '}'