Shell Programming and Scripting

Hi All,

I get a flat file with its last field data splitting onto a new line.I got this program from Vgersh which when run would cancatenate the split data back to the end of the previous records.But this program fails when it encounters a page break between the split data and the previous record.So if these page breaks are removed,then the program works fine.

Program

Code:

#!/usr/bin/ksh

BEGIN {
  FS=OFS="|"

  FLD_max=11
  
  stderr="cat 2>&1" 
}
(fld + NF-1) > FLD_max {
       if (fld == FLD_max)
          print rec
       else
          printf("Incomplete record: [%d] :: [%s]\n", FNR, rec) | stderr
       rec=$0; fld=NF;next
}
NF < FLD_max {printf("Bad record: [%d] :: [%s]\n", FNR, $0) | stderr; rec=(rec != "") ? rec $0 : $0; fld+=(NF-1);next }
{rec=$0; fld=NF}
END {
  if (rec != "" && split(rec, a, FS) >= FLD_max ) print rec
}

Input...

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?
/*there is a page break here(a kind of straight line shown in Ultra Edit,but not showing here.This needs to be removed*/
ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd
000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎקóR¢¢ÒRS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`=Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®
???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`
á"Ô:`ÓÏI¤?9V?

Output:

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎקóR¢¢ÒRS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`=Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`á"Ô:`ÓÏI¤?9V?

Thanks
Kumar

_If I understand correctly the requirement_ with GNU Awk (on Linux, for example) you could try something like this (if all the records start with 0):

Code:

awk '$1=$1' RS="\n0"  inputfile

The records doesnt start with 0.In order to mask the actual data,i just put some dummy values while maintaining the structure of the records.The record start with two numeric formats...like 100**** and 99****

Regards,
Kumar

So, what about (with GNU Awk):

Code:

awk '$1=$1{print $0 RT}' ORS= RS="\n(100|99)" inputfile

Code:

#!/usr/bin/nawk -f

BEGIN {
  FS=OFS="|"

  FLD_max=11

  FF=sprintf("\f")
  
  stderr="cat 2>&1" 
}
$0 ~ FF { gsub(FF, ""); $1=$1 }

(fld + NF-1) > FLD_max {
       if (fld == FLD_max)
          print rec
       else
          printf("Incomplete record: [%d] :: [%s]\n", FNR, rec) | stderr
       rec=$0; fld=NF;next
}
NF < FLD_max {printf("Bad record: [%d] :: [%s]\n", FNR, $0) | stderr; rec=(rec != "") ? rec $0 : $0; fld+=(NF-1);next }
{rec=$0; fld=NF}
END {
  if (rec != "" && split(rec, a, FS) >= FLD_max ) print rec
}

vgersh99

You are an absolute genius,i feel.It works really great.Thank you so much.

Regards,
Kumar