The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
how to remove the first line from a flat file ? xli UNIX for Dummies Questions & Answers 21 12-16-2008 01:37 AM
remove specific lines from flat file using perl meghana Shell Programming and Scripting 12 02-12-2008 09:50 PM
Help on page breaks simhasuri Shell Programming and Scripting 1 01-31-2008 04:51 PM
Page Breaks rama71 High Level Programming 9 06-29-2005 01:28 PM
Insert page breaks into .csv file welsht Shell Programming and Scripting 2 02-04-2005 02:39 PM

 
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
Prev Previous Post   Next Post Next
  #1 (permalink)  
Old 04-16-2007
kumarsaravana_s kumarsaravana_s is offline
Registered User
  
 

Join Date: Feb 2007
Location: Bangalore
Posts: 105
How to remove page breaks from a flat file???

Hi All,

I get a flat file with its last field data splitting onto a new line.I got this program from Vgersh which when run would cancatenate the split data back to the end of the previous records.But this program fails when it encounters a page break between the split data and the previous record.So if these page breaks are removed,then the program works fine.

Program


Code:
#!/usr/bin/ksh

BEGIN {
  FS=OFS="|"

  FLD_max=11
  
  stderr="cat 2>&1" 
}
(fld + NF-1) > FLD_max {
       if (fld == FLD_max)
          print rec
       else
          printf("Incomplete record: [%d] :: [%s]\n", FNR, rec) | stderr
       rec=$0; fld=NF;next
}
NF < FLD_max {printf("Bad record: [%d] :: [%s]\n", FNR, $0) | stderr; rec=(rec != "") ? rec $0 : $0; fld+=(NF-1);next }
{rec=$0; fld=NF}
END {
  if (rec != "" && split(rec, a, FS) >= FLD_max ) print rec
}

Input...

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?
/*there is a page break here(a kind of straight line shown in Ultra Edit,but not showing here.This needs to be removed*/
ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd
000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎ×§óR¢¢Ò RS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`= Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®
???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`
á"Ô:`ÓÏI¤?9V?

Output:

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎ×§óR¢¢ÒRS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`=Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`á"Ô:`ÓÏI¤?9V?

Thanks
Kumar

Last edited by vino; 04-16-2007 at 09:42 AM.. Reason: Please put your code within code tags.
 

Bookmarks

Tags
linux

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 09:19 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0