Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to remove page breaks from a flat file???
Top Forums Shell Programming and Scripting How to remove page breaks from a flat file??? Post 302114464 by kumarsaravana_s on Monday 16th of April 2007 08:37:21 AM
Old 04-16-2007
How to remove page breaks from a flat file???
Hi All,

I get a flat file with its last field data splitting onto a new line.I got this program from Vgersh which when run would cancatenate the split data back to the end of the previous records.But this program fails when it encounters a page break between the split data and the previous record.So if these page breaks are removed,then the program works fine.

Program

Code:
#!/usr/bin/ksh

BEGIN {
  FS=OFS="|"

  FLD_max=11
  
  stderr="cat 2>&1" 
}
(fld + NF-1) > FLD_max {
       if (fld == FLD_max)
          print rec
       else
          printf("Incomplete record: [%d] :: [%s]\n", FNR, rec) | stderr
       rec=$0; fld=NF;next
}
NF < FLD_max {printf("Bad record: [%d] :: [%s]\n", FNR, $0) | stderr; rec=(rec != "") ? rec $0 : $0; fld+=(NF-1);next }
{rec=$0; fld=NF}
END {
  if (rec != "" && split(rec, a, FS) >= FLD_max ) print rec
}

Input...

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?
/*there is a page break here(a kind of straight line shown in Ultra Edit,but not showing here.This needs to be removed*/
ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd
000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎקóR¢¢Ò RS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`= Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®
???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`
á"Ô:`ÓÏI¤?9V?

Output:

000000|Apr 14 2007 7:59:58:376AM| |ASDFASFSDA |000000|0|0|0|3111|SDFSDF|æPP:?µß?ÚÐý?K
000004|Apr 14 2007 7:59:58:790AM| |ASFASFAS|000000|0|0|0|111|DSFSDF|?Í¢º²c?
ÄÜ?Îd000000|Apr 14 2007 7:59:59:970AM| |ASFAFASA |00000|0|0|0|1111|SFDSFSD|?ÒÎקóR¢¢ÒRS?
00000|Apr 14 2007 8:00:01:693AM| |ASFSAFAS |000000|0|0|0|111SDFSDF|Âh>`=Û?èäN?´ÈH
000000|Apr 14 2007 8:00:02:350AM| |ASFAFA|00000|0|0|0111|SDFSD1|?®???ø»à濦«?
000000|Apr 14 2007 8:00:02:700AM| |ASFSAFASSA |00000|0|0|0|9964|SDFSD|3`á"Ô:`ÓÏI¤?9V?

Thanks
Kumar
Last edited by vino; 04-16-2007 at 09:42 AM.. Reason: Please put your code within code tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Insert page breaks into .csv file

I have large .csv files that I need to get page breaks into. I am taking comma delimited files of over a million records and putting them into a pdf file. Is there a way, using sed or otherwise, to insert some type of page break character into my file? (2 Replies)
Discussion started by: welsht
2 Replies

2. Programming

Page Breaks

Hi, I have a program in Pro*c when I run it I have no problem with the output but when it runs via the at command and except the output has page breaks every 66 lines. I don't want those page breaks to be in the output. any idea? (9 Replies)
Discussion started by: rama71
9 Replies

3. Shell Programming and Scripting

Help on page breaks

Hi, I am new to Unix (AIX). I have a header (in a text file) that needs to be wrtitten on all the pages of a result file (text file). After the header is written, data needs to be read from a file A(text file) and inserted to the result file. If the number of lines reaches 80 in a page, page... (1 Reply)
Discussion started by: simhasuri
1 Replies

4. UNIX for Dummies Questions & Answers

how to remove the first line from a flat file ?

Hi, I want to remove the first line from a flat file using unix command as simple as possible. Can anybody give me a hand ? Thanks in advance. xli (21 Replies)
Discussion started by: xli
21 Replies

5. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

6. Shell Programming and Scripting

Remove line breaks in csv file using shell script

Hi All, I've a csv file in which the record is getting break into 1 line or more than one line. I want to combine those splits into one line and remove the unwanted character existing in the record i.e. double quote symbol ("). The line gets break only when the record contains double... (4 Replies)
Discussion started by: rajak.net
4 Replies

7. Shell Programming and Scripting

script for adding page number before page breaks

Hi, If there is an expert that can help: I have many txt files that are produced from pdftotext that include page breaks the page breaks seem to be unix style hex 0C. I want to add page numbers before each page break as in : Page XXXX Regards antman (9 Replies)
Discussion started by: antman
9 Replies

8. UNIX for Advanced & Expert Users

Remove duplicates in flat file

Hi all, I have a issues while loading a flat file to the DB. It is taking much time. When analyzed i found out that there are duplicates entry in the flat file. There are 2 type of Duplicate entry. 1) is entire row is duplicate. ( i can use sort | uniq) to remove the duplicated entry. 2) the... (4 Replies)
Discussion started by: samjoshuab
4 Replies

9. UNIX for Dummies Questions & Answers

Page breaks and line breaks

Hi All, Need an urgent solution to an issue . We have created a ksh file or shell script which generates 1 DAT file. the DAT file contains extract of a select statement . Now the issue is , when we are executing the ksh file , the output is coimng with page breaks and line breaks . We have... (4 Replies)
Discussion started by: Ayaskant
4 Replies

10. Shell Programming and Scripting

Remove first NULL Character in Flat File

We have a flat file with below data : ^@^@^@^@00000305^@^@^@^@^@^@430^@430^@^@^@^@^@^@^@^@^@09079989530As we can see ^@ is Null character in this file I want to remove only the first few null characters before string 00000305 How can we do that, any idea. I want a new file without first few... (5 Replies)
Discussion started by: simpltyansh
5 Replies

LEARN ABOUT DEBIAN

gchem3d

GCHEM3D(1)						       gnome-chemistry-utils							GCHEM3D(1)

NAME

       gchem3d - a small chemical viewer application

SYNOPSIS

       gchem3d [OPTION(S)...] [FILE...]

DESCRIPTION

       gchem3d is a small chemical viewer application, which can show several chemical file formats.

OPTIONS

       The following options are accepted:

       -b COLOR, --bgcolor=COLOR
	   Use the given color as background color.  COLOR can be one of "black" (default), "white", "#rrggbb" (don't forget to escape the "#"
	   character in the shell).

       -d MODEL, --display3d=MODEL
	   Choose how molecules are displayed.	MODEL can be one of "BallnStick" (default), "SpaceFill".

       -?, --help
	   Show application help options.

       --help-all, --help-*
	   Print all or just a group of help options. These options are not documented here. Instead see gtk-options(7) and gnome-options(7).

       -v, --version
	   Print gchem3d version information.

SEE ALSO

       gnome-options(7), gtk-options(7)

AUTHORS

       Jean Brefort <jean.brefort@normalesup.org>
	   Program author.

       Daniel Leidert <daniel.leidert@wgdd.de>
	   Manpage author.

COPYRIGHT

       Copyright (C) 2002-2007 Jean Brefort
       Copyright (C) 2004-2007 Daniel Leidert

       Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 or any
       later version published by the Free Software Foundation.

gcu 0.12				      $Date: 2009-03-19 10:53:47 +0100 (jeu. 19 mars 2009) $					GCHEM3D(1)
All times are GMT -4. The time now is 07:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy