Sponsored Content
Top Forums Shell Programming and Scripting How to remove page breaks from a flat file??? Post 302114609 by kumarsaravana_s on Tuesday 17th of April 2007 08:51:31 AM
Old 04-17-2007
Quote:
Originally Posted by vgersh99
Code:
#!/usr/bin/nawk -f

BEGIN {
  FS=OFS="|"

  FLD_max=11

  FF=sprintf("\f")
  
  stderr="cat 2>&1" 
}
$0 ~ FF { gsub(FF, ""); $1=$1 }

(fld + NF-1) > FLD_max {
       if (fld == FLD_max)
          print rec
       else
          printf("Incomplete record: [%d] :: [%s]\n", FNR, rec) | stderr
       rec=$0; fld=NF;next
}
NF < FLD_max {printf("Bad record: [%d] :: [%s]\n", FNR, $0) | stderr; rec=(rec != "") ? rec $0 : $0; fld+=(NF-1);next }
{rec=$0; fld=NF}
END {
  if (rec != "" && split(rec, a, FS) >= FLD_max ) print rec
}

vgersh99

You are an absolute genius,i feel.It works really great.Thank you so much.

Regards,
Kumar
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Insert page breaks into .csv file

I have large .csv files that I need to get page breaks into. I am taking comma delimited files of over a million records and putting them into a pdf file. Is there a way, using sed or otherwise, to insert some type of page break character into my file? (2 Replies)
Discussion started by: welsht
2 Replies

2. Programming

Page Breaks

Hi, I have a program in Pro*c when I run it I have no problem with the output but when it runs via the at command and except the output has page breaks every 66 lines. I don't want those page breaks to be in the output. any idea? (9 Replies)
Discussion started by: rama71
9 Replies

3. Shell Programming and Scripting

Help on page breaks

Hi, I am new to Unix (AIX). I have a header (in a text file) that needs to be wrtitten on all the pages of a result file (text file). After the header is written, data needs to be read from a file A(text file) and inserted to the result file. If the number of lines reaches 80 in a page, page... (1 Reply)
Discussion started by: simhasuri
1 Replies

4. UNIX for Dummies Questions & Answers

how to remove the first line from a flat file ?

Hi, I want to remove the first line from a flat file using unix command as simple as possible. Can anybody give me a hand ? Thanks in advance. xli (21 Replies)
Discussion started by: xli
21 Replies

5. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

6. Shell Programming and Scripting

Remove line breaks in csv file using shell script

Hi All, I've a csv file in which the record is getting break into 1 line or more than one line. I want to combine those splits into one line and remove the unwanted character existing in the record i.e. double quote symbol ("). The line gets break only when the record contains double... (4 Replies)
Discussion started by: rajak.net
4 Replies

7. Shell Programming and Scripting

script for adding page number before page breaks

Hi, If there is an expert that can help: I have many txt files that are produced from pdftotext that include page breaks the page breaks seem to be unix style hex 0C. I want to add page numbers before each page break as in : Page XXXX Regards antman (9 Replies)
Discussion started by: antman
9 Replies

8. UNIX for Advanced & Expert Users

Remove duplicates in flat file

Hi all, I have a issues while loading a flat file to the DB. It is taking much time. When analyzed i found out that there are duplicates entry in the flat file. There are 2 type of Duplicate entry. 1) is entire row is duplicate. ( i can use sort | uniq) to remove the duplicated entry. 2) the... (4 Replies)
Discussion started by: samjoshuab
4 Replies

9. UNIX for Dummies Questions & Answers

Page breaks and line breaks

Hi All, Need an urgent solution to an issue . We have created a ksh file or shell script which generates 1 DAT file. the DAT file contains extract of a select statement . Now the issue is , when we are executing the ksh file , the output is coimng with page breaks and line breaks . We have... (4 Replies)
Discussion started by: Ayaskant
4 Replies

10. Shell Programming and Scripting

Remove first NULL Character in Flat File

We have a flat file with below data : ^@^@^@^@00000305^@^@^@^@^@^@430^@430^@^@^@^@^@^@^@^@^@09079989530As we can see ^@ is Null character in this file I want to remove only the first few null characters before string 00000305 How can we do that, any idea. I want a new file without first few... (5 Replies)
Discussion started by: simpltyansh
5 Replies
FULLA(1)							       HUGIN								  FULLA(1)

NAME
fulla - Correct lens distortion, vignetting and chromatic aberration SYNOPSIS
fulla [options] inputfile(s) DESCRIPTION
Apply radial or flat-field vignetting correction as well as geometrical radial distortion and transversal chromatic aberration correction. fulla can be used to batch correct a large number of files. Radial distortion coefficients can be read from a PTLens database (only database version 4.2 is supported). Vignetting correction is done the same way as described in the nona script file documentation. Vignetting correction can be done based on a flat-field or a radial scaling. Vignetting correction by flat-field division: I = I / c, c = flat-field / mean(flat-field) Radial vignetting correction: I = I / c, c = a + b*r^2 + c*r^4 + d*r^6 The radial distortion correction is identical to the one implemented by PanoTools: r_new = a*r^4 + b*r^3 + c*r^2 + d*r [1] Since transversal chromatic aberrations (TCA) leads to differently scaled objects in the red, green and blue channel, it can be corrected by scaling the red and blue channel so that they match the green channel. Most applications, like Photoshop or Picture Window Pro offer TCA correction by linear scaling. This is equal to varying the d parameter in equation [1]. See http://hugin.sf.net/tutorials/tca for a method to determine these parameters. OPTIONS
-g a:b:c:d Radial distortion coefficient for all channels, (a, b, c, d) -b a:b:c:d Radial distortion coefficients for blue channel, (a, b, c, d). This is applied on top of the -g distortion coefficients, use for TCA corr -r a:b:c:d Radial distortion coefficients for red channel, (a, b, c, d). This is applied on top of the -g distortion coefficients, use for TCA corr -p Try to read radial distortion coefficients (usually supplied using -g) from the PTLens database. The environment variable PTLENS_PROFILE should point to the profile.txt file. Since newer version of the PTLens database are incompatible with fulla, download the PTLens database from the hugin sourceforge website instead: http://www.sf.net/projects/hugin -m Canon Camera manufacturer, for PTLens database query. EXIF data inside JPEG files is read, if this option is not given -n Camera Camera name, for PTLens database query. EXIF data inside JPEG files is read, if this option is not given. -l Lens Lens name, for PTLens database query if not specified, a list of possible lenses is displayed -d 50 Specify focal length in mm, for PTLens database query. EXIF data inside JPEG files is read, if this option is not given. -s Do not rescale the image to avoid black borders. -f filename Vignetting correction by flat-field division I = I / c, c = flat-field / mean(flat-field) -c a:b:c:d Radial vignetting correction by division: I = I / c, c = a + b*r^2 + c*r^4 + d*r^6 -i value Gamma of input data. used for gamma correction before and after flat-field correction -t n Number of threads that should be used during processing -h Display help summary. -o name Set output filename. If more than one image is given, the name will be uses as suffix (default suffix: _corr) -e value Compression of the output files For JPEG output: 0-100 For tiff output: DEFLATE, LZW -v Verbose AUTHORS
Written by Pablo d'Angelo. Also contains contributions from Douglas Wilkins, Ippei Ukai, Ed Halley, Bruno Postle, Gerry Patterson and Brent Townshend. This man page was written by Cyril Brulebois <cyril.brulebois@enst-bretagne.fr> and is licensed under the same terms as the hugin package itself. "Version: 2011.4.0" 2011-12-02 FULLA(1)
All times are GMT -4. The time now is 04:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy