10-02-2009
Removing Header & Trailer from a file
Hi All,
I am karthik. I am new to this forum. I have one requirement.
I have a file with header and footer.
Header may be like
HDR0001
or
FILE20090110
(Assume it is unknown so far, but i am sure there is a header in the file)
likewise file has the trailer too.
I just want to remove the header & footer and BCP data into a table.
Note: The file size would be 1.5 GB to 2.0 GB.
So i am looking for some good way to remove the header and trailer.
Inputs are welcome!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
I get some flat files with trailer which gives the totol records count and i want to remove the trailer from the file.
i used the following command it works fine with a single file.
cat file_name | grep -v 'Total records:' > file1
mv file file_name
But i dont know how to remove the... (12 Replies)
Discussion started by: kumarsaravana_s
12 Replies
2. UNIX for Dummies Questions & Answers
All,
I am new to unix and i have the following requirement.
I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file..
Could... (35 Replies)
Discussion started by: ksrams
35 Replies
3. Shell Programming and Scripting
Hi all,
I have a requirement to create a Header &Trailer for a file which is having 20 millions of records.
If I use the following method, i think it will take more time.
cat "Header"> file1.txt
cat Data_File>>file1.txt
cat "Trailer">>file1.txt
since second CAT command has to read all... (4 Replies)
Discussion started by: Raamc
4 Replies
4. Shell Programming and Scripting
How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited
I need append 3 files as above: File1:
P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01
File 2:... (2 Replies)
Discussion started by: e1994264
2 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I want to add the below Header to all the files in sequence File1,File2,File3...etc
"ABC,<number of chracter in the file>"
e,g - If File1 is as below
pqrstuvdt
abcdefgh
then I want to add the above header into it ,So that File1 becomes as below
ABC,17
pqrstuvdt
abcdefgh
... (9 Replies)
Discussion started by: spari2
9 Replies
6. Shell Programming and Scripting
Hi All,
I am trying write a simple command using AWK and SED to this but without any success.
Here is what I am using:
head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt
Input:
Header
1234567
abcdefgh... (2 Replies)
Discussion started by: nvuradi
2 Replies
7. Shell Programming and Scripting
Hi,
I need a script that validates a file header/detail/trailer. File layout is:
Header - Rec_Type|File_name|File_Date
Detail - Rec_Type|field1|field2|field3...
Trailder - Rec_Type|File_name|File_Date|Record_count
Sample Data:
HDR|customer_data.dat|20120709... (7 Replies)
Discussion started by: ash_sh
7 Replies
8. Shell Programming and Scripting
please see my requirement, I hope I am clear. (9 Replies)
Discussion started by: mirwasim
9 Replies
9. Shell Programming and Scripting
I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file
Also is it possible to output the TOM and PAT records in the same output file ?
I need the output file names same as xyz_pattern_Datetimestamp.txt
... (23 Replies)
Discussion started by: techedipro
23 Replies
10. UNIX for Advanced & Expert Users
I have a EBCDIC multi layout file which has a header record which is 21 bytes, The Detail records are 2427 bytes long and the trailer record is 9 bytes long.
Is there a command to remove the header as well as trailer record and read only the detail records while at the same time not altering... (1 Reply)
Discussion started by: abhilashnair
1 Replies
LEARN ABOUT DEBIAN
astro::fits::header::cfitsio
Astro::FITS::Header::CFITSIO(3pm) User Contributed Perl Documentation Astro::FITS::Header::CFITSIO(3pm)
NAME
Astro::FITS::Header::CFITSIO - Manipulates FITS headers from a FITS file
SYNOPSIS
use Astro::FITS::Header::CFITSIO;
$header = new Astro::FITS::Header::CFITSIO( Cards => @array );
$header = new Astro::FITS::Header::CFITSIO( File => $file );
$header = new Astro::FITS::Header::CFITSIO( fitsID => $ifits );
$header->writehdr( File => $file );
$header->writehdr( fitsID => $ifits );
DESCRIPTION
This module makes use of the CFITSIO module to read and write directly to a FITS HDU.
It stores information about a FITS header block in an object. Takes an hash as an argument, with either an array reference pointing to an
array of FITS header cards, or a filename, or (alternatively) and FITS identifier.
REVISION
$Id$
METHODS
configure
Reads a FITS header from a FITS HDU
$header->configure( Cards => @cards );
$header->configure( fitsID => $ifits );
$header->configure( File => $file );
$header->configure( File => $file, ReadOnly => $bool );
Accepts an FITS identifier or a filename. If both fitsID and File keys exist, fitsID key takes priority.
If "File" is specified, the file is normally opened in ReadWrite mode. The "ReadOnly" argument takes a boolean value which determines
whether the file is opened ReadOnly.
writehdr
Write a FITS header to a FITS file
$header->writehdr( File => $file );
$header->writehdr( fitsID => $ifits );
Its accepts a FITS identifier or a filename. If both fitsID and File keys exist, fitsID key takes priority.
Returns undef on error, true if the header was written successfully.
NOTES
This module requires Pete Ratzlaff's Astro::FITS::CFITSIO module, and William Pence's "cfitsio" subroutine library (v2.1 or greater).
SEE ALSO
Astro::FITS::Header, Astro::FITS::Header::Item, Astro::FITS::Header::NDF, Astro::FITS::CFITSIO
AUTHORS
Alasdair Allan <aa@astro.ex.ac.uk>, Jim Lewis <jrl@ast.cam.ac.uk>, Diab Jerius.
COPYRIGHT
Copyright (C) 2007-2009 Science & Technology Facilities Council. Copyright (C) 2001-2006 Particle Physics and Astronomy Research Council.
All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation,
Inc., 59 Temple Place,Suite 330, Boston, MA 02111-1307, USA
perl v5.14.2 2012-06-30 Astro::FITS::Header::CFITSIO(3pm)