10-02-2009
Removing Header & Trailer from a file
Hi All,
I am karthik. I am new to this forum. I have one requirement.
I have a file with header and footer.
Header may be like
HDR0001
or
FILE20090110
(Assume it is unknown so far, but i am sure there is a header in the file)
likewise file has the trailer too.
I just want to remove the header & footer and BCP data into a table.
Note: The file size would be 1.5 GB to 2.0 GB.
So i am looking for some good way to remove the header and trailer.
Inputs are welcome!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
I get some flat files with trailer which gives the totol records count and i want to remove the trailer from the file.
i used the following command it works fine with a single file.
cat file_name | grep -v 'Total records:' > file1
mv file file_name
But i dont know how to remove the... (12 Replies)
Discussion started by: kumarsaravana_s
12 Replies
2. UNIX for Dummies Questions & Answers
All,
I am new to unix and i have the following requirement.
I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file..
Could... (35 Replies)
Discussion started by: ksrams
35 Replies
3. Shell Programming and Scripting
Hi all,
I have a requirement to create a Header &Trailer for a file which is having 20 millions of records.
If I use the following method, i think it will take more time.
cat "Header"> file1.txt
cat Data_File>>file1.txt
cat "Trailer">>file1.txt
since second CAT command has to read all... (4 Replies)
Discussion started by: Raamc
4 Replies
4. Shell Programming and Scripting
How can we a shell script and pass date parameters .I have 3 files comming from Datastage with |" delimited
I need append 3 files as above: File1:
P0000|"47416954|"AU|"000|"INS|"0000|"|"20060601|"99991231|"|"|"|"|"01
File 2:... (2 Replies)
Discussion started by: e1994264
2 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I want to add the below Header to all the files in sequence File1,File2,File3...etc
"ABC,<number of chracter in the file>"
e,g - If File1 is as below
pqrstuvdt
abcdefgh
then I want to add the above header into it ,So that File1 becomes as below
ABC,17
pqrstuvdt
abcdefgh
... (9 Replies)
Discussion started by: spari2
9 Replies
6. Shell Programming and Scripting
Hi All,
I am trying write a simple command using AWK and SED to this but without any success.
Here is what I am using:
head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt
Input:
Header
1234567
abcdefgh... (2 Replies)
Discussion started by: nvuradi
2 Replies
7. Shell Programming and Scripting
Hi,
I need a script that validates a file header/detail/trailer. File layout is:
Header - Rec_Type|File_name|File_Date
Detail - Rec_Type|field1|field2|field3...
Trailder - Rec_Type|File_name|File_Date|Record_count
Sample Data:
HDR|customer_data.dat|20120709... (7 Replies)
Discussion started by: ash_sh
7 Replies
8. Shell Programming and Scripting
please see my requirement, I hope I am clear. (9 Replies)
Discussion started by: mirwasim
9 Replies
9. Shell Programming and Scripting
I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file
Also is it possible to output the TOM and PAT records in the same output file ?
I need the output file names same as xyz_pattern_Datetimestamp.txt
... (23 Replies)
Discussion started by: techedipro
23 Replies
10. UNIX for Advanced & Expert Users
I have a EBCDIC multi layout file which has a header record which is 21 bytes, The Detail records are 2427 bytes long and the trailer record is 9 bytes long.
Is there a command to remove the header as well as trailer record and read only the detail records while at the same time not altering... (1 Reply)
Discussion started by: abhilashnair
1 Replies
nl(1) General Commands Manual nl(1)
Name
nl - line numbering filter
Syntax
nl [-h type] [-b type] [-f type] [-v start#] [-i incr] [-p ] [-l num] [-s sep] [-w width] [-n format] [-d delim] file
Description
The command reads lines from the named file or from the standard input, if no file is named, and reproduces the lines on the standard out-
put. Lines are numbered on the left in accordance with the command options in effect.
The command views the text it reads in terms of logical pages. Line numbering is reset at the start of each logical page. A logical page
consists of a header, a body, and a footer section. Empty sections are valid. Different line numbering options are independently avail-
able for header, body, and footer. For example, you can elect not to number header and footer lines while numbering blank lines in the
body.
The start of logical page sections is signaled by input lines containing nothing but the following delimiter characters:
Line contents Start of
::: header
:: body
: footer
Unless otherwise specified, assumes that the text it is reading is in the body of a single logical page.
Options
Command options may appear in any order and may be intermingled with an optional file name. Only one file may be named.
-b type Specifies which logical page body lines are to be numbered. The following are recognized types and their meaning: a,
number all lines; t, number lines with printable text only; n, no line numbering; pstring, number only lines that con-
tain the regular expression specified in string.
The default type for logical page body is t (text lines numbered).
-h type Same as -b type except for header. Default type for logical page header is n (no lines numbered).
-f type Same as -b type except for footer. Default for logical page footer is n (no lines numbered).
-p Do not restart numbering at logical page delimiters.
-v start# The initial value used to number logical page lines. Default is 1.
-i incr The increment value used to number logical page lines. Default is 1.
-s sep The character used in separating the line number and the corresponding text line. Default sep is a tab.
-w width The number of characters used for the line number. Default width is 6.
-n format The line numbering format. Recognized values are the following: ln, left justified, leading zeroes suppressed; rn,
right justified, leading zeroes suppressed; rz, right justified, leading zeroes kept. Default format is rn (right jus-
tified).
-l num The number of blank lines to be considered as one. For example, -l2 results in only the second adjacent blank being
numbered (if the appropriate -ha, -ba, or -fa option is set). Default is 1.
-d xx The delimiter characters specifying the start of a logical page section may be changed from the default characters (:)
to two user-specified characters. If only one character is entered, the second character remains the default character
(:). No space should appear between the -d and the delimiter characters. To enter a backslash, you must type two
backslashes (//).
Examples
nl -v10 -i10 -d!+ file1
This command numbers file1 starting at line number 10 with an increment of ten. The logical page delimiters are !+.
See Also
pr(1)
nl(1)