Sponsored Content
Top Forums Shell Programming and Scripting Selecting Specific Columns and Insert the delimiter TAB Post 302547008 by filter on Saturday 13th of August 2011 04:16:07 AM
Old 08-13-2011
Selecting Specific Columns and Insert the delimiter TAB

Hi,
I am writing a Perl Script for the below :

I have a data file that consists of the header information which is 231 Lines and the footer information as 4 lines. The total number of line including the header and footer 1.2 Million with Pipe Delimited file.

For example:
Header Information:
Quote:
START-OF-FILE
FILENAME=fixedincome_bo_euro.out
DATA=bo
REGION=euro
TYPE=out
PROGRAMNAME=getdata
DATEFORMAT=yyyymmdd

... so on 231 Lines
Footer Information:
Quote:
END-OF-DATA
DATARECORDS=1221264
TIMEFINISHED=Fri Aug 12 18:57:09 BST 2011
END-OF-FILE
Data looks like:
Each line has around ~210 columns and is Pipe delimited.
Quote:
TT3069982 Corp|0|198|FSPIN|4.000000| | |FINE SPINNERS|FINE SPIN-CALLED|INDUSTRIAL|Corp|2|FIXED|PERP/CALL|PERPETL PAY,EX-DIV|3|DOMESTIC|EN|GBP|MORTGAGE BACKED|2000000.00|.00|1.0000|1.0000|1.00| |NOT LISTED|100.00000| | |N.A.|N.A.| |100.000000| | | | | | | | | | | | | | | | | | | | | |234953|500000|TT3069982| | | | | | | | | |N.A.| | | | | | | | | | | | |Y|N|N| | | |GB| |Basic Materials|Chemicals|Chemicals-Fibers|N.A.|GB|FSPIN 4 03/29/49|N| |DOMESTIC| |N.A.| | |N| |N|COTT3069982|Fine Spinners|GBP|GBP|N|N|Y|1|N|N|GBP|N|N|Y|19920228|FINE SPINNERS|Anytime| |N.A.| | |N|N|EN|EN|Does Not Apply|20490329|N|42| |Y|N|100.000000|N|20110820|.000000000| |N| | | | |N.A.|N.A.|N.A.|N.A.|N.A.| | | | | | |N|N|N|N| |Grandfathered| |2| | |N.A.|N| | |N| | | | |N| | |20490329| | |N|N|N| | |N|3| | | |N.A.|2| |41|CALENDAR| |N|N|BBG00035Y4Y1|
The outfile should contain the lines with only specific Columns and should be TAB delimited.
Specific Columns:
Quote:
3 4 5-7 10 11 12 13 15 16-19 20-24 25-26 27 28-32 33 36 37 40 55-58 59 60
61 62 63-66 68 69-72 73 74-75 76 77 78-79 80-86 87 88-94 95 96-99 100 101-103 105-107 109-110 112-123 125-128 130-131 133-135 137 111 124 132 136 187 Only.
So I have started writing the Perl script:

Quote:
#!/usr/bin/perl
$file='fileA';
open(F,$file)|| die ("could not open file $file: $!");
@array = <F>;
close F;
open(OUT,'>','outfile');
print OUT @array[231..$#array-4];
close OUT;
I am using array spice to eliminate the Header and footer information..Please correct me if I am wrong.

Now, Once I load the file into an array, how do I select the above selected columns and then insert the delimiter as TAB in Perl.

Would that be easier if I use hashes or array ?

Could someone Please help me out in this. Really appreciate your thoughts.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Cutting a tab delimiter file

I have a 30 column tab delimited record file. I need to extract the first 10column. The following command to cut was not working cut -f 1-10 -d "\t" filename. Could any one keep on this . Thanks in Advance (4 Replies)
Discussion started by: vinod.thayil
4 Replies

2. Shell Programming and Scripting

how to differentiate columns of a file in perl with no specific delimiter

Hi everybody, This time I am having one issue in perl. I have to create comma separated file using the following type of information. The problem is the columns do not have any specific delimiter. So while using split I am getting different value. Some where it is space(S) and some where it is... (9 Replies)
Discussion started by: Amiya Rath
9 Replies

3. Shell Programming and Scripting

append data in a file by using tab delimiter

Hi, I need to append the data in to a file by using tab delimiter. eg: echo "Data1" >> filename.txt echo "\t" >> filename.txt (its not working) echo "Data2" >> filename.txt. the result sould be like this. Data1 Data2 (6 Replies)
Discussion started by: Sharmila_P
6 Replies

4. UNIX for Advanced & Expert Users

how to search delimiter tab in a line and replace it

hi every one plz help me i want to search for a line contains tabspace This is a line The should be changed see the above line is seperated with tab space i want to replace that tab space in to # as This is a line#The should be changed i have tried with... (4 Replies)
Discussion started by: kkraja
4 Replies

5. UNIX for Dummies Questions & Answers

Delimiter: Tab or Space?

Hello, Is there a direct command to check if the delimiter in your file is a tab or a space? And how can they be converted from one to another. Thanks, G (4 Replies)
Discussion started by: Gussifinknottle
4 Replies

6. Shell Programming and Scripting

Selecting specific 'id's from lines and columns using 'SED' or 'AWK'

Hello experts, I am new to this group and to 'SED' and 'AWK'. I have data (text file) with 5 columns (C_1-5) and 100s of lines (only 10 lines are shown below as an example). I have to find or select only the id numbers (C-1) of specific lines with '90' in the same line (of C_3) AND with '20' in... (6 Replies)
Discussion started by: kamskamu
6 Replies

7. UNIX for Dummies Questions & Answers

Making a Tab delimiter file to Comma

How can i make a tab delimiter file to a comma delimiter??? (13 Replies)
Discussion started by: saggiboy10
13 Replies

8. Shell Programming and Scripting

Compare two tab-delimiter files

Hi, I have two files like: file1 chr1 40 chr1 50 chr2 10 chr2 60 file2 chr1 30 chr1 50 chr2 15 chr2 20 and want to get the difference of column 2 when column 1 is the same in both files. (4 Replies)
Discussion started by: linseyr
4 Replies

9. Shell Programming and Scripting

Insert space in specific column among many columns

Hello, I have some problem in inserting the space for the pairs of columns. I have the input file : I used this code below in replacing it using space in specific column (replace space in each two columns) sed -e "s/,/ /2" -e "s/,/ /3" inputfile Output showed : However, I have many... (3 Replies)
Discussion started by: awil
3 Replies

10. Shell Programming and Scripting

Delete and insert columns in a tab delimited file

Hi all , I have a file having 12 columns tab delimited . I need to read this file and remove the column 3 and column 4 and insert a word in column 3 as "AVIALABLE " Is there a way to do this . I am trying like below Thanks DJ cat $FILENAME|awk -F"\t" '{ print $1 "\t... (3 Replies)
Discussion started by: Hypesslearner
3 Replies
NL(1)							    BSD General Commands Manual 						     NL(1)

NAME
nl -- line numbering filter SYNOPSIS
nl [-p] [-b type] [-d delim] [-f type] [-h type] [-i incr] [-l num] [-n format] [-s sep] [-v startnum] [-w width] [file] DESCRIPTION
The nl utility reads lines from the named file or the standard input if the file argument is omitted, applies a configurable line numbering filter operation and writes the result to the standard output. The nl utility treats the text it reads in terms of logical pages. Unless specified otherwise, line numbering is reset at the start of each logical page. A logical page consists of a header, a body and a footer section; empty sections are valid. Different line numbering options are independently available for header, body and footer sections. The starts of logical page sections are signaled by input lines containing nothing but one of the following sequences of delimiter charac- ters: Line "Start of" ::: header :: body : footer If the input does not contain any logical page section signaling directives, the text being read is assumed to consist of a single logical page body. The following options are available: -b type Specify the logical page body lines to be numbered. Recognized type arguments are: a Number all lines. t Number only non-empty lines. n No line numbering. pexpr Number only those lines that contain the basic regular expression specified by expr. The default type for logical page body lines is t. -d delim Specify the delimiter characters used to indicate the start of a logical page section in the input file. At most two characters may be specified; if only one character is specified, the first character is replaced and the second character remains unchanged. The default delim characters are ``:''. -f type Specify the same as -b type except for logical page footer lines. The default type for logical page footer lines is n. -h type Specify the same as -b type except for logical page header lines. The default type for logical page header lines is n. -i incr Specify the increment value used to number logical page lines. The default incr value is 1. -l num If numbering of all lines is specified for the current logical section using the corresponding -b a, -f a or -h a option, specify the number of adjacent blank lines to be considered as one. For example, -l 2 results in only the second adjacent blank line being num- bered. The default num value is 1. -n format Specify the line numbering output format. Recognized format arguments are: ln Left justified. rn Right justified, leading zeros suppressed. rz Right justified, leading zeros kept. The default format is rn. -p Specify that line numbering should not be restarted at logical page delimiters. -s sep Specify the characters used in separating the line number and the corresponding text line. The default sep setting is a single tab character. -v startnum Specify the initial value used to number logical page lines; see also the description of the -p option. The default startnum value is 1. -w width Specify the number of characters to be occupied by the line number; in case the width is insufficient to hold the line number, it will be truncated to its width least significant digits. The default width is 6. EXIT STATUS
The nl utility exits 0 on success, and >0 if an error occurs. SEE ALSO
pr(1) STANDARDS
The nl utility conforms to X/Open Portability Guide Issue 4, Version 2 (``XPG4.2'') with the exception of not supporting the intermingling of the file operand with the options, which the standard considers an obsolescent feature to be removed from a further issue. HISTORY
The nl utility first appeared in AT&T System V Release 2 UNIX. BSD
February 15, 1999 BSD
All times are GMT -4. The time now is 09:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy