Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Selecting Specific Columns and Insert the delimiter TAB
Top Forums Shell Programming and Scripting Selecting Specific Columns and Insert the delimiter TAB Post 302547008 by filter on Saturday 13th of August 2011 04:16:07 AM
Old 08-13-2011
Selecting Specific Columns and Insert the delimiter TAB
Hi,
I am writing a Perl Script for the below :

I have a data file that consists of the header information which is 231 Lines and the footer information as 4 lines. The total number of line including the header and footer 1.2 Million with Pipe Delimited file.

For example:
Header Information:
Quote:
START-OF-FILE
FILENAME=fixedincome_bo_euro.out
DATA=bo
REGION=euro
TYPE=out
PROGRAMNAME=getdata
DATEFORMAT=yyyymmdd

... so on 231 Lines
Footer Information:
Quote:
END-OF-DATA
DATARECORDS=1221264
TIMEFINISHED=Fri Aug 12 18:57:09 BST 2011
END-OF-FILE
Data looks like:
Each line has around ~210 columns and is Pipe delimited.
Quote:
TT3069982 Corp|0|198|FSPIN|4.000000| | |FINE SPINNERS|FINE SPIN-CALLED|INDUSTRIAL|Corp|2|FIXED|PERP/CALL|PERPETL PAY,EX-DIV|3|DOMESTIC|EN|GBP|MORTGAGE BACKED|2000000.00|.00|1.0000|1.0000|1.00| |NOT LISTED|100.00000| | |N.A.|N.A.| |100.000000| | | | | | | | | | | | | | | | | | | | | |234953|500000|TT3069982| | | | | | | | | |N.A.| | | | | | | | | | | | |Y|N|N| | | |GB| |Basic Materials|Chemicals|Chemicals-Fibers|N.A.|GB|FSPIN 4 03/29/49|N| |DOMESTIC| |N.A.| | |N| |N|COTT3069982|Fine Spinners|GBP|GBP|N|N|Y|1|N|N|GBP|N|N|Y|19920228|FINE SPINNERS|Anytime| |N.A.| | |N|N|EN|EN|Does Not Apply|20490329|N|42| |Y|N|100.000000|N|20110820|.000000000| |N| | | | |N.A.|N.A.|N.A.|N.A.|N.A.| | | | | | |N|N|N|N| |Grandfathered| |2| | |N.A.|N| | |N| | | | |N| | |20490329| | |N|N|N| | |N|3| | | |N.A.|2| |41|CALENDAR| |N|N|BBG00035Y4Y1|
The outfile should contain the lines with only specific Columns and should be TAB delimited.
Specific Columns:
Quote:
3 4 5-7 10 11 12 13 15 16-19 20-24 25-26 27 28-32 33 36 37 40 55-58 59 60
61 62 63-66 68 69-72 73 74-75 76 77 78-79 80-86 87 88-94 95 96-99 100 101-103 105-107 109-110 112-123 125-128 130-131 133-135 137 111 124 132 136 187 Only.
So I have started writing the Perl script:

Quote:
#!/usr/bin/perl
$file='fileA';
open(F,$file)|| die ("could not open file $file: $!");
@array = <F>;
close F;
open(OUT,'>','outfile');
print OUT @array[231..$#array-4];
close OUT;
I am using array spice to eliminate the Header and footer information..Please correct me if I am wrong.

Now, Once I load the file into an array, how do I select the above selected columns and then insert the delimiter as TAB in Perl.

Would that be easier if I use hashes or array ?

Could someone Please help me out in this. Really appreciate your thoughts.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Cutting a tab delimiter file

I have a 30 column tab delimited record file. I need to extract the first 10column. The following command to cut was not working cut -f 1-10 -d "\t" filename. Could any one keep on this . Thanks in Advance (4 Replies)
Discussion started by: vinod.thayil
4 Replies

2. Shell Programming and Scripting

how to differentiate columns of a file in perl with no specific delimiter

Hi everybody, This time I am having one issue in perl. I have to create comma separated file using the following type of information. The problem is the columns do not have any specific delimiter. So while using split I am getting different value. Some where it is space(S) and some where it is... (9 Replies)
Discussion started by: Amiya Rath
9 Replies

3. Shell Programming and Scripting

append data in a file by using tab delimiter

Hi, I need to append the data in to a file by using tab delimiter. eg: echo "Data1" >> filename.txt echo "\t" >> filename.txt (its not working) echo "Data2" >> filename.txt. the result sould be like this. Data1 Data2 (6 Replies)
Discussion started by: Sharmila_P
6 Replies

4. UNIX for Advanced & Expert Users

how to search delimiter tab in a line and replace it

hi every one plz help me i want to search for a line contains tabspace This is a line The should be changed see the above line is seperated with tab space i want to replace that tab space in to # as This is a line#The should be changed i have tried with... (4 Replies)
Discussion started by: kkraja
4 Replies

5. UNIX for Dummies Questions & Answers

Delimiter: Tab or Space?

Hello, Is there a direct command to check if the delimiter in your file is a tab or a space? And how can they be converted from one to another. Thanks, G (4 Replies)
Discussion started by: Gussifinknottle
4 Replies

6. Shell Programming and Scripting

Selecting specific 'id's from lines and columns using 'SED' or 'AWK'

Hello experts, I am new to this group and to 'SED' and 'AWK'. I have data (text file) with 5 columns (C_1-5) and 100s of lines (only 10 lines are shown below as an example). I have to find or select only the id numbers (C-1) of specific lines with '90' in the same line (of C_3) AND with '20' in... (6 Replies)
Discussion started by: kamskamu
6 Replies

7. UNIX for Dummies Questions & Answers

Making a Tab delimiter file to Comma

How can i make a tab delimiter file to a comma delimiter??? (13 Replies)
Discussion started by: saggiboy10
13 Replies

8. Shell Programming and Scripting

Compare two tab-delimiter files

Hi, I have two files like: file1 chr1 40 chr1 50 chr2 10 chr2 60 file2 chr1 30 chr1 50 chr2 15 chr2 20 and want to get the difference of column 2 when column 1 is the same in both files. (4 Replies)
Discussion started by: linseyr
4 Replies

9. Shell Programming and Scripting

Insert space in specific column among many columns

Hello, I have some problem in inserting the space for the pairs of columns. I have the input file : I used this code below in replacing it using space in specific column (replace space in each two columns) sed -e "s/,/ /2" -e "s/,/ /3" inputfile Output showed : However, I have many... (3 Replies)
Discussion started by: awil
3 Replies

10. Shell Programming and Scripting

Delete and insert columns in a tab delimited file

Hi all , I have a file having 12 columns tab delimited . I need to read this file and remove the column 3 and column 4 and insert a word in column 3 as "AVIALABLE " Is there a way to do this . I am trying like below Thanks DJ cat $FILENAME|awk -F"\t" '{ print $1 "\t... (3 Replies)
Discussion started by: Hypesslearner
3 Replies

LEARN ABOUT BSD

ascii

ASCII(7)						 Miscellaneous Information Manual						  ASCII(7)

NAME

       ascii - map of ASCII character set

SYNOPSIS

       cat /usr/pub/ascii

DESCRIPTION

       Ascii is a map of the ASCII character set, to be printed as needed.  It contains:

       |000 nul|001 soh|002 stx|003 etx|004 eot|005 enq|006 ack|007 bel|
       |010 bs |011 ht |012 nl |013 vt |014 np |015 cr |016 so |017 si |
       |020 dle|021 dc1|022 dc2|023 dc3|024 dc4|025 nak|026 syn|027 etb|
       |030 can|031 em |032 sub|033 esc|034 fs |035 gs |036 rs |037 us |
       |040 sp |041  ! |042  " |043  # |044  $ |045  % |046  & |047  ' |
       |050  ( |051  ) |052  * |053  + |054  , |055  - |056  . |057  / |
       |060  0 |061  1 |062  2 |063  3 |064  4 |065  5 |066  6 |067  7 |
       |070  8 |071  9 |072  : |073  ; |074  < |075  = |076  > |077  ? |
       |100  @ |101  A |102  B |103  C |104  D |105  E |106  F |107  G |
       |110  H |111  I |112  J |113  K |114  L |115  M |116  N |117  O |
       |120  P |121  Q |122  R |123  S |124  T |125  U |126  V |127  W |
       |130  X |131  Y |132  Z |133  [ |134   |135  ] |136  ^ |137  _ |
       |140  ` |141  a |142  b |143  c |144  d |145  e |146  f |147  g |
       |150  h |151  i |152  j |153  k |154  l |155  m |156  n |157  o |
       |160  p |161  q |162  r |163  s |164  t |165  u |166  v |167  w |
       |170  x |171  y |172  z |173  { |174  | |175  } |176  ~ |177 del|

       | 00 nul| 01 soh| 02 stx| 03 etx| 04 eot| 05 enq| 06 ack| 07 bel|
       | 08 bs | 09 ht | 0a nl | 0b vt | 0c np | 0d cr | 0e so | 0f si |
       | 10 dle| 11 dc1| 12 dc2| 13 dc3| 14 dc4| 15 nak| 16 syn| 17 etb|
       | 18 can| 19 em | 1a sub| 1b esc| 1c fs | 1d gs | 1e rs | 1f us |
       | 20 sp | 21  ! | 22  " | 23  # | 24  $ | 25  % | 26  & | 27  ' |
       | 28  ( | 29  ) | 2a  * | 2b  + | 2c  , | 2d  - | 2e  . | 2f  / |
       | 30  0 | 31  1 | 32  2 | 33  3 | 34  4 | 35  5 | 36  6 | 37  7 |
       | 38  8 | 39  9 | 3a  : | 3b  ; | 3c  < | 3d  = | 3e  > | 3f  ? |
       | 40  @ | 41  A | 42  B | 43  C | 44  D | 45  E | 46  F | 47  G |
       | 48  H | 49  I | 4a  J | 4b  K | 4c  L | 4d  M | 4e  N | 4f  O |
       | 50  P | 51  Q | 52  R | 53  S | 54  T | 55  U | 56  V | 57  W |
       | 58  X | 59  Y | 5a  Z | 5b  [ | 5c   | 5d  ] | 5e  ^ | 5f  _ |
       | 60  ` | 61  a | 62  b | 63  c | 64  d | 65  e | 66  f | 67  g |
       | 68  h | 69  i | 6a  j | 6b  k | 6c  l | 6d  m | 6e  n | 6f  o |
       | 70  p | 71  q | 72  r | 73  s | 74  t | 75  u | 76  v | 77  w |
       | 78  x | 79  y | 7a  z | 7b  { | 7c  | | 7d  } | 7e  ~ | 7f del|

FILES

       /usr/pub/ascii

7th Edition							   May 20, 1985 							  ASCII(7)
All times are GMT -4. The time now is 09:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy