Sponsored Content
Top Forums Shell Programming and Scripting Read a tab separated file with empty column Post 302904753 by maturix on Friday 6th of June 2014 08:39:39 AM
Old 06-06-2014
Read a tab separated file with empty column

Hi all,

I'm trying to read a tab separated file and apply some functions on each column. I have an issue with empty column.

Exemple:
Code:
$ #cat with the sed to allow you to see my tab
$ cat foo.txt| sed 's/\t/;/g' 
a;1;x
b;;y

I wanted to something like that:
Code:
while read col1 col2 col3
do
 function1($col1)
 function2($col2)
 function3($col3)
done < foo.txt

This don't work as the second line has a empty column and so the "e" will be in $col2 instead of $col3.

The only solution I find is to do
Code:
sed -i 's/\t/;/g' foo.txt
IFS=';'
while read col1 col2 col3
do
 function1($col1)
 function2($col2)
 function3($col3)
done < foo.txt
sed -i 's/;/\t/g' foo.txt
IFS=' \t\n'

Is there any other cleaner and faster solution ?

Thanks for your help.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Adding EMPTY columns to Tab-delimited txt file

Hi I have a txt file with 4 columns where I need to add 4 empty columns in the middle meaning that I need what is currently column 4 to be column 8 in a new file. The idea is that I have to use the file as input in a program that reads the data in column 1 and 8, so the content of the other... (8 Replies)
Discussion started by: Banni
8 Replies

2. Shell Programming and Scripting

parse file into tab separated columns

Hello, I am trying to parse a file that resembles the last three groupings into something looking like the first two lines. I've fiddled with sed and awk a bit, but can't get anything to work properly. I need them separated by some delimiter. The file is some 23,000 lines of the stuff.... ... (9 Replies)
Discussion started by: dkozel
9 Replies

3. UNIX for Dummies Questions & Answers

Sum up a decimal column in a tab separated text file and error handling

Hi, I have a small requirement where i need to sum up a column in a text file. Input file 66ab 000000 534385 -00000106350.00 66cd 000000 534485 -00013364511.00 66ad 000000 534485 -00000426548.00 672a 000000 534485 000000650339.82... (5 Replies)
Discussion started by: pssandeep
5 Replies

4. Shell Programming and Scripting

Convert a tab separated file using bash

Dear all, I have a file in this format (like a matrix) - A B C .. X A 1 4 2 .. 2 B 2 6 4 .. 8 C 3 5 5 .. 4 . . . ... . X . . ... . and want to convert it into a file with this format: A A = 1 A B = 4 A C = 2 ... A X = 2 B A = 2 B B = 6 etc (2 Replies)
Discussion started by: TheTransporter
2 Replies

5. UNIX for Dummies Questions & Answers

tab-separated file to matrix conversion

hello all, i have an input file like that A A X0 A B X1 A C X2 ... A Z Xx B A X1 B B X3 .... Z A Xx Z B X4 and i want to have an output like that A B C D A X0 X1 X2 Xy B X1 X3 X4 (4 Replies)
Discussion started by: TheTransporter
4 Replies

6. UNIX for Dummies Questions & Answers

Filling a tab-separated file with known missing entries in columns

Hello all, I have a file which is tab separated like that: PHE_205_A TIP_127_W ARG_150_B MET_1150_A TIP_12_W VAL_11_B GLU_60_A TIP_130_W ARG_143_B LEU_1033_A TIP_203_W ARG_14_B SER_1092_A TIP_203_W THR_1090_A TIP_203_W SER_1092_A TIP_25_W ... (6 Replies)
Discussion started by: TheTransporter
6 Replies

7. Shell Programming and Scripting

Problem with a tab separated file

Hi, I have created a tab separated file from the following input file. ADDRESS1 CITY STATE POSTAL COUNTRY LON LAT 32 PRINZREGENTENSTRASSE ROSENHEIM BAYERN 83022 DEU 1212182 4785699 263 VIA DANTE ALIGHIERI BARI PUGLIA 70122 ITA 1686233 4112154 30 VIA MILANO ... (1 Reply)
Discussion started by: ramky79
1 Replies

8. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing? (9 Replies)
Discussion started by: SirHenry1
9 Replies

9. Shell Programming and Scripting

How to replace & with and in tab separated file?

Hi, I have a tab separated. I want to replace all the "&" in 8th column of the file with "and" .I am trying with awk -F, -vOFS=\\t '{$8=($8=="&")?"and":$8}1' test> test1.txt My file is abc def ghk hjk lkm hgb jkluy acvf & bhj hihuhu fgg me mine he her go went has has & had hgf hgy ... (1 Reply)
Discussion started by: jagdishrout
1 Replies

10. UNIX for Beginners Questions & Answers

Replace a column in tab delimited file with column in other tab delimited file,based on match

Hello Everyone.. I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files FileP.txt ... (2 Replies)
Discussion started by: YogeshG
2 Replies
funsky(1)							SAORD Documentation							 funsky(1)

NAME
funsky - convert between image and sky coordinates SYNOPSIS
funsky iname[ext] # RA,Dec (deg) or image pix from stdin funsky iname[ext] [lname] # RA, Dec (deg) or image pix from list funsky iname[ext] [col1] [col2] # named cols:units from stdin funsky iname[ext] [lname] [col1] [col2] # named cols:units from list OPTIONS
-d # always use integer tlmin conversion (as ds9 does) -r # convert x,y to RA,Dec (default: convert RA,Dec to x,y) -o # include offset from the nominal target position (in arcsec) -v # display input values also (default: display output only) -T # output display in rdb format (w/header,tab delimiters) DESCRIPTION
Funsky converts input sky coordinates (RA, Dec) to image coordinates (or vice versa) using the WCS information contained in the specified FITS file. Several calling sequences are supported in order to make it easy to specify coordinate positions in different ways. The first required argument is always the input FITS file (or extension) containing the WCS information in an extension header. Note that the data from this file is not used. By default, the program converts input RA and Dec values to X and Y using this WCS information. If the WCS is associated with a FITS image, then the X,Y values are image values. If the WCS is associated with a binary table, then the X, Y val- ues are physical values. To convert X,Y to RA and Dec, use the -r (reverse) switch. If no other command arguments are supplied, then the input positions are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA in hours (or X in pixels) followed by a Dec in degrees (or Y in pixels). The usual delim- iters are supported (spaces, commas, tabs). For example: # read from stdin, default column names and units [sh] funsky snr.ev 22.982695 58.606523 # input RA (hrs), Dec(deg) 510.00 510.00 22.982127 58.607634 # input 512.00 510.50 22.981700 58.614301 # input 513.50 513.50 ^D # end of input If a second argument is supplied, this argument is assumed to be a file containing RA (X) and Dec (Y) positions. The file can either be an ASCII table or a FITS binary table. The order of columns is unimportant, if the table has a column header. In this case, the names of the columns must be one of "RA", "DEC", or "X", "Y" for sky to image and image to sky conversions, respectively. If the table has no header, then once again, RA (X) is assumed to first, followed by DEC (Y). For example: # read from file, default column names and units [sh] cat hd.in RA DEC --------- --------- 22.982695 58.606523 22.982127 58.607634 22.981700 58.614301 [sh] funsky snr.ev hd.in 510.00 510.00 512.00 510.50 513.50 513.50 If three arguments are supplied, then the input positions again are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA (or X in pixels) followed by a Dec (or Y in pixels), with the usual delimiters supported. However, the second and third arguments now specify the column names and/or sky units using a colon-delimited syntax: [colname]:[h|d|r] If the colname is omitted, the names default to "RA", "DEC", "X", "Y", "COL1", or "COL2" as above. If the units are omitted, the default is hours for RA and degrees for Dec. When the -r switch is used (convert from image to sky) the units are applied to the output instead of the input. The following examples will serve to illustrate the options: # read from stdin, specifying column names (def. units: hours, degrees) [sh] cat hd.in MYRA MYDEC --------- --------- 22.982695 58.606523 22.982127 58.607634 22.981700 58.614301 [sh] funsky snr.ev MYRA MYDEC < hd.in 510.00 510.00 512.00 510.50 513.50 513.50 # read from stdin, specifying column names and units [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev MYRA:d MYDEC:d < dd.in 510.00 510.00 512.00 510.50 513.50 513.50 # read stdin, convert image to sky, specifying output sky units [sh] cat im.in 510.00 510.00 512.00 510.50 513.50 513.50 [sh] cat im.in | funsky -r snr.ev :d :d 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 Finally, four command arguments specify both and input file and column names and/or units: [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev dd.in MYRA:d MYDEC:d 510.00 510.00 512.00 510.50 513.50 513.50 # read file, convert image to sky, specifying output sky units [sh] cat im.in 510.00 510.00 512.00 510.50 513.50 513.50 [sh] funsky -r snr.ev im.in :d :d 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 By default, the output of funsky consists only of the converted coordinate position(s), one per output line. This makes parsing in shell scripts easy. Use the -v (verbose) switch to specify that the input coordinates should be pre-pended to each line. For example: [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev dd.in MYRA:d MYDEC:d 510.00 510.00 512.00 510.50 513.50 513.50 [sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d 344.740432 58.606523 510.00 510.00 344.731900 58.607634 512.00 510.50 344.725500 58.614301 513.50 513.50 In addition, a full starbase table can be output using the -T (table) switch. This switch can be used with or without the -v switch. If the -T and -v are both specified, then a descriptive header parameters are output before the table (mainly to remind you of the sky units): # output table in non-verbose mode [sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d X Y ------------ ------------ 510.00 510.00 512.00 510.50 513.50 513.50 # output table in verbose mode [sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d # IFILE = /Users/eric/data/snr.ev # ICOL1 = MYRA # ICOL2 = MYDEC # IUNITS1 = d # IUNITS2 = d # OCOL1 = X # OCOL2 = Y MYRA MYDEC X Y ------------ ------------ ------------ ------------ 344.740432 58.606523 510.00 510.00 344.731900 58.607634 512.00 510.50 344.725500 58.614301 513.50 513.50 Finally, the -d (ds9) switch mimicks ds9's use of integer TLMIN and TLMAX values for all coordinate transformations. FITS conventions seem to call for use of floating point TLMIN and TLMAX when the data are floats. This convention is followed by funsky but results in a small discrepancy with ds9's converted values for floating point data. We will remedy this conflict in the future, maybe. SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funsky(1)
All times are GMT -4. The time now is 12:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy