Read a tab separated file with empty column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read a tab separated file with empty column
# 1  
Old 06-06-2014
Read a tab separated file with empty column

Hi all,

I'm trying to read a tab separated file and apply some functions on each column. I have an issue with empty column.

Exemple:
Code:
$ #cat with the sed to allow you to see my tab
$ cat foo.txt| sed 's/\t/;/g' 
a;1;x
b;;y

I wanted to something like that:
Code:
while read col1 col2 col3
do
 function1($col1)
 function2($col2)
 function3($col3)
done < foo.txt

This don't work as the second line has a empty column and so the "e" will be in $col2 instead of $col3.

The only solution I find is to do
Code:
sed -i 's/\t/;/g' foo.txt
IFS=';'
while read col1 col2 col3
do
 function1($col1)
 function2($col2)
 function3($col3)
done < foo.txt
sed -i 's/;/\t/g' foo.txt
IFS=' \t\n'

Is there any other cleaner and faster solution ?

Thanks for your help.
# 2  
Old 06-06-2014
Hi,

If you use bash or ksh93 or zsh try:
Code:
while IFS=$'\t' read col1 col2 col3
do
  ...
done < foo.txt

to use a TAB-character as field separator.

More generally (in any other kind of POSIX shell) you could use:
Code:
while IFS="     " read col1 col2 col3
do

Where the hard tab is inserted between the quotes (typically using CTRL-V TAB)

or by using command substitution
Code:
while IFS=$(printf "\t") read col1 col2 col3
do

With these methods IFS is set local to the read command and so does not need to reset afterwards. Also there is no need for file manipulation with sed..

Last edited by Scrutinizer; 06-06-2014 at 11:05 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 3  
Old 06-06-2014
As you don't mention the shell you use, I'll comment just on bash. From man bash:
Quote:
Word Splitting . . .
. . . A sequence of IFS whitespace characters is also treated as a delimiter. . . .
which implies you have to use a different IFS char. Try:
Code:
sed 's/\t/;/g' foo.txt | while IFS=';' read A B C; do echo "$A", "$B", "$C"; done

# 4  
Old 06-06-2014
Having fought BASH on this a while, changing IFS does not help...

Code:
IFS=":" read A B <<<"A::B"
$ echo $A
A
$ echo $B
:B
$

I would try converting "empty" columns into non-empty columns, plugging them with spaces or something. Or just use a language which considers them as multiple columns already, like awk.
# 5  
Old 06-06-2014
Quote:
Originally Posted by Corona688
Having fought BASH on this a while, changing IFS does not help...

Code:
IFS=":" read A B <<<"A::B"
$ echo $A
A
$ echo $B
:B
$

I would try converting "empty" columns into non-empty columns, plugging them with spaces or something. Or just use a language which considers them as multiple columns already, like awk.
That is because everything after the empty string (of the second column) is assigned to variable Bas well. This is by design. Try:
Code:
IFS=":" read A B dummy <<<"A::B"
$ echo $A
A
$ echo $B

$

Either the number of variables needs to be equal to the number of columns in the file. Or the number of variables can be fewer, but then unimportant columns at the end need to land in some sort of dummy variable..

Last edited by Scrutinizer; 06-06-2014 at 10:52 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace a column in tab delimited file with column in other tab delimited file,based on match

Hello Everyone.. I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files FileP.txt ... (2 Replies)
Discussion started by: YogeshG
2 Replies

2. Shell Programming and Scripting

How to replace & with and in tab separated file?

Hi, I have a tab separated. I want to replace all the "&" in 8th column of the file with "and" .I am trying with awk -F, -vOFS=\\t '{$8=($8=="&")?"and":$8}1' test> test1.txt My file is abc def ghk hjk lkm hgb jkluy acvf & bhj hihuhu fgg me mine he her go went has has & had hgf hgy ... (1 Reply)
Discussion started by: jagdishrout
1 Replies

3. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing? (9 Replies)
Discussion started by: SirHenry1
9 Replies

4. Shell Programming and Scripting

Problem with a tab separated file

Hi, I have created a tab separated file from the following input file. ADDRESS1 CITY STATE POSTAL COUNTRY LON LAT 32 PRINZREGENTENSTRASSE ROSENHEIM BAYERN 83022 DEU 1212182 4785699 263 VIA DANTE ALIGHIERI BARI PUGLIA 70122 ITA 1686233 4112154 30 VIA MILANO ... (1 Reply)
Discussion started by: ramky79
1 Replies

5. UNIX for Dummies Questions & Answers

Filling a tab-separated file with known missing entries in columns

Hello all, I have a file which is tab separated like that: PHE_205_A TIP_127_W ARG_150_B MET_1150_A TIP_12_W VAL_11_B GLU_60_A TIP_130_W ARG_143_B LEU_1033_A TIP_203_W ARG_14_B SER_1092_A TIP_203_W THR_1090_A TIP_203_W SER_1092_A TIP_25_W ... (6 Replies)
Discussion started by: TheTransporter
6 Replies

6. UNIX for Dummies Questions & Answers

tab-separated file to matrix conversion

hello all, i have an input file like that A A X0 A B X1 A C X2 ... A Z Xx B A X1 B B X3 .... Z A Xx Z B X4 and i want to have an output like that A B C D A X0 X1 X2 Xy B X1 X3 X4 (4 Replies)
Discussion started by: TheTransporter
4 Replies

7. Shell Programming and Scripting

Convert a tab separated file using bash

Dear all, I have a file in this format (like a matrix) - A B C .. X A 1 4 2 .. 2 B 2 6 4 .. 8 C 3 5 5 .. 4 . . . ... . X . . ... . and want to convert it into a file with this format: A A = 1 A B = 4 A C = 2 ... A X = 2 B A = 2 B B = 6 etc (2 Replies)
Discussion started by: TheTransporter
2 Replies

8. UNIX for Dummies Questions & Answers

Sum up a decimal column in a tab separated text file and error handling

Hi, I have a small requirement where i need to sum up a column in a text file. Input file 66ab 000000 534385 -00000106350.00 66cd 000000 534485 -00013364511.00 66ad 000000 534485 -00000426548.00 672a 000000 534485 000000650339.82... (5 Replies)
Discussion started by: pssandeep
5 Replies

9. Shell Programming and Scripting

parse file into tab separated columns

Hello, I am trying to parse a file that resembles the last three groupings into something looking like the first two lines. I've fiddled with sed and awk a bit, but can't get anything to work properly. I need them separated by some delimiter. The file is some 23,000 lines of the stuff.... ... (9 Replies)
Discussion started by: dkozel
9 Replies

10. UNIX for Dummies Questions & Answers

Adding EMPTY columns to Tab-delimited txt file

Hi I have a txt file with 4 columns where I need to add 4 empty columns in the middle meaning that I need what is currently column 4 to be column 8 in a new file. The idea is that I have to use the file as input in a program that reads the data in column 1 and 8, so the content of the other... (8 Replies)
Discussion started by: Banni
8 Replies
Login or Register to Ask a Question