awk & cut record separator problem


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk & cut record separator problem
# 1  
Old 03-09-2008
awk & cut record separator problem

Hi All,

I've got some strange behaviour going on when trying to manipulate a file that contains spaces.

My input file looks something like this:

xxxxxxxxx,yyyy,sss sss sss,bbbbbbb

If I use awk:

Quote:
awk -F',' '{ print $3 }'
When running from the command line I get:

sss sss sss

But when running from a loop I get:

Quote:
sss


sss


sss
My loop looks like this:
Code:
for LINES in $( cat tgp.txt )
do  TITLE=$( echo "$LINES" | awk -F'~' '{ print $3 }' )
echo "$TITLE"
done

I get the same behaviour for cut commands like this:

Quote:
echo $LINES | cut -f2 -d'~'
Has some sort of system input/output field separator been changed? Or am I just doing something wrong? Smilie

Thanks, p.
# 2  
Old 03-09-2008
Add following line

Code:
IFS=","

or

Code:
IFS=""


Last edited by fpmurphy; 03-09-2008 at 01:20 PM..
# 3  
Old 03-09-2008
Here's some more testing to prove my point:

Quote:
# cat test
xxxxxxxxxxxxxxx,bbbb,hh ttnt thsth,ccccccc
xxxxxxxxxxxxxxx,kkkk,ccccc cccc,ttttttt
Quote:
# awk -F, '{ print $3 }' test
hh ttnt thsth
ccccc cccc
Quote:
# for LINE in $( cat test ); do title=$( echo $LINE | awk -F, '{ print $3 }' ); echo $line; echo $title; done

hh





ccccc

I am right in thinking this is not normal behaviour?
# 4  
Old 03-09-2008
Quote:
Originally Posted by fpmurphy
Add following line

Code:
IFS="',"

Sorry, must've posted my reply at the same time...

Where do I put this? in my shell script?
# 5  
Old 03-09-2008
fpmurphy - you were right! Thanks Smilie

I just had to tweek it like this:

Quote:
IFS=""
i.e. setting it to NULL.

I put this in my script before the cut's and it's doing the job!

Thanks again,

p.
# 6  
Old 03-09-2008
Interestingly the next point in my script where a file had to be manipulated exhibited the same problem - this file only had one field but the script saw just one entry eventhough there are many lines... I found that adding:

Quote:
unset IFS
fixed this problem... very weird.
# 7  
Old 03-09-2008
Quote:
Originally Posted by pondlife

My loop looks like this:
Code:
for LINES in $( cat tgp.txt )
do  TITLE=$( echo "$LINES" | awk -F'~' '{ print $3 }' )
echo "$TITLE"
done

seriously, for what you are doing in the above code, the "better" way is just to use awk. (or a while loop to iterate lines in files , with IFS set. )
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with awk regular expression for RS record separator

Hi, I'm using gawk to read a text file and count the sentences. I want to use a record separator of a period, exclamation mark and a question mark. The problem is that the file contains words like "Mr. Smith" so the periods in the appellation are tripping my record separator. This is my... (12 Replies)
Discussion started by: 1Brajesh
12 Replies

2. UNIX for Dummies Questions & Answers

Cut & awk

I am using the following code to modify all odd lines in a file: awk 'NR % 2 { print } !(NR % 2)' FWD-1.fas | cut -c5-600 I however, do not want cut to affect the odd lines Any help? (3 Replies)
Discussion started by: Xterra
3 Replies

3. Shell Programming and Scripting

Use string as Record separator in awk

Hello to all, Please some help on this. I have the file in format as below. How can I set the record separator as the string below in red "No. Time Source Destination Protocol Length Info" I've tried code below but it doesn't seem to... (6 Replies)
Discussion started by: cgkmal
6 Replies

4. Shell Programming and Scripting

awk - single quotes as record separator

How do I use single quotes as record separator in awk? I just couldn't figure that out. I know how to use single quotes as field separator, and double quotes as both field and record separator ... (1 Reply)
Discussion started by: locoroco
1 Replies

5. Shell Programming and Scripting

apply record separator to multiple files within a directory using awk

Hi, I have a bunch of records within a directory where each one has this form: (example file1) 1 2 50 90 80 90 43512 98 0909 79869 -9 7878 33222 8787 9090 89898 7878 8989 7878 6767 89 89 78676 9898 000 7878 5656 5454 5454 and i want for all of these files to be... (3 Replies)
Discussion started by: amarn
3 Replies

6. Shell Programming and Scripting

cut & AWK

HI all, i have the source file like below, 05 BL-FEE-CYC-CDE PIC S9(03) COMP-3. 05 BL-FEE-ERR-MSG PIC X(00030). 05 BL-FEE-TYPE PIC X(00001). 418181*# 05 ... (7 Replies)
Discussion started by: baskivs
7 Replies

7. Shell Programming and Scripting

awk, string as record separator, transposing rows into columns

I'm working on a different stage of a project that someone helped me address elsewhere in these threads. The .docs I'm cycling through look roughly like this: 1 of 26 DOCUMENTS Copyright 2010 The Age Company Limited All Rights Reserved The Age (Melbourne, Australia) November 27, 2010... (9 Replies)
Discussion started by: spindoctor
9 Replies

8. Shell Programming and Scripting

Using > as record separator

I have tried to use ">" as record separator, but it doesn't work. I have tried this: awk BEGIN{RS=">"}'{print $0}' input output: awk: BEGIN{RS=>}{print $0} awk: ^ syntax error awk BEGIN{RS="\>"}'{print $0}' input awk: BEGIN{RS=\>}{print $0} awk: ^ backslash not... (2 Replies)
Discussion started by: locoroco
2 Replies

9. Shell Programming and Scripting

awk - double quotes as record separator

How do I use double quotes as a record seperator in awk? (4 Replies)
Discussion started by: locoroco
4 Replies

10. Shell Programming and Scripting

record separator

can anyone tell me any way to change record separator (default is new line). RS in nawk as not working. Thanks in advance. Regards Rochit (7 Replies)
Discussion started by: rochitsharma
7 Replies
Login or Register to Ask a Question

Featured Tech Videos