Strange behaviour when output to terminal vs file (awk) | Unix Linux Forums | UNIX for Dummies Questions & Answers

  Go Back    


UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

Strange behaviour when output to terminal vs file (awk)

UNIX for Dummies Questions & Answers


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 12-12-2012
beca123456 beca123456 is offline
Registered User
 
Join Date: Apr 2012
Last Activity: 20 June 2014, 9:51 PM EDT
Posts: 70
Thanks: 36
Thanked 0 Times in 0 Posts
Strange behaviour when output to terminal vs file (awk)

Hi all !

I noticed something very weird.

I have a large pipe delimited file (20 fields/3,000 records) that looks like that:

Code:
AAA|BBB|11111|22222|...|($NF of record 1)
CCC|DDD|33333|44444|...|($NF of record 2)
CCC|DDD|55555|66666|...|($NF of record 3)

For the lines with same 1st and 2nd fields, I write the 3rd up to 20th fields at the end of the previous line.
In order to obtain this:

Code:
AAA|BBB|11111|22222|...|($NF of record 1)
CCC|DDD|33333|44444|...|($NF of record 2)|55555|66666|...|$NF of record 3

I use this command:

Code:
gawk 'BEGIN{FS=OFS="|"}{a[$1$2]=(a[$1$2]?a[$1$2]:$1 FS $2) FS $3 FS $4 FS $5 FS $6 FS $7 FS $8 FS $9 FS $10 FS $11 FS $12 FS $13 FS $14 FS $15 FS $16 FS $17 FS $18 FS $19 FS $20}END{for(i in a) print a[i]}' input

When I redirect the command to my terminal (iTerm) it works perfectly (output has 200 records).
When I redirect the command to an output file and count (wc -l) the number of lines, it also gives me 200 records.

But when I open the file with my usual text editor (Smultron), the duplicated 1st and 2nd fields just disappeared and the remaining fields are written in a new line (giving me the same number of lines as the input):

Code:
AAA|BBB|11111|22222|...|($NF of record 1)
CCC|DDD|33333|44444|...|($NF of record 2)
|55555|66666|...|($NF of record 3)

I really don't understand what is wrong.
I used this command many times, it is the first time it does that. Plus I tested the command with an input file with just 3 rows (like the example above, and it woks).

Have you already got this problem?!!!

Thanks for your help.

---------- Post updated at 09:09 PM ---------- Previous update was at 08:46 PM ----------

***EDIT***
- Both iTerm and Smultron use Unicode (UTF-8)
- It worked properly when I don't go up to the last field
Sponsored Links
    #2  
Old 12-16-2012
RudiC RudiC is offline Forum Advisor  
Registered User
 
Join Date: Jul 2012
Last Activity: 24 July 2014, 5:37 PM EDT
Location: Aachen, Germany
Posts: 3,878
Thanks: 62
Thanked 920 Times in 873 Posts
Does your Smultron editor force a line break on long lines?
Sponsored Links
    #3  
Old 12-16-2012
Don Cragun's Avatar
Don Cragun Don Cragun is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 24 July 2014, 9:12 PM EDT
Location: San Jose, CA, USA
Posts: 4,135
Thanks: 162
Thanked 1,412 Times in 1,197 Posts
Note that with 20 fields on each of 3,000 input records being combined into 200 output lines, your average output lines with have 300 fields and some lines could have many more. You don't give us any indication of what system you're using, nor of the contents of most of the input fields. The awk utility and most editors are only defined to work on text files, and by definition, lines in a text file can't be longer than LINE_MAX bytes (including the terminating newline character). (Try:
Code:
getconf LINE_MAX

to determine the value of LINE_MAX on your system. The standards only require that implementations support lines up to 2,048 bytes per line.) Are you sure that none of your output lines exceed LINE_MAX?

If you ask awk to print a line that is longer than LINE_MAX bytes long, the results are unspecified. If you use ed, ex, grep, sed, vi (or any of LOTS of other standard utilities that are described as processing text files) to read or write or create internal lines longer than LINE_MAX bytes long, the results are unspecified. There are very few standard text processing utilities that are defined to work on lines with arbitrary lengths (cut, fold, and paste).
    #4  
Old 12-16-2012
bdittmar bdittmar is offline
Registered User
 
Join Date: Mar 2008
Last Activity: 15 November 2013, 9:55 AM EST
Location: Germany
Posts: 67
Thanks: 2
Thanked 4 Times in 4 Posts
Found this !

Hello,

found this :

Smultron is an elegant and powerful text editor for Mac OS X that is easy to use.

So i think it's a MAC.

Regards
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Strange perl print output behaviour gvj UNIX for Dummies Questions & Answers 5 09-23-2011 09:15 AM
Strange terminal behaviour after killing ssh RECrerar Shell Programming and Scripting 2 08-10-2011 04:36 AM
strange behaviour from sed??? alirezan Shell Programming and Scripting 2 10-04-2008 06:19 PM
Strange Program behaviour mrpugster UNIX for Dummies Questions & Answers 2 09-12-2008 03:20 PM
A Strange Behaviour!!! navojit dutta Shell Programming and Scripting 5 12-21-2007 03:35 AM



All times are GMT -4. The time now is 09:34 PM.