Strange behaviour when output to terminal vs file (awk)
Hi all !
I noticed something very weird.
I have a large pipe delimited file (20 fields/3,000 records) that looks like that:
For the lines with same 1st and 2nd fields, I write the 3rd up to 20th fields at the end of the previous line.
In order to obtain this:
I use this command:
When I redirect the command to my terminal (iTerm) it works perfectly (output has 200 records).
When I redirect the command to an output file and count (wc -l) the number of lines, it also gives me 200 records.
But when I open the file with my usual text editor (Smultron), the duplicated 1st and 2nd fields just disappeared and the remaining fields are written in a new line (giving me the same number of lines as the input):
I really don't understand what is wrong.
I used this command many times, it is the first time it does that. Plus I tested the command with an input file with just 3 rows (like the example above, and it woks).
Have you already got this problem?!!!
Thanks for your help.
---------- Post updated at 09:09 PM ---------- Previous update was at 08:46 PM ----------
***EDIT***
- Both iTerm and Smultron use Unicode (UTF-8)
- It worked properly when I don't go up to the last field
Note that with 20 fields on each of 3,000 input records being combined into 200 output lines, your average output lines with have 300 fields and some lines could have many more. You don't give us any indication of what system you're using, nor of the contents of most of the input fields. The awk utility and most editors are only defined to work on text files, and by definition, lines in a text file can't be longer than LINE_MAX bytes (including the terminating newline character). (Try:
to determine the value of LINE_MAX on your system. The standards only require that implementations support lines up to 2,048 bytes per line.) Are you sure that none of your output lines exceed LINE_MAX?
If you ask awk to print a line that is longer than LINE_MAX bytes long, the results are unspecified. If you use ed, ex, grep, sed, vi (or any of LOTS of other standard utilities that are described as processing text files) to read or write or create internal lines longer than LINE_MAX bytes long, the results are unspecified. There are very few standard text processing utilities that are defined to work on lines with arbitrary lengths (cut, fold, and paste).
Hello All,
I have a text file containing output from a command that contains lots of escape/control characters that when viewed using vi or view, looks like jibberish. But when viewed using the cat command the output is formatted properly.
Is there any way to take the output from the cat... (7 Replies)
Can someone please explain the strange behaviour.. I was just trying a few things to learn awk..
in the below code when I start the braces in the same line, the output is as expected, when I start at next line, output is displayed twice.
Please see the file, code I tried and output below.
... (2 Replies)
Hi,
Today I have found the following case in perl:
print "length:$lengths\tsum:". $count{$lengths}+$count_pair{$lengths}."\tindi:$count{$lengths}\t$count_pair{$lengths}\n";This give output as That means the first part of print is not printing. Only the values after the additions are printed.... (5 Replies)
Hi,
We have a problem where occasionally an ssh will hang for no apparent reason preventing the rest of the script continuing. To deal with this I am trying to write a wrapper script to kill a hung ssh command after a specified period.
The scripts use a sleep command running in the... (2 Replies)
Hello,
I was trying to identify lines who has a word of the following pattern "xyyx" (where x, and ys are different characters).
I was trying the following grep -
egrep '(\S)()\2\1'
This pattern do catches the wanted pattern, but it also catches "GGGG" or "CCCC" patterns. I was trying to... (5 Replies)
Hi All,
I got a strange problem here. I have a perl script which is fetching data from a database table and writing a file with that data.
If i run that script from linux command line, the file it creates is a normal ascii text file without any binary character in it.But... (9 Replies)
$ echo a.bc | sed -e "s/\|/\\|/g"
|a|.|b|c|
$
Is the behavior of the sed statement expected ? Or is this a bug in sed ?
OS details
Linux 2.6.9-55.0.0.0.2.ELsmp #1 SMP Wed May 2 14:59:56 PDT 2007 i686 i686 i386 GNU/Linux (8 Replies)
Can some-one give me a view to this :
I have a directory in an unix server, having permissions r-xr-xr-x .This directory is basically a source directory.
Now there is another directory basically the destination directory which has all the permissions.
Note:I log in as not the owner,but user... (5 Replies)