Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Strange behaviour when output to terminal vs file (awk) Post 302744999 by Don Cragun on Sunday 16th of December 2012 08:17:27 AM
Old 12-16-2012
Note that with 20 fields on each of 3,000 input records being combined into 200 output lines, your average output lines with have 300 fields and some lines could have many more. You don't give us any indication of what system you're using, nor of the contents of most of the input fields. The awk utility and most editors are only defined to work on text files, and by definition, lines in a text file can't be longer than LINE_MAX bytes (including the terminating newline character). (Try:
Code:
getconf LINE_MAX

to determine the value of LINE_MAX on your system. The standards only require that implementations support lines up to 2,048 bytes per line.) Are you sure that none of your output lines exceed LINE_MAX?

If you ask awk to print a line that is longer than LINE_MAX bytes long, the results are unspecified. If you use ed, ex, grep, sed, vi (or any of LOTS of other standard utilities that are described as processing text files) to read or write or create internal lines longer than LINE_MAX bytes long, the results are unspecified. There are very few standard text processing utilities that are defined to work on lines with arbitrary lengths (cut, fold, and paste).
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

A Strange Behaviour!!!

Can some-one give me a view to this : I have a directory in an unix server, having permissions r-xr-xr-x .This directory is basically a source directory. Now there is another directory basically the destination directory which has all the permissions. Note:I log in as not the owner,but user... (5 Replies)
Discussion started by: navojit dutta
5 Replies

2. UNIX for Advanced & Expert Users

Strange sed behaviour

$ echo a.bc | sed -e "s/\|/\\|/g" |a|.|b|c| $ Is the behavior of the sed statement expected ? Or is this a bug in sed ? OS details Linux 2.6.9-55.0.0.0.2.ELsmp #1 SMP Wed May 2 14:59:56 PDT 2007 i686 i686 i386 GNU/Linux (8 Replies)
Discussion started by: vino
8 Replies

3. UNIX for Dummies Questions & Answers

Strange Program behaviour

Had a strange thing going on with my code. It's ok I figured it out for myself.... (2 Replies)
Discussion started by: mrpugster
2 Replies

4. Shell Programming and Scripting

Strange behaviour with perl i/o?

Hi All, I got a strange problem here. I have a perl script which is fetching data from a database table and writing a file with that data. If i run that script from linux command line, the file it creates is a normal ascii text file without any binary character in it.But... (9 Replies)
Discussion started by: DILEEP410
9 Replies

5. Shell Programming and Scripting

Strange RegExp Behaviour

Hello, I was trying to identify lines who has a word of the following pattern "xyyx" (where x, and ys are different characters). I was trying the following grep - egrep '(\S)()\2\1' This pattern do catches the wanted pattern, but it also catches "GGGG" or "CCCC" patterns. I was trying to... (5 Replies)
Discussion started by: itskov
5 Replies

6. Shell Programming and Scripting

Strange terminal behaviour after killing ssh

Hi, We have a problem where occasionally an ssh will hang for no apparent reason preventing the rest of the script continuing. To deal with this I am trying to write a wrapper script to kill a hung ssh command after a specified period. The scripts use a sleep command running in the... (2 Replies)
Discussion started by: RECrerar
2 Replies

7. UNIX for Dummies Questions & Answers

Strange perl print output behaviour

Hi, Today I have found the following case in perl: print "length:$lengths\tsum:". $count{$lengths}+$count_pair{$lengths}."\tindi:$count{$lengths}\t$count_pair{$lengths}\n";This give output as That means the first part of print is not printing. Only the values after the additions are printed.... (5 Replies)
Discussion started by: gvj
5 Replies

8. Shell Programming and Scripting

Strange behaviour of arrays in awk

Imagine 2 files f1 f2: file1_l1_c1 code_to_find file1_l1_c3 file1_l2_c1 file1_code2 file1_l2_c3 file1_l3_c1 file1_code3 file1_l3_c3 file2_l1_c1 file2_l1_c2 code_to_find file2_l2_c1 file2_l2_c2 file2_code5 file2_l3_c1 file2_l3_c2 file2_code3 Say we want to print lines from f2 having... (5 Replies)
Discussion started by: ripat
5 Replies

9. Shell Programming and Scripting

awk Strange behaviour in AIX

Can someone please explain the strange behaviour.. I was just trying a few things to learn awk.. in the below code when I start the braces in the same line, the output is as expected, when I start at next line, output is displayed twice. Please see the file, code I tried and output below. ... (2 Replies)
Discussion started by: Kulasekar
2 Replies

10. Shell Programming and Scripting

Print Terminal Output Exactly how it Appears in the Terminal to a New Text File

Hello All, I have a text file containing output from a command that contains lots of escape/control characters that when viewed using vi or view, looks like jibberish. But when viewed using the cat command the output is formatted properly. Is there any way to take the output from the cat... (7 Replies)
Discussion started by: mrm5102
7 Replies
FOLD(1P)						     POSIX Programmer's Manual							  FOLD(1P)

PROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the correspond- ing Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux. NAME
fold - filter for folding lines SYNOPSIS
fold [-bs][-w width][file...] DESCRIPTION
The fold utility is a filter that shall fold lines from its input files, breaking the lines to have a maximum of width column positions (or bytes, if the -b option is specified). Lines shall be broken by the insertion of a <newline> such that each output line (referred to later in this section as a segment) is the maximum width possible that does not exceed the specified number of column positions (or bytes). A line shall not be broken in the middle of a character. The behavior is undefined if width is less than the number of columns any single character in the input would occupy. If the <carriage-return>s, <backspace>s, or <tab>s are encountered in the input, and the -b option is not specified, they shall be treated specially: <backspace> The current count of line width shall be decremented by one, although the count never shall become negative. The fold utility shall not insert a <newline> immediately before or after any <backspace>. <carriage-return> The current count of line width shall be set to zero. The fold utility shall not insert a <newline> immediately before or after any <carriage-return>. <tab> Each <tab> encountered shall advance the column position pointer to the next tab stop. Tab stops shall be at each column position n such that n modulo 8 equals 1. OPTIONS
The fold utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines. The following options shall be supported: -b Count width in bytes rather than column positions. -s If a segment of a line contains a <blank> within the first width column positions (or bytes), break the line after the last such <blank> meeting the width constraints. If there is no <blank> meeting the requirements, the -s option shall have no effect for that output segment of the input line. -w width Specify the maximum line length, in column positions (or bytes if -b is specified). The results are unspecified if width is not a positive decimal number. The default value shall be 80. OPERANDS
The following operand shall be supported: file A pathname of a text file to be folded. If no file operands are specified, the standard input shall be used. STDIN
The standard input shall be used only if no file operands are specified. See the INPUT FILES section. INPUT FILES
If the -b option is specified, the input files shall be text files except that the lines are not limited to {LINE_MAX} bytes in length. If the -b option is not specified, the input files shall be text files. ENVIRONMENT VARIABLES
The following environment variables shall affect the execution of fold: LANG Provide a default value for the internationalization variables that are unset or null. (See the Base Definitions volume of IEEE Std 1003.1-2001, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.) LC_ALL If set to a non-empty string value, override the values of all the other internationalization variables. LC_CTYPE Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files), and for the determination of the width in column positions each character would occupy on a constant-width font output device. LC_MESSAGES Determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error. NLSPATH Determine the location of message catalogs for the processing of LC_MESSAGES . ASYNCHRONOUS EVENTS
Default. STDOUT
The standard output shall be a file containing a sequence of characters whose order shall be preserved from the input files, possibly with inserted <newline>s. STDERR
The standard error shall be used only for diagnostic messages. OUTPUT FILES
None. EXTENDED DESCRIPTION
None. EXIT STATUS
The following exit values shall be returned: 0 All input files were processed successfully. >0 An error occurred. CONSEQUENCES OF ERRORS
Default. The following sections are informative. APPLICATION USAGE
The cut and fold utilities can be used to create text files out of files with arbitrary line lengths. The cut utility should be used when the number of lines (or records) needs to remain constant. The fold utility should be used when the contents of long lines need to be kept contiguous. The fold utility is frequently used to send text files to printers that truncate, rather than fold, lines wider than the printer is able to print (usually 80 or 132 column positions). EXAMPLES
An example invocation that submits a file of possibly long lines to the printer (under the assumption that the user knows the line width of the printer to be assigned by lp): fold -w 132 bigfile | lp RATIONALE
Although terminal input in canonical processing mode requires the erase character (frequently set to <backspace>) to erase the previous character (not byte or column position), terminal output is not buffered and is extremely difficult, if not impossible, to parse correctly; the interpretation depends entirely on the physical device that actually displays/prints/stores the output. In all known internationalized implementations, the utilities producing output for mixed column-width output assume that a <backspace> backs up one column position and outputs enough <backspace>s to return to the start of the character when <backspace> is used to provide local line motions to support underlining and emboldening operations. Since fold without the -b option is dealing with these same constraints, <backspace> is always treated as backing up one column position rather than backing up one character. Historical versions of the fold utility assumed 1 byte was one character and occupied one column position when written out. This is no longer always true. Since the most common usage of fold is believed to be folding long lines for output to limited-length output devices, this capability was preserved as the default case. The -b option was added so that applications could fold files with arbitrary length lines into text files that could then be processed by the standard utilities. Note that although the width for the -b option is in bytes, a line is never split in the middle of a character. (It is unspecified what happens if a width is specified that is too small to hold a sin- gle character found in the input followed by a <newline>.) The tab stops are hardcoded to be every eighth column to meet historical practice. No new method of specifying other tab stops was invented. FUTURE DIRECTIONS
None. SEE ALSO
cut COPYRIGHT
Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol- ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html . IEEE
/The Open Group 2003 FOLD(1P)
All times are GMT -4. The time now is 11:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy