Sponsored Content
Top Forums Shell Programming and Scripting sort truncates line when they contain nulls Post 302187110 by era on Saturday 19th of April 2008 09:52:22 AM
Old 04-19-2008
If the files are pure 7-bit ASCII, you can replace the NUL with an extended character. Just make sure you don't pick one which already exists in the file. And make sure you don't use its UTF8 representation, which is by definition multiple bytes.

Or if you can find a 7-bit printable character which doesn't occur in the file. try that. (Tab? Tilde? Underscore? @?)

Code:
tr '\000' @ <file | sort | tr @ '\000' >output

... assuming your tr understands backslashed octal.

Grepping for special characters can be tricky, too; presumably, your grep will also treat NUL as end of string. Try replacing all occurrences of your character and comparing the result against the original; if they are binary identical, you have found a character which doesn't occur in the file.

Code:
 tr -d @ <file | cmp - file

... assuming your cmp accepts - to mean standard input.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

GREPing for Nulls

I just had a filesystem / file corruption issue on my HSP's server due to disk capacity limits and fileswapping. I discovered that certain files got corrupted when fileswapping was not successful and they ended up with a string of control characters, or what I believe to be nulls, in them. Does... (4 Replies)
Discussion started by: Dr. DOT
4 Replies

2. Shell Programming and Scripting

PS truncates the o/p

Hi , I have faced a strange situation in Solaris. the command ps -eo pid,args | grep 'SOMEPROCESS' truncates the output. outpt looks like 111 xxxxxxxxxxxxx SOMEPROCES 123 xxxxxxxxxxxxx SOMEPROCES 323 xxxxxxxxxxxxx SOMEPROCES The above doesn't return the complete command/args, infact if... (1 Reply)
Discussion started by: braindrain
1 Replies

3. UNIX for Advanced & Expert Users

who truncates the output? redirection? tty? Bug?

Hi, Output of running berkeley ps is truncated to 80 chars when using redirections. $ /usr/ucb/ps -e 12490|cat #truncated to 80 chars PID TT S TIME COMMAND 12490 pts/24 S 0:00 sleep 4000 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa getting longer lines is done by changing the stty $... (7 Replies)
Discussion started by: fredy
7 Replies

4. Shell Programming and Scripting

Sort a file line by line alphabetically

infile: z y x c b a desired output: x y z a b c I don't want to sort the lines into this: a b c x y z nor this: c b a z y x The number of fields per line and number of lines is indeterminate. The field separator is always a space. Thanks for the use of your collective brains.... (11 Replies)
Discussion started by: H2OBoodle
11 Replies

5. Programming

Blanks vs: Nulls

I'm relatively new to Pro*C programming. In the following example: char name; EXEC SQL SELECT 'John Doe' INTO :name FROM DUAL; "John Doe" is in positions 0-7, blanks in 8-19, and a null in 20. I would really prefer the null to be in position 8 and I don't care what's after that. I wrote a... (1 Reply)
Discussion started by: ebock
1 Replies

6. Shell Programming and Scripting

include NULLs in line length check

Hello, I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with... (10 Replies)
Discussion started by: ironmix
10 Replies

7. Shell Programming and Scripting

Sort a line and Insert sorted word(s) in a line

Hello, I am looking to automate a task - which is updating an existing access control instruction of a server and making sure that the attributes defined in the instruction is in sorted order. The instructions will be of a specific syntax. For example lets assume below listed is one of an... (6 Replies)
Discussion started by: sanjayroc
6 Replies

8. Shell Programming and Scripting

Replace nulls with a value in a file

Hi, I've a PIPE delimited file with about 5 fields. Sometimes the records in the 4th field is null, so I want to replace it based on values we get it on 2nd field in the same file. Following is an example. ABCD|X-TYPE 3.0|2010|X-TYPE|20000 CDEF|C-TYPE 2.5|2011|C-TYPE|10000 XYZ|LX... (4 Replies)
Discussion started by: rudoraj
4 Replies

9. Shell Programming and Scripting

/usr/bin/expect script truncates data

I have a script on a Linux machine that connects remotely, via telnet on a windows machine to launch several commands and colect their output. On the Linux machine the output of these commands is redirected in a file. The script: #!/usr/bin/expect log_user 0 spawn telnet 10.10.10.10... (6 Replies)
Discussion started by: black_fender
6 Replies
trbsd(1)						      General Commands Manual							  trbsd(1)

NAME
trbsd - Translates characters SYNOPSIS
trbsd [-Acs] string1 string2 trbsd -d [-Ac] string1 The trbsd command copies characters from the standard input to the standard output with substitution or deletion of selected characters. OPTIONS
Translates on a byte-by-byte basis. When you specify this option, trbsd does not support extended characters. Complements (inverts) the set of characters in string1 with respect to the universe of characters whose codes are 001 through 377 octal if you specify -A, and all characters if you do not specify -A. Deletes all characters in string1 from output. Changes characters that are repeated output charac- ters in string2 into single characters. DESCRIPTION
Input characters from string1 are replaced with the corresponding characters in string2. The trbsd command cannot handle an ASCII NUL (00) in string1 or string2; it always deletes NUL from the input. The tr command is a System V compatible version of trbsd. Abbreviations such as a-z, standing for a string of characters whose ASCII codes run from character a to character z, inclusive, can be used to introduce ranges of characters. Note that brackets are not special characters. Use the escape character (backslash) to remove the special meaning from any character in a string. Use the followed by 1, 2, or 3 octal digits for the code of a character. If a given character appears more than once in string1, the character in string2 corresponding to its last appearance in string1 will be used in the translation. EXAMPLES
To translate braces into parentheses, enter: trbsd '{}' '()' <textfile >newfile This translates each { (left brace) to a ( (left parenthesis) and each } (right brace) to a ) (right parenthesis). All other char- acters remain unchanged. To translate lowercase ASCII characters to uppercase, enter: trbsd a-z A-Z <textfile >newfile The two strings can be of different lengths: trbsd 0-9 # <textfile >newfile This translates each digit to a # (number sign); if string2 is too short, it is padded to the length of string1 by duplicating its last character. To translate each string of digits to a single # (number sign), enter: trbsd -s 0-9 # <textfile >newfile To trans- late all ASCII characters that are not specified, enter: trbsd -c ' -~' 'A-_' <textfile >newfile This translates each nonprinting ASCII character to the corresponding control key letter (01 translates to A, 02 to B, and so on). ASCII DEL (177), the character that follows ~ (tilde), translates to a ? (question mark). SEE ALSO
Commands: ed(1), sh(1), tr(1) Files: ascii(5) trbsd(1)
All times are GMT -4. The time now is 08:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy