Sponsored Content
Top Forums Shell Programming and Scripting sort truncates line when they contain nulls Post 302187110 by era on Saturday 19th of April 2008 09:52:22 AM
Old 04-19-2008
If the files are pure 7-bit ASCII, you can replace the NUL with an extended character. Just make sure you don't pick one which already exists in the file. And make sure you don't use its UTF8 representation, which is by definition multiple bytes.

Or if you can find a 7-bit printable character which doesn't occur in the file. try that. (Tab? Tilde? Underscore? @?)

Code:
tr '\000' @ <file | sort | tr @ '\000' >output

... assuming your tr understands backslashed octal.

Grepping for special characters can be tricky, too; presumably, your grep will also treat NUL as end of string. Try replacing all occurrences of your character and comparing the result against the original; if they are binary identical, you have found a character which doesn't occur in the file.

Code:
 tr -d @ <file | cmp - file

... assuming your cmp accepts - to mean standard input.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

GREPing for Nulls

I just had a filesystem / file corruption issue on my HSP's server due to disk capacity limits and fileswapping. I discovered that certain files got corrupted when fileswapping was not successful and they ended up with a string of control characters, or what I believe to be nulls, in them. Does... (4 Replies)
Discussion started by: Dr. DOT
4 Replies

2. Shell Programming and Scripting

PS truncates the o/p

Hi , I have faced a strange situation in Solaris. the command ps -eo pid,args | grep 'SOMEPROCESS' truncates the output. outpt looks like 111 xxxxxxxxxxxxx SOMEPROCES 123 xxxxxxxxxxxxx SOMEPROCES 323 xxxxxxxxxxxxx SOMEPROCES The above doesn't return the complete command/args, infact if... (1 Reply)
Discussion started by: braindrain
1 Replies

3. UNIX for Advanced & Expert Users

who truncates the output? redirection? tty? Bug?

Hi, Output of running berkeley ps is truncated to 80 chars when using redirections. $ /usr/ucb/ps -e 12490|cat #truncated to 80 chars PID TT S TIME COMMAND 12490 pts/24 S 0:00 sleep 4000 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa getting longer lines is done by changing the stty $... (7 Replies)
Discussion started by: fredy
7 Replies

4. Shell Programming and Scripting

Sort a file line by line alphabetically

infile: z y x c b a desired output: x y z a b c I don't want to sort the lines into this: a b c x y z nor this: c b a z y x The number of fields per line and number of lines is indeterminate. The field separator is always a space. Thanks for the use of your collective brains.... (11 Replies)
Discussion started by: H2OBoodle
11 Replies

5. Programming

Blanks vs: Nulls

I'm relatively new to Pro*C programming. In the following example: char name; EXEC SQL SELECT 'John Doe' INTO :name FROM DUAL; "John Doe" is in positions 0-7, blanks in 8-19, and a null in 20. I would really prefer the null to be in position 8 and I don't care what's after that. I wrote a... (1 Reply)
Discussion started by: ebock
1 Replies

6. Shell Programming and Scripting

include NULLs in line length check

Hello, I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with... (10 Replies)
Discussion started by: ironmix
10 Replies

7. Shell Programming and Scripting

Sort a line and Insert sorted word(s) in a line

Hello, I am looking to automate a task - which is updating an existing access control instruction of a server and making sure that the attributes defined in the instruction is in sorted order. The instructions will be of a specific syntax. For example lets assume below listed is one of an... (6 Replies)
Discussion started by: sanjayroc
6 Replies

8. Shell Programming and Scripting

Replace nulls with a value in a file

Hi, I've a PIPE delimited file with about 5 fields. Sometimes the records in the 4th field is null, so I want to replace it based on values we get it on 2nd field in the same file. Following is an example. ABCD|X-TYPE 3.0|2010|X-TYPE|20000 CDEF|C-TYPE 2.5|2011|C-TYPE|10000 XYZ|LX... (4 Replies)
Discussion started by: rudoraj
4 Replies

9. Shell Programming and Scripting

/usr/bin/expect script truncates data

I have a script on a Linux machine that connects remotely, via telnet on a windows machine to launch several commands and colect their output. On the Linux machine the output of these commands is redirected in a file. The script: #!/usr/bin/expect log_user 0 spawn telnet 10.10.10.10... (6 Replies)
Discussion started by: black_fender
6 Replies
TR(1)							      General Commands Manual							     TR(1)

NAME
tr - translate characters SYNOPSIS
tr [ -cds ] [ string1 [ string2 ] ] DESCRIPTION
Tr copies the standard input to the standard output with substitution or deletion of selected characters. Input characters found in string1 are mapped into the corresponding characters of string2. When string2 is short it is padded to the length of string1 by duplicat- ing its last character. Any combination of the options -cds may be used: -c complements the set of characters in string1 with respect to the universe of characters whose ASCII codes are 01 through 0377 octal; -d deletes all input characters in string1; -s squeezes all strings of repeated output characters that are in string2 to single characters. In either string the notation a-b means a range of characters from a to b in increasing ASCII order. The character `' followed by 1, 2 or 3 octal digits stands for the character whose ASCII code is given by those digits. A `' followed by any other character stands for that character. The following example creates a list of all the words in `file1' one per line in `file2', where a word is taken to be a maximal string of alphabetics. The second string is quoted to protect `' from the Shell. 012 is the ASCII code for newline. tr -cs A-Za-z '12' <file1 >file2 SEE ALSO
ed(1), ascii(7), expand(1) BUGS
Won't handle ASCII NUL in string1 or string2; always deletes NUL from input. 7th Edition April 29, 1985 TR(1)
All times are GMT -4. The time now is 07:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy