Sponsored Content
Top Forums Shell Programming and Scripting sort truncates line when they contain nulls Post 302187110 by era on Saturday 19th of April 2008 09:52:22 AM
Old 04-19-2008
If the files are pure 7-bit ASCII, you can replace the NUL with an extended character. Just make sure you don't pick one which already exists in the file. And make sure you don't use its UTF8 representation, which is by definition multiple bytes.

Or if you can find a 7-bit printable character which doesn't occur in the file. try that. (Tab? Tilde? Underscore? @?)

Code:
tr '\000' @ <file | sort | tr @ '\000' >output

... assuming your tr understands backslashed octal.

Grepping for special characters can be tricky, too; presumably, your grep will also treat NUL as end of string. Try replacing all occurrences of your character and comparing the result against the original; if they are binary identical, you have found a character which doesn't occur in the file.

Code:
 tr -d @ <file | cmp - file

... assuming your cmp accepts - to mean standard input.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

GREPing for Nulls

I just had a filesystem / file corruption issue on my HSP's server due to disk capacity limits and fileswapping. I discovered that certain files got corrupted when fileswapping was not successful and they ended up with a string of control characters, or what I believe to be nulls, in them. Does... (4 Replies)
Discussion started by: Dr. DOT
4 Replies

2. Shell Programming and Scripting

PS truncates the o/p

Hi , I have faced a strange situation in Solaris. the command ps -eo pid,args | grep 'SOMEPROCESS' truncates the output. outpt looks like 111 xxxxxxxxxxxxx SOMEPROCES 123 xxxxxxxxxxxxx SOMEPROCES 323 xxxxxxxxxxxxx SOMEPROCES The above doesn't return the complete command/args, infact if... (1 Reply)
Discussion started by: braindrain
1 Replies

3. UNIX for Advanced & Expert Users

who truncates the output? redirection? tty? Bug?

Hi, Output of running berkeley ps is truncated to 80 chars when using redirections. $ /usr/ucb/ps -e 12490|cat #truncated to 80 chars PID TT S TIME COMMAND 12490 pts/24 S 0:00 sleep 4000 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa getting longer lines is done by changing the stty $... (7 Replies)
Discussion started by: fredy
7 Replies

4. Shell Programming and Scripting

Sort a file line by line alphabetically

infile: z y x c b a desired output: x y z a b c I don't want to sort the lines into this: a b c x y z nor this: c b a z y x The number of fields per line and number of lines is indeterminate. The field separator is always a space. Thanks for the use of your collective brains.... (11 Replies)
Discussion started by: H2OBoodle
11 Replies

5. Programming

Blanks vs: Nulls

I'm relatively new to Pro*C programming. In the following example: char name; EXEC SQL SELECT 'John Doe' INTO :name FROM DUAL; "John Doe" is in positions 0-7, blanks in 8-19, and a null in 20. I would really prefer the null to be in position 8 and I don't care what's after that. I wrote a... (1 Reply)
Discussion started by: ebock
1 Replies

6. Shell Programming and Scripting

include NULLs in line length check

Hello, I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with... (10 Replies)
Discussion started by: ironmix
10 Replies

7. Shell Programming and Scripting

Sort a line and Insert sorted word(s) in a line

Hello, I am looking to automate a task - which is updating an existing access control instruction of a server and making sure that the attributes defined in the instruction is in sorted order. The instructions will be of a specific syntax. For example lets assume below listed is one of an... (6 Replies)
Discussion started by: sanjayroc
6 Replies

8. Shell Programming and Scripting

Replace nulls with a value in a file

Hi, I've a PIPE delimited file with about 5 fields. Sometimes the records in the 4th field is null, so I want to replace it based on values we get it on 2nd field in the same file. Following is an example. ABCD|X-TYPE 3.0|2010|X-TYPE|20000 CDEF|C-TYPE 2.5|2011|C-TYPE|10000 XYZ|LX... (4 Replies)
Discussion started by: rudoraj
4 Replies

9. Shell Programming and Scripting

/usr/bin/expect script truncates data

I have a script on a Linux machine that connects remotely, via telnet on a windows machine to launch several commands and colect their output. On the Linux machine the output of these commands is redirected in a file. The script: #!/usr/bin/expect log_user 0 spawn telnet 10.10.10.10... (6 Replies)
Discussion started by: black_fender
6 Replies
vis(1)							      General Commands Manual							    vis(1)

NAME
vis, inv - make unprintable and non-ASCII characters in a file visible or invisible SYNOPSIS
file ... file ... DESCRIPTION
reads characters from each file in sequence and writes them to the standard output, converting those that are not printable or not ASCII into a visible form. inv performs the inverse function, reading printable characters from each file, returning them to non-printable or non-ASCII form, if appropriate, then writing them to standard output; Non-printable ASCII characters are represented using C-like escape conventions: backslash backspace escape form-feed new-line carriage return space horizontal tab vertical tab the character whose ASCII code is the 3-digit octal number n. the character whose ASCII code is the 2-digit hexadecimal number n. Non-ASCII single- or multi-byte characters are examined one byte at a time. For each byte, if it can be displayed as an ASCII character, it is treated as if it is an ASCII character; Otherwise, it is represented in the following conventions: the 8-bit character whose code value is the 3-digit octal number n. the 8-bit character whose code value is the 2-digit hexadecimal number n. Space, horizontal-tab, and new-line characters can be treated as printable (and therefore passed unaltered to the output) or non-printable depending on the options selected. Backslash, although printable, is expanded by vis, to a pair of backslashes so that when they are passed back through inv, they convert back to a single backslash. If no input file is given, or if the argument is encountered, and inv read from the standard input. Options and recognize the following options: Treat new-line, space, and horizontal tab as non-printable characters. expands them visibly as and rather than passing them directly to the output. discards these characters, expecting only the printable expansions. New-line characters are inserted by every 16 bytes so that the output will be in a form that is usable by most editors. Make and silent about non-existent files, identical input and output, and write errors. Normally, no input file can be the same as the output file unless it is a special file. Treat horizontal-tab and space characters as non-printable in the same manner that treats them. Cause output to be unbuffered (byte-by-byte); normally, output is buffered. Cause output to be in hexadecimal form rather than the default octal form. Either form is accepted to as input. EXTERNAL INFLUENCES
Environment Variables determines the language in which messages are displayed. International Code Set Support Single- and multi-byte character code sets are supported. WARNINGS
Redirecting output to an input file destroys the original data. Therefore, command forms such as should be avoided unless the source file can be safely discarded. AUTHOR
was developed by HP. SEE ALSO
cat(1), echo(1), od(1). vis(1)
All times are GMT -4. The time now is 08:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy