Sponsored Content
Top Forums Shell Programming and Scripting sort truncates line when they contain nulls Post 302187760 by era on Tuesday 22nd of April 2008 02:29:01 AM
Old 04-22-2008
Quote:
Originally Posted by massrobe
I can not change the byte because it is part of my data.
The idea is to change it temporarily so sort can work, then change it back. You just need to take care to use a byte which doesn't occur in your data.

For example, octal \200 or \001 might work if they don't occur in the data file already. So you'd change the NULs to (something unique), sort, and change (something unique) back to NUL. Now the data should be sorted, with the NULs preserved.

(\200 might be problematic too, because it's NUL with the eight bit set, and some procedure might still live in 7-bit land and strip the 8th bit internally; try some other high-value byte between \201 and \377 if it doesn't work.)
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

GREPing for Nulls

I just had a filesystem / file corruption issue on my HSP's server due to disk capacity limits and fileswapping. I discovered that certain files got corrupted when fileswapping was not successful and they ended up with a string of control characters, or what I believe to be nulls, in them. Does... (4 Replies)
Discussion started by: Dr. DOT
4 Replies

2. Shell Programming and Scripting

PS truncates the o/p

Hi , I have faced a strange situation in Solaris. the command ps -eo pid,args | grep 'SOMEPROCESS' truncates the output. outpt looks like 111 xxxxxxxxxxxxx SOMEPROCES 123 xxxxxxxxxxxxx SOMEPROCES 323 xxxxxxxxxxxxx SOMEPROCES The above doesn't return the complete command/args, infact if... (1 Reply)
Discussion started by: braindrain
1 Replies

3. UNIX for Advanced & Expert Users

who truncates the output? redirection? tty? Bug?

Hi, Output of running berkeley ps is truncated to 80 chars when using redirections. $ /usr/ucb/ps -e 12490|cat #truncated to 80 chars PID TT S TIME COMMAND 12490 pts/24 S 0:00 sleep 4000 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa getting longer lines is done by changing the stty $... (7 Replies)
Discussion started by: fredy
7 Replies

4. Shell Programming and Scripting

Sort a file line by line alphabetically

infile: z y x c b a desired output: x y z a b c I don't want to sort the lines into this: a b c x y z nor this: c b a z y x The number of fields per line and number of lines is indeterminate. The field separator is always a space. Thanks for the use of your collective brains.... (11 Replies)
Discussion started by: H2OBoodle
11 Replies

5. Programming

Blanks vs: Nulls

I'm relatively new to Pro*C programming. In the following example: char name; EXEC SQL SELECT 'John Doe' INTO :name FROM DUAL; "John Doe" is in positions 0-7, blanks in 8-19, and a null in 20. I would really prefer the null to be in position 8 and I don't care what's after that. I wrote a... (1 Reply)
Discussion started by: ebock
1 Replies

6. Shell Programming and Scripting

include NULLs in line length check

Hello, I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with... (10 Replies)
Discussion started by: ironmix
10 Replies

7. Shell Programming and Scripting

Sort a line and Insert sorted word(s) in a line

Hello, I am looking to automate a task - which is updating an existing access control instruction of a server and making sure that the attributes defined in the instruction is in sorted order. The instructions will be of a specific syntax. For example lets assume below listed is one of an... (6 Replies)
Discussion started by: sanjayroc
6 Replies

8. Shell Programming and Scripting

Replace nulls with a value in a file

Hi, I've a PIPE delimited file with about 5 fields. Sometimes the records in the 4th field is null, so I want to replace it based on values we get it on 2nd field in the same file. Following is an example. ABCD|X-TYPE 3.0|2010|X-TYPE|20000 CDEF|C-TYPE 2.5|2011|C-TYPE|10000 XYZ|LX... (4 Replies)
Discussion started by: rudoraj
4 Replies

9. Shell Programming and Scripting

/usr/bin/expect script truncates data

I have a script on a Linux machine that connects remotely, via telnet on a windows machine to launch several commands and colect their output. On the Linux machine the output of these commands is redirected in a file. The script: #!/usr/bin/expect log_user 0 spawn telnet 10.10.10.10... (6 Replies)
Discussion started by: black_fender
6 Replies
term(4) 						     Kernel Interfaces Manual							   term(4)

NAME
term - format of compiled term file SYNOPSIS
DESCRIPTION
Compiled terminfo descriptions are placed under the directory In order to avoid a linear search of a huge HP-UX system directory, a two- level scheme is used: where name is the name of the terminal, and c is the first character of name. Thus, can be found in the file Syn- onyms for the same terminal are implemented by multiple links to the same compiled file. The format has been chosen so that it is the same on all hardware. An 8-bit or longer byte is assumed, but no assumptions about byte ordering or sign extension are made. The compiled file is created using the program (see tic(1M)), and read by the routine. The file is divided into the following six parts: 1. The header section begins the file and contains six short integers in the following format: 1. Magic number (octal 0432); 2. Size, in bytes, of the names section; 3. Number of bytes in the Boolean section; 4. Number of short integers in the numbers section; 5. Number of offsets (short integers) in the strings section; 6. Size, in bytes, of the string table. Short integers are stored in two 8-bit bytes. The first byte contains the least significant 8 bits of the value; the second byte contains the most significant 8 bits. (Thus, the value represented is 256*second+first.) The value -1 is represented by other negative values are illegal. The -1 generally means that a capability is missing from this terminal. Note that this format corresponds to the hardware of the VAX and PDP-11. Machines where this does not correspond to the hardware read the integers as two bytes and compute the result. 2. The terminal names section comes next. It contains the first line of the terminfo description, listing the various names for the terminal, separated by the character. The section is terminated with an ASCII NUL character. 3. In the Boolean section, the Boolean flags have one byte for each flag. This byte is either or as the flag is absent or present, respectively. The capabilities are in the same order as they are listed in the file Between the Boolean section and the number section, a null byte will be inserted, if necessary, to ensure that the number section begins on an even byte. All short integers are aligned on a short word boundary. 4. The numbers section is similar to the flags section. Each capability consists of two bytes, and is stored as a short inte- ger. If the value represented is -1, the capability is considered missing. 5. The strings section is also similar. Each capability is stored as a short integer in the format above. A value of -1 means the capability is missing. Otherwise, the value is taken as an offset from the beginning of the string table. Special char- acters in or notation are stored in their interpreted form, not the printing representation. Padding information and parame- ter information are stored intact in uninterpreted form. 6. The final section is the string table. It contains all the values of string capabilities referenced in the string section. Each string is null terminated. Note that it is possible for to expect a different set of capabilities than are actually present in the file. Either the database might have been updated since has been recompiled (resulting in extra unrecognized entries in the file) or the program may have been recompiled more recently than the database was updated (resulting in missing entries). The routine must be prepared for both possibilities, which is why the numbers and sizes are included. Also, new capabilities must always be added at the end of the lists of Boolean, number, and string capabilities. The following example is an octal dump of the description for the HP Portable Computer (HP-110): 110|hp110|hp110a portable computer, am, xhp, da, db, mir, cols#80, lines#16, lm#0, cbt=Ei, bel=^G, cr= , tbc=E3, clear=E&a0y0CEJ, el=EK, ed=EJ, hpa=E&a%p1%dC, cup=E&a%p1%dy%p2%dC, cud1=EB, cub1=, cuf1=EC, cuu1=EA, cvvis=E&j@, dch1=EP, dl1=EM, smir=EQ, smso=E&dB, sgr0=E&d@, rmir=ER, rmso=E&d@, is2=E&j@, if=/usr/share/lib/tabset/stdcrt, il1=EL, kbs=, kcud1=EB, khome=Eh, kcub1=ED, kcuf1=EC, kcuu1=EA, rmkx=E&s0A, smkx=E&s1A, vpa=E&a%p1%dY, ind= , hts=E1, ht= , 0000 032 001 # 025  223 254 1 1 0 | 0020 h p 1 1 0 | h p 1 1 0 a p o r 0040 t a b l e c o m p u t e r 0060 001 001 001 001 001 0100 P 377 377 020 377 377 377 377 0120 377 377 377 377 003 005 377 377 007 0140 024 027 032 377 377 $ 4 377 377 377 377 0160 7 377 377 377 377 9 377 377 < ? D 0200 G 377 377 377 377 377 377 377 377 377 377 377 377 377 377 0220 377 377 J 377 377 377 377 377 377 M 377 377 377 377 0240 377 377 R 377 377 377 377 W Z 377 377 377 377 0260 377 377 377 377 377 377 _ 377 377 d 377 377 { 0300 377 377 ~ 377 377 377 377 377 377 377 377 377 377 200 0320 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 0340 377 377 377 377 377 377 377 377 377 377 377 377 203 377 377 0360 377 377 206 377 377 377 377 377 377 211 377 377 377 377 0400 377 377 214 217 225 377 377 377 377 377 377 377 377 0420 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 0520 377 377 233 377 377 245 377 377 377 377 247 377 377 0540 252 377 377 377 377 377 377 377 377 377 377 377 377 377 377 0560 377 377 377 377 377 377 377 377 377 377 033 i 007 0600 033 3 033 & a 0 y 0 C 033 J 033 K 0620 033 J 033 & a % p 1 % d C 033 & 0640 a % p 1 % d y % p 2 % d C 033 B 0660  033 C 033 A 033 & j @ 033 P 0700 033 M 033 Q 033 & d B 033 & d @ 0720 033 R 033 & d @ 033 & j @ / u 0740 s r / l i b / t a b s e t / s t 0760 d c r t 033 L  033 B 033 h 1000 033 D 033 C 033 A 033 & s 0 A 033 1020 & s 1 A 033 & a % p 1 % d Y 1040 033 1 1046 WARNINGS
Total compiled entries cannot exceed 4096 bytes. The name field cannot exceed 128 bytes. Hewlett-Packard Company supports only those terminals that are listed on the current list of supported devices. However, both non-sup- ported and supported terminals may be in the terminfo database. If non-supported terminals are used, they may not work correctly. FILES
compiled terminal capability data base SEE ALSO
tic(1M), untic(1M), terminfo(4). term(4)
All times are GMT -4. The time now is 05:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy