Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sort file specifying record length

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sort file specifying record length
# 1  
Old 10-10-2011
sort file specifying record length
I've been searching high and low for this...but, maybe I'm just missing something. I have a file to be sorted that, unfortunately, contains binary data at the end of the line. As you may guess, this binary data may contain a newline character, which messes up the sort. I think I could resolve this by letting 'sort' know the record length (line length) of each line, but I can't seem to find out how to do that. Is there a way to do this? If so, can someone point me the way?

Thanks in advance!!
# 2  
Old 10-10-2011
There's nothing you can tell sort that would make the situation workable. The problem is that sort is a line-oriented tool. If there is a newline-valued byte in the binary data, sort will treat the binary data before and after that byte as individual lines.

You are going to need some way to pre-process the input before feeding it to sort, so that it can do its job properly.

If you'd like further help, we'll need to know as much as possible about the characteristics of the input data (including the always annoying-when-not-mentioned special cases).

Regards and welcome to the forum,
Alister
# 3  
Old 10-10-2011
I think I can work it to use some perl...I was just hoping 'sort' could do it somehow. Thanks for your quick response!
# 4  
Old 10-10-2011
Just a thought (without more info) but have you tried using strings then pipe that to sort.
# 5  
Old 10-10-2011
Just tried it...it appears to do that same thing. Thanks for the idea, however!
# 6  
Old 10-11-2011
GNU sort has -z option which would delimit records with nulls instead of new lines, but I guess it won't help in this case.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. Shell Programming and Scripting

Record length

Hi, The record length may be differ in afile. I want to display the records if the record length is not equal to 50 using sed/awk command. Thanks in Advance (6 Replies)
Discussion started by: NareshN
6 Replies

3. Shell Programming and Scripting

Add substring in a file containing fixed length record.

I am new to awk and writing a script using awk. I have file containing fixed length records, I wish to extract 2 substring(each substring is padded with zeros on left e.g 000000003623) and add each substring respectively for every record in the file to get total sum of respective substring for all... (5 Replies)
Discussion started by: Devesh5683
5 Replies

4. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

5. Shell Programming and Scripting

search and replace fixed length record file

Hi I need to be search a file of fixed length records and when I hit a particular record that match a search string, substitute a known position field In the example file below FHEAD000000000120090806143011 THEAD0000000002Y0000000012 P00000000000000001234 TTAIL0000000003... (0 Replies)
Discussion started by: nedkelly007
0 Replies

6. Shell Programming and Scripting

Need a sort solution for fixed length file

I have a 1250 byte record that I need to sort in column 10-19 and in column 301. I have tried the sort command, but it looks like it needs delimiters to work. The record can have spaces in a lot of its 1250 columns, but 10-19, and 301 are guaranteed. These columns are numeric too. A sample... (1 Reply)
Discussion started by: mb1201
1 Replies

7. UNIX for Dummies Questions & Answers

What the command to find out the record length of a fixed length file?

I want to find out the record length of a fixed length file? I forgot the command. Any body know? (9 Replies)
Discussion started by: tranq01
9 Replies

8. UNIX for Dummies Questions & Answers

Creating new file with new record length

Hi there, using the command below > cat file1 | tr "\001" " " > tmp2 how can I alter the record length of tmp2 to fix length of 350 bytes ? Thanks a lot! (7 Replies)
Discussion started by: mrjunsy
7 Replies

9. Shell Programming and Scripting

Pivot variable record length file and change delimiter

Hi experts. I got a file (500mb max) and need to pivot it (loading into ORCL) and change BLANK delimiter to PIPE |. Sometimes there are multipel BLANKS (as a particular value may be BLANK, or simply two BLANKS instead of one BLANK). thanks for your input! Cheers, Layout... (3 Replies)
Discussion started by: thomasr
3 Replies

10. Shell Programming and Scripting

fixed record length

hello! I have a file with fixed record length... format: 123445asdfg 4343777 sfgg I wanna convert it to 123445,asdfg ,4343,777 ,sfgg is there any way to do it? sed/grep/awk?? at the moment I use sed -e 's_ \(\)_,\1_g' but it works only if there are spaces between... (16 Replies)
Discussion started by: george_
16 Replies
Login or Register to Ask a Question