Add a header to a sort file instruction


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Add a header to a sort file instruction
# 1  
Old 04-08-2009
Add a header to a sort file instruction

Hello,

I have a header which I have to add to a sorted file, however if I use
cat header sortedfile > newfile, the operation takes 2 minutes as the sorted file is over 400mb.

I have noticed that when I sort the 400mb unsorted file, this only takes 14 seconds to create the output.

As the creation of the whole file is extremely time critical, I am obliged to find the quickest way.

I thought it might be quicker to cat the file to the sort command to create the final file so I tried this (actual command)

Code:
cat /applis/projets/COP_DEV/data/HMC/fichseq/ee1ESP0LOT16AUX_01_177_0_DS.NEWHEADER | sort -T /applis/projets/COP_DEV/data/HMC/fichseq -k 1.5,1.27 -k 1.750n,1.759n -k 1.760n,1.761n -k 1.762n,1.765n /applis/projets/COP_DEV/data/HMC/fichseq/ee1ESP0LOT16AUX_01_177_0_DS.PRETRI > /applis/projets/COP_DEV/data/HMC/fichseq/ee1ESP0LOT16AUX_01_177_0_DS.POSTTRI

but I dont get the header in the file.

Questions
Will cat to the sort command actually work ?
Will the header remain unsorted.

Alternatively, I can put the header in the file and sort the whole file. Is there any way to sort the file in situ so that a new file is not created ?

TIA

Colin
# 2  
Old 04-08-2009
Quote:
cat header sortedfile > newfile, the operation takes 2 minutes as the sorted file is over 400mb.

I have noticed that when I sort the 400mb unsorted file, this only takes 14 seconds to create the output.
Did that 14 second sort write everything to disk too by redirection or just output to your screen? Question might sound dumb, but just the be sure that it is written to disk too.

You could try if
Code:
cp header.orig header       #just to save your header for future use
cat sortedfile >> header

is faster than
Code:
cat header sortedfile > newfile

# 3  
Old 04-08-2009
Hello,

I finally went for the following

Code:
HEADER=` head -1 /applis/projets/COP_DEV/data/HMC/fichseq/ee1ESP0LOT16AUX_01_177_0_DS.NEWHEADER`
sed -i "1i$HEADER" /applis/projets/COP_DEV/data/HMC/fichseq/ee1ESP0LOT16AUX_01_177_0_DS

Thanks for all your comments
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

2. UNIX for Dummies Questions & Answers

Sort a las file keep the header as it is

I have several las files with a header and each file start Version and text and before the data starts end up with ~Ascii, then the numbers starts: ------------------------------------------------------------------------- ~Version .....text.... ~Ascii 2 abc 230 1 name 1 abc 400 1... (17 Replies)
Discussion started by: tk2000
17 Replies

3. UNIX for Dummies Questions & Answers

Sort a las file keep the header as it is

I have several las files with a header and each file start Version and text and before the data starts end up with ~Ascii, then the numbers starts: ------------------------------------------------------------------------- Code: ~Version .....text.... ~Ascii 2 abc 230 1 name 1 abc ... (1 Reply)
Discussion started by: tk2000
1 Replies

4. UNIX for Dummies Questions & Answers

How can i sort a .txt file without loosing the header information?

Hi, I'm trying to sort 2 different .txt tab delimited files with the command line: sort -k 1b,1 inputfile > outputfile But doing that i'm also sorting the header (that ends at the end of my file). How can i sort a .txt file without sorting the header but conserving the header in the... (3 Replies)
Discussion started by: alisrpp
3 Replies

5. Shell Programming and Scripting

Add column header and row header

Hi, I have an input like this 1 2 3 4 2 3 4 5 4 5 6 7 I would like to count the no. of columns and print a header with a prefix "Col". I would also like to count the no. of rows and print as first column with each line number with a prefix "Row" So, my output would be ... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

add header in each file of folder

My files location Dir=/pp/Output/Test/ I have 100 .txt files on above location. I want add header "pt all" on each file. Sample Text file Hi Kalol Jt all q Output i want lt all Kalol Jt all (2 Replies)
Discussion started by: asavaliya
2 Replies

7. UNIX for Dummies Questions & Answers

Sort a tab file with header.

How to sort a tab delimited file first on col1 and then on col2. Also I need to keep the header intact. file.txt val1 val2 val3 val4 a b c d m n o p e f g h i j k l ... (3 Replies)
Discussion started by: mary271
3 Replies

8. Shell Programming and Scripting

sort a report file having header and footer

I am having report file with header and footer . The details in between header and footer are separated by a pipe charater. I want to sort the file by considering multiple columns in between header and footer. pls help (4 Replies)
Discussion started by: suryanarayana
4 Replies

9. Shell Programming and Scripting

Ignore Header and Footer and Sort the data in fixed width file

Hi Experts, I want to Sort the data in fixed width file where i have Header and Footer also in file. I m using below commad to do the sort based on field satarting from 15 position to 17 position , but it is not ignoring the Header and Footer of the file while sorting. In the output i am... (5 Replies)
Discussion started by: sasikari
5 Replies

10. UNIX for Dummies Questions & Answers

Sort and uniq lines of a file while keeping a header line

So, I have a file that has some duplicate lines. The file has a header line that I would like to keep at the top. I could do this by extracting the header from the file, 'sort -u' the remaining lines, and recombine them. But they are quite big, so if there is a way to do it with a single... (1 Reply)
Discussion started by: Digby
1 Replies
Login or Register to Ask a Question