Visit Our UNIX and Linux User Community


sort and split file by 2 cols (1 col after the other)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sort and split file by 2 cols (1 col after the other)
# 1  
Old 07-26-2009
sort and split file by 2 cols (1 col after the other)

Dear All,

I am a newbie to shell scripting so this one is really over my head.

I have a text file with five fields as below:

76576.867188 6232.454102 2.008904 55.000000 3
76576.867188 6232.454102 3.607231 55.000000 4
76576.867188 6232.454102 1.555146 65.000000 3
76576.867188 6232.454102 3.226928 65.000000 4
76576.867188 6232.454102 2.180096 100.000000 4
66576.867188 4232.454102 2.944000 0.000000 1
66576.867188 4232.454102 3.549000 0.000000 2
66576.867188 4232.454102 4.517000 0.000000 3
.......

I want to a bash script to sort and split the file using the 4th column and then further split each resulting file by the 5th column.

I tried:

#!/bin/bash

sort $1 | awk '{ file=substr($0,35,5)".tmp"; print >> file }'

for i in 'ls *.tmp'
do
awk '{ file=substr($0,44,1)".txt"; print >> file }'
done

but it is not working at all.

Could someone please point me in the right direction?

Thanks.
# 2  
Old 07-26-2009
What is the desired output?

To keep the forums high quality for all users, please take the time to format your posts correctly.

Use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.)
# 3  
Old 07-26-2009
You don't need a full script Smilie
Code:
sort -nk4 file | awk '{newfile=$NF".txt";print >  newfile}'

# 4  
Old 07-27-2009
To Franklin52 and All,

Sorry for not making myself clear in my earlier post.

I want output files like:

55.000 3.txt
76576.867188 6232.454102 2.008904 55.000000 3
...................................................55.000000 3

55.000 4.txt
76576.867188 6232.454102 3.607231 55.000000 4
...................................................55.000000 4

65.000000 3.txt
76576.867188 6232.454102 1.555146 65.000000 3
...................................................65.000000 3

65.000000 4.txt
76576.867188 6232.454102 3.226928 65.000000 4
...................................................65.000000 4

etc.

I am however trying out danmero's sugggestion.

Thanks

---------- Post updated at 11:20 AM ---------- Previous update was at 09:52 AM ----------

Hi All

If I keep the 4th field at the same length as below:

76576.867188 6232.454102 2.008904 55.00 3
76576.867188 6232.454102 3.607231 55.00 4
76576.867188 6232.454102 1.555146 65.00 3
76576.867188 6232.454102 3.226928 65.00 4
76576.867188 6232.454102 2.180096 100.0 4
66576.867188 4232.454102 2.944000 0.000 1
66576.867188 4232.454102 3.549000 0.000 2
66576.867188 4232.454102 4.517000 0.000 3

then this:

Code:
sort -nk4 myfile | awk '{ file=substr($0,35,7)".txt"; print >> file }'

works.

But I cannot figure it out for variable 4th field length.

danmero,

I get this error with your code:

".txt' <No such file or directory>n't redirect to '1"
# 5  
Old 07-27-2009
Avoid spaces in file names, try this:

Code:
sort -nk4 myfile | awk '{ print > $4 "_" $5 ".txt" }'

# 6  
Old 07-27-2009
Quote:
Originally Posted by Ghetz
But I cannot figure it out for variable 4th field length.
The question is why you need the 4th field length to archive your goal?
Take a closer look at my and Franklin52 solutions.
# 7  
Old 07-27-2009
Franklin
I get an error message similar to danmero's with your code:

Quote:
.txt' <No such file or directory>n't redirect to '0.000_1
danmero

I don't need the length of the 4th field per se.
I focused on it out of fustration I guess.

I am still trying to get your code or franklin52's to work.

Previous Thread | Next Thread
Test Your Knowledge in Computers #271
Difficulty: Easy
While working at Bletchley Park, in the first week of June each year Alan Turing would get a bad attack of hay fever, and he would cycle to the office wearing a service gas mask to keep from breathing pollen. In the first week of June each year he would get a bad attack of hay fever, and he would cycle to the office wearing a service gas mask to keep from breathing the pollen.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sort files to the split second

I need a similar sort, I need to sort the directory by time, but time needs to be to the second or smaller. Using ls -lt but that is only to the minute. (3 Replies)
Discussion started by: mwlaursen
3 Replies

2. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

3. Shell Programming and Scripting

Modifying col values based on another col

Hi, Please help with this. I have several excel files (with and .xlsx format) with 10-15 columns each. They all have the same type of data but the columns are not ordered in the same way. Here is a 3 column example. What I want to do add the alphabet from column 2 to column 3, provided... (9 Replies)
Discussion started by: newbie83
9 Replies

4. Shell Programming and Scripting

Printing from col x to end of line, except last col

Hello, I have some tab delimited data and I need to move the last col. I could hard code it, awk '{ print $1,$NF,$2,$3,$4,etc }' infile > outfile but it would be nice to know the syntax to print a range cols. I know in cut you can do, cut -f 1,4-8,11- to print fields 1,... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

5. Shell Programming and Scripting

how to Insert values in multiple lines(records) within a pipe delimited text file in specific cols

this is Korn shell unix. The scenario is I have a pipe delimited text file which needs to be customized. say for example,I have a pipe delimited text file with 15 columns(| delimited) and 200 rows. currently the 11th and 12th column has null values for all the records(there are other null columns... (4 Replies)
Discussion started by: vasan2815
4 Replies

6. UNIX for Advanced & Expert Users

Print line based on highest value of col (B) and repetion of values in col (A)

Hello everyone, I am writing a script to process data from the ATP world tour. I have a file which contains: t=540 y=2011 r=1 p=N409 t=540 y=2011 r=2 p=N409 t=540 y=2011 r=3 p=N409 t=540 y=2011 r=4 p=N409 t=520 y=2011 r=1 p=N409 t=520 y=2011 r=2 p=N409 t=520 y=2011 r=3 p=N409 The... (4 Replies)
Discussion started by: imahmoud
4 Replies

7. Ubuntu

Match col 1 of File 1 with col 1 File 2 and create a 3rd file

Hello, I have a 1.6 GB file that I would like to modify by matching some ids in col1 with the ids in col 1 of file2.txt and save the results into a 3rd file. For example: File 1 has 1411 rows, I ignore how many columns it has (thousands) File 2 has 311 rows, 1 column Would like to... (7 Replies)
Discussion started by: sogi
7 Replies

8. Shell Programming and Scripting

How to find number of Cols in a file ?

Hi I have a requirement wherein the file is comma separated. Each records seems to have different number of columns, how I can detect like a row index wise, how many columns are present ? Thanks in advance. (2 Replies)
Discussion started by: videsh77
2 Replies

9. Shell Programming and Scripting

Sort & Split records in a file

Hi, I am new to scripting. I need a script to sort and the records in a file and then split them into different files. For example, the file is: H1...................... H2...................... D2.................... D2.................... H1........................... (15 Replies)
Discussion started by: Sunitha_edi82
15 Replies

10. Shell Programming and Scripting

join cols from multi files into one file

Hi Fields in Files 1,2,3,4 are pipe"|" separated. Say I want to grep col1 from File1 col3 from File2 col4 from File3 and print to File4 in the following order: col3|col1|col4 what is the best way of doing this? Thanks (2 Replies)
Discussion started by: vbshuru
2 Replies

Featured Tech Videos