Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

HELP: I need to sort a text file in an uncommon manner, can't get desired results

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting HELP: I need to sort a text file in an uncommon manner, can't get desired results
# 1  
Old 11-04-2009
HELP: I need to sort a text file in an uncommon manner, can't get desired results
Hi All

I have a flat text file. Each line in it contains a "/full path/filename". The last three columns are predictable, but directory depth of each line varies.

I want to sort on the last three columns, starting from the last, 2nd last and 3rd last. In that order. The last three columns have a constant column width:

Code:
/../...../.../2009/365/2300.Z

Last col(6): 24h time 0000.Z -> 2300.Z (compressed file)
2nd Last col(3}: Day of Year, 001 -> 365 (366 in leap years)
3rd Last col(4): Year
I'll explain with an example:

- desired sort key: last col, 2nd last col, 3rd last col

Example: (unsorted)

Code:
cat dirList.txt

/home/jake/quarterly/2009/307/1300.Z
/home/jake/quarterly/2009/303/1400.Z
/home/jake/bimonthly/submitted/2009/007/1800.Z
/home/jake/yearly/2009/199/2300.Z

As you can see, the number of columns in this example vary, however, regardless the number of columns, I would like to sort the lines based on last, 2nd last & 3rd last columns, the results should look like this:

Code:
cat dirList.txt | unix | magic | commands | here

/home/jake/bimonthly/submitted/2009/007/1800.Z
/home/jake/yearly/2009/199/2300.Z
/home/jake/quarterly/2009/303/1400.Z
/home/jake/quarterly/2009/307/1300.Z

Perhaps an awk + sort command line combo will do the trick, but I'm Google'd out on searching for an answer.

Any help would be greatly appreciated.

Tks,
Jake.
Last edited by pludi; 11-04-2009 at 12:04 PM.. Reason: code tags, please...
# 2  
Old 11-04-2009
This should do it:
Code:
bash-3.2$ cat dirlist.txt
/home/jake/quarterly/2009/307/1300.Z
/home/jake/quarterly/2009/303/1400.Z
/home/jake/bimonthly/submitted/2009/007/1800.Z
/home/jake/yearly/2009/199/2300.Z
bash-3.2$
bash-3.2$
bash-3.2$ perl -le 'print sort { @a=($a=~m#(\d{4})/(\d{3})/(\d{4})\.Z$#);
@b=($b=~m#(\d{4})/(\d{3})/(\d{4})\.Z$#);
$a[0].$a[1].$a[2] <=> $b[0].$b[1].$b[2] } <>' dirlist.txt
/home/jake/bimonthly/submitted/2009/007/1800.Z
/home/jake/yearly/2009/199/2300.Z
/home/jake/quarterly/2009/303/1400.Z
/home/jake/quarterly/2009/307/1300.Z

# 3  
Old 11-04-2009
sed bonanza: Smilie
Code:
sed 's|.*\(/[^/]\+/[^/]\+/[^/]\+\)|\1 &|' dirlist.txt| sort -nt'/'|sed 's|[^ ]* ||'

Code:
/home/jake/bimonthly/submitted/2009/007/1800.Z
/home/jake/yearly/2009/199/2300.Z
/home/jake/quarterly/2009/303/1400.Z
/home/jake/quarterly/2009/307/1300.Z

# 4  
Old 11-04-2009
pludi:

Awesome, that did it. I keep forgetting the valuable magic (and complexity) of perl.

A thousand thank-you's.

J.

Scrutinizer:

The 'sed' command example did sort from low day of year to highest in each sub-directory. However, I blame my poor wording of my original question. What I needed in the output was a total sort of all file names in a quasi-chronological order (based on filename), oldest first so the output would contain the oldest files, regardless of subdirectory depth, at the top of the file.

Thank you very much for your answer!

J.
Last edited by JakeKatz; 11-04-2009 at 01:00 PM.. Reason: Clarifying my answer
# 5  
Old 11-04-2009
Hi Jake, that is very gentle since you really wrote that quite clearly Smilie. Anyway just for completeness then:
Code:
sed 's|.*\(/[^/]\+\)\(/[^/]\+\)\(/[^/]\+\)|\3\2\1 &|' dirlist.txt| sort -nt'/'|sed 's|[^ ]* ||'

# 6  
Old 11-04-2009
You can do it with gawk:
Code:
WHINY_USERS=1 gawk -F/ '{a[$(NF-2),$(NF-1),$NF]=$0}END{for(i in a)print a[i]}'

or
Code:
gawk -F/ '{a[$(NF-2), $(NF-1), $NF] = $0}
END { n = asorti(a, b); for (i = 1; i <= n; ++i) print a[b[i]]}'

# 7  
Old 11-05-2009
Excellent!

That did exactly what I needed.

And I just remember mom saying "Never bite the hand that feeds you" Smilie

Thanks for your help!

J.

Quote:
Originally Posted by Scrutinizer
Hi Jake, that is very gentle since you really wrote that quite clearly Smilie. Anyway just for completeness then:
Code:
sed 's|.*\(/[^/]\+\)\(/[^/]\+\)\(/[^/]\+\)|\3\2\1 &|' dirlist.txt| sort -nt'/'|sed 's|[^ ]* ||'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create a text file and a pdf file from Linux command results.

Hello. The task : Using multiple commands like : gdisk -l $SOME_DISK >> $SOME_FILEI generate some text file. For readiness I must insert page break. When the program is finished I want to convert the final text file to a pdf file. When finished, I got two files : One text file and One pdf... (1 Reply)
Discussion started by: jcdole
1 Replies

2. Shell Programming and Scripting

Needed shell script to append desired text to each line in a file

Hi, I had generated a report in my tool as followsoutput.txt 43.35 9 i needed the script to generate a new file like below i want to append the text to each of these lines of my filenewoutputfile.txt should be Total Amount : 43.35 Record Count:9 Regards, Vasa Saikumar. ... (2 Replies)
Discussion started by: hemanthsaikumar
2 Replies

3. Shell Programming and Scripting

How to sort and compare files in more efficient manner?

Hello All, Iam using below method to sort and compare files. First iam doing sorting and changing the same file and then doing comparing and taking the final result to another file. sort -o temp.txt file1 mv temp.txt file1 sort -o temp.txt file2 mv temp.txt file2 sort -o temp.txt... (6 Replies)
Discussion started by: Vikram_Tanwar12
6 Replies

4. UNIX for Dummies Questions & Answers

[Solved] Slight variation from the desired results

Hello, I am writing a small one liner script to display the tables in my database. I am working with Centos 5.5 and postgresql the command is "psql -c "\dt" | awk '{print$3}'" I just want the 3rd column from the result set, but now the problem is I am getting the third column but with... (3 Replies)
Discussion started by: nnani
3 Replies

5. Shell Programming and Scripting

Cron job is not working in the desired manner on IBM AIX

Hi, I have created a cron job on IBM AIX but it is not working in desired manner ! Here are the steps which I have followed :- #!/bin/ksh #------------------------------------------------------------------ find /some/file/at/the/user/side/test.log -exec cp {}... (8 Replies)
Discussion started by: acidburn_007
8 Replies

6. Shell Programming and Scripting

comparing 2 files and creating third file with uncommon content

I want to compare 2 files and create third file with uncommon content. e.g. file1 ajay suhas tom nisha vijay mahish file2 ajay suhas tom nisha expected output file content vijay mahish Is it possible in single command ? Thanks, Ajay (6 Replies)
Discussion started by: ajaypatil_am
6 Replies

7. Shell Programming and Scripting

Sed in vi - \r and \n not giving desired results

I use many different machines at work, each with different versions of o/s's and installed applications. Sed in vi is particularly inconvenient in the sense that sometimes it will accept the "\r" as a carriage return, sometimes not. Same thing with "\n". For instance, if I have a list of hosts... (7 Replies)
Discussion started by: MaindotC
7 Replies

8. Shell Programming and Scripting

Print some results in a text file using script in linux

hello everyone, i really need your help to write a script which would just print following kind of result into a text file (result.txt) XYZ test Results ID: <unique-id> Date: <date> ------------------------------------------------- | Task | Result | Time |... (3 Replies)
Discussion started by: viriimind
3 Replies

9. Shell Programming and Scripting

Helping in parsing subset of text from a big results file

Hi All, I need some help to effectively parse out a subset of results from a big results file. Below is an example of the text file. Each block that I need to parse starts with "reading sequence file 10.codon" (next block starts with another number) and ends with **p-Value(s)**. I have given... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

10. Shell Programming and Scripting

egrep not giving desired results

I have written a shell script which looks like below: grep -v ',0,' ./DATA/abc.001 > ./DATA/abc.mid egrep $GREPSEARCH ./DATA/ebc.mid > ./DATA/abc.cut the variable GREPSEARCH has values like the below: ... (3 Replies)
Discussion started by: igandu
3 Replies
Login or Register to Ask a Question