Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sorting files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sorting files
# 8  
Old 11-07-2011
Quote:
Originally Posted by radoulov
Code:
printf '%s\n' *.log | 
  sort -t- -k6n | 
    awk -F- 'NR > 1 { 
      _[$5,$6]++ || $0 = RS $0 
       }1'

Amazing!... I really like the way you code, it is sooo compact!...

--ahamed
# 9  
Old 11-07-2011
Quote:
Originally Posted by kristinu
[...]
It would be helpful to understand what the following is doing if it's not too much bother.

Code:
_[$5,$6]++ || $0 = RS $0

I know it is trying to do matching of some kind, but not much idea what.

This part of the script creates an associative array (named _),
keyed by the combination of the values of the fifth and the sixth fields
(based on the current field separator which in this case is set to a single dash (-)),
the values of the associative array are auto-incremented integers.
This example should make it clear:

This is the output of the sort command:

Code:
zsh-4.3.12[t]% printf '%s\n' *.log |
  sort -t- -k6n 
n02-z30-sr65-rgdt0p25-dc0p08-4x3drw.log
n02-z30-sr65-rgdt0p25-dc0p01-8x6drw.log
n02-z30-sr65-rgdt0p25-dc0p03-8x6drw.log
n02-z30-sr65-rgdt0p25-dc0p05-8x6drw.log
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run3.log

Consider the following:

Code:
zsh-4.3.12[t]% printf '%s\n' *.log |
  sort -t- -k6n |
    awk -F- 'NR > 1 {
  print "$5 is:", $5, "$6 is:", $6, "the value of _[$5, $6]++ is:", _[$5, $6]++
  }'
$5 is: dc0p01 $6 is: 8x6drw.log the value of _[$5, $6]++ is: 0
$5 is: dc0p03 $6 is: 8x6drw.log the value of _[$5, $6]++ is: 0
$5 is: dc0p05 $6 is: 8x6drw.log the value of _[$5, $6]++ is: 0
$5 is: dc0p002 $6 is: 16x12drw the value of _[$5, $6]++ is: 0
$5 is: dc0p002 $6 is: 16x12drw the value of _[$5, $6]++ is: 1
$5 is: dc0p002 $6 is: 16x12drw the value of _[$5, $6]++ is: 2
$5 is: dc0p004 $6 is: 16x12drw the value of _[$5, $6]++ is: 0
$5 is: dc0p004 $6 is: 16x12drw the value of _[$5, $6]++ is: 1
$5 is: dc0p004 $6 is: 16x12drw the value of _[$5, $6]++ is: 2
$5 is: dc0p006 $6 is: 16x12drw the value of _[$5, $6]++ is: 0
$5 is: dc0p006 $6 is: 16x12drw the value of _[$5, $6]++ is: 1
$5 is: dc0p006 $6 is: 16x12drw the value of _[$5, $6]++ is: 2
$5 is: dc0p008 $6 is: 16x12drw the value of _[$5, $6]++ is: 0
$5 is: dc0p008 $6 is: 16x12drw the value of _[$5, $6]++ is: 1
$5 is: dc0p008 $6 is: 16x12drw the value of _[$5, $6]++ is: 2
$5 is: dc0p01 $6 is: 16x12drw the value of _[$5, $6]++ is: 0
$5 is: dc0p01 $6 is: 16x12drw the value of _[$5, $6]++ is: 1
$5 is: dc0p01 $6 is: 16x12drw the value of _[$5, $6]++ is: 2

So we just need to add a newline (the default RS) in front of the lines
where _[$5, $6]++ is 0 (0 is false, that's why _[$5, $6]++ || ..., || in awk is the logical OR operator).
# 10  
Old 11-07-2011
Quote:
Originally Posted by ahamed101
Amazing!... I really like the way you code, it is sooo compact!...
Thank you!
# 11  
Old 11-08-2011
I understand what you have said but am stuck on what the below does.
What type of keying is it?

Code:
[$5,$6]

---------- Post updated at 03:45 PM ---------- Previous update was at 03:39 PM ----------

Got it!!! Smilie

---------- Post updated 11-08-11 at 09:38 AM ---------- Previous update was 11-07-11 at 03:45 PM ----------

concerning the sort command
Code:
sort -t- -k6n

What is the meaning of -k6n

I know that -k6 picks the 6th field which will be used for sorting.

What is n supposed to do?

This is the list of files:

Code:
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p002-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p004-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p006-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p008-16x12drw-run3.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run1.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run2.log
n02-z30-sr65-rgdt0p25-dc0p01-16x12drw-run3.log

Last edited by kristinu; 11-08-2011 at 10:53 AM..
# 12  
Old 11-08-2011
man sort
n is for numeric sort

--ahamed
# 13  
Old 11-08-2011
same as -n then. What's the numeric sort about?

---------- Post updated at 10:08 AM ---------- Previous update was at 10:04 AM ----------

I have created a list file list.txt

Code:
abc-123-yz789
abc-456-yz456
abc-789-yz123
def-123-vwx789
def-456-vwx456
def-789-vwx123
ghi-123-stu789
ghi-456-stu456
ghi-789-stu123
jkl-123-pqr789
jkl-456-pqr456
jkl-789-pqr123
mno-123-mno789
mno-456-mno456
mno-789-mno123
pqr-123-jkl789
pqr-456-jkl456
pqr-789-jkl123
stu-123-ghi789
stu-456-ghi456
stu-789-ghi123
vwx-123-def789
vwx-456-def456
vwx-789-def123
yz-123-abc789
yz-456-abc456
yz-789-abc123

I get a different result when using the two commands below
Code:
sort -t- -k3 list.txt
sort -t- -k3n list.txt

# 14  
Old 11-08-2011
Yes, you will get a different result... So whats your point?... One uses numeric sort and other doesn't...

--ahamed
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Complex data sorting in excel files or text files

Dear all, I have a complex data file shown below,,,,, A_ABCD_13208 0 0 4.16735 141044 902449 1293900 168919 C_ABCD_13208 0 0 4.16735 141044 902449 1293900 168919 A_ABCDEF715 52410.9 18598.2 10611 10754.7 122535 252426 36631.4 C_DBCDI_1353 0... (19 Replies)
Discussion started by: AAWT
19 Replies

2. Shell Programming and Scripting

i need help in sorting two files

i have file a 123 234 456 567 678 and file b 123|xxx|hhh|ppp or zzz 234|rrr|ttt|xxx 432|ttt|mmm|nnn 678|cft|byt|mop i want to compare file a to file b such that when each of the lines in file a can be found in file b column1 and also xxx or hhh or ppp or zzz can be... (12 Replies)
Discussion started by: blackzinga80
12 Replies

3. UNIX for Dummies Questions & Answers

help with sorting files

find / -type f 2> /dev/null | find -inum +1 2> /dev/null | find -mtime -30 2> /dev/null what i am trying to do i search all regular files in root directory with one or more inodes modified within last 30 days. the /dev/null is to suppress the permission denied outputs. i am now trying to... (5 Replies)
Discussion started by: iluvsushi
5 Replies

4. UNIX for Dummies Questions & Answers

Sorting files in different directories

hi everybody, first time writing. Here's my question: I've got several files in different directories like this: aa/t1 aa/bb/t2 aa/t2 aa/bb/cc/t1 aa/t3 and would like to get this sorting: aa/t1 aa/bb/cc/t1 aa/t2 aa/bb/t2 aa/t3 (1 Reply)
Discussion started by: Camisa
1 Replies

5. Shell Programming and Scripting

sorting files

hi i have file like below: col1,col2,col3,col4 val1,val2,val3,val4 abc1,abc2,abc3,abc4 this is a 4 column file with 3 rows. i want to sort the file like.. first on col1, then on col2 and so ..on.. i want the sort order to be descending. Pls help.. Thnks Sumit (2 Replies)
Discussion started by: sumit207
2 Replies

6. UNIX for Dummies Questions & Answers

Need some help for sorting files

I am new to shell scripting can u guys please provide a small script for the following senario step1:need to find some files in a directory for ex having 020908 step2:sort them and redirecting to new file (ex:sort abc > abc.sort) i am trying this but giveing flag error ls -l... (4 Replies)
Discussion started by: cgreddy2020
4 Replies

7. Shell Programming and Scripting

need help listing/sorting files

I am currently attempting to create a file which I access from an oracle form. At the minute I do a host command and run an ls -l e.g. /bin/ls -l /dir/dir/dir/ > /tmp/list.txt I then read this file within my oracle form. However I want the user to be able to restrict, sort and filter the... (2 Replies)
Discussion started by: dave_angel
2 Replies

8. Shell Programming and Scripting

Sorting Files

How to sort such files which contains records of varying length and varying lines? (With respect to Bash shell) Eg: Each record begins with a sting of 1/0(binary) which may or may not be followed by properties like AB,BS etc. I have to sort such records on the basis of 1/0 string and keep the... (2 Replies)
Discussion started by: sandeep_hi
2 Replies

9. Shell Programming and Scripting

Sorting files

Hi, What is the command for sorting files according to their size Thanx in advance (4 Replies)
Discussion started by: sendhil
4 Replies

10. UNIX for Advanced & Expert Users

sorting files

ok so I'm having major issues trying to figure this out: I have this program that I'm inputting the files in hte current directory which are image files...it spits out 5 line chunks describing the files... filename: (name of file) size: (100 x 200) arbitrary data arbitrary data arbitrary... (4 Replies)
Discussion started by: Infraredskies
4 Replies
Login or Register to Ask a Question