Sort cmd to get GROUP BY?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Sort cmd to get GROUP BY?
# 1  
Old 10-15-2008
SOLVED:Sort cmd to get GROUP BY?

Is there any way using the sort command to get something like a GROUP BY clause?

I'm trying to sort through race_event records and group them by those individuals that raced together in the same race on the same date at the same track, but i run into a problem when there were days when the track had qualifying races, plus regular races on the same day; so there could be two Race 1's, Race 2's, etc for the same date at the same track.
I tried a sort like:
Code:
sort -t':' -k3 -k4 -k9 -k29 -k8 -k28 ~/Desktop/1998/AllRaceEvents98.txt > ~/Desktop/sortedevents.txt

where
k3 is a date
k4 is the racetrack
k9 is the class of race (would show it was a qualifier or a regular race)
k29 is the race number
k8 is the purse of race (would show 0 for qualifier and dollar amount for regular race)
k28 is the names of the top three finishers

but this does not sort right: it does not group them together into their respective races correctly. Any suggestions?

Last edited by RacerX; 10-16-2008 at 08:31 AM.. Reason: solved
# 2  
Old 10-15-2008
Hammer & Screwdriver I think you need to experiment with full key definitions

Without seeing the data, I can only simulate. Perhaps the following example will help illustrate:

sample file
Code:
> cat file99
joe:1:alpha:4:5
jim:4:beta:16:9
bette:2:comma:99:0
dennis:3:per:12:12
harry:2:comma:12:13
olivia:1:beta:4:4

Trying to sort based on key field 3 then field 2. Field 3 should put the two beta people together and then the second field should put olivia before jim based on the 1 and 4 in their records. This did not happen.
Code:
> sort -t: -k3 -k2 file99
joe:1:alpha:4:5
jim:4:beta:16:9
olivia:1:beta:4:4
harry:2:comma:12:13
bette:2:comma:99:0
dennis:3:per:12:12

I then tried by putting explicit field definitions (start and end locations). So I am sorting on the first 3 characters of field 3 to get the beta people together. Then sort on field 2 - getting olivia before jim.
Code:
> sort -t: -k3.1,3.3 -k2 file99
joe:1:alpha:4:5
olivia:1:beta:4:4
jim:4:beta:16:9
harry:2:comma:12:13
bette:2:comma:99:0
dennis:3:per:12:12

# 3  
Old 10-15-2008
i tried doing a sort using explicit definition as suggested on a couple of fields but again i could not get it to come out right.

I think i don't understand enough about how sort actually works: when i specify certain fields to sort on, what does it do sorting-wise with the fields i have not specified?

I'm beginning to think it's also sorting on a field i have not specified and that is why it is messing up the grouping.
# 4  
Old 10-15-2008
Could you post sample input and example of the desired output?
# 5  
Old 10-15-2008
Code:
Data records deleted

I'd like to get all the individuals that raced in the same race, on the same date at the same track together, but you'll notice after i do the sort in the instance above two of the individuals (bolded) ended up getting grouped with another race down on lines 616120 and 616121 when they should have followed the other individuals in their race at lines 616072 and 616073.

I hope that makes some sort of sense....

Last edited by RacerX; 10-16-2008 at 08:41 AM.. Reason: solved
# 6  
Old 10-15-2008
Hammer & Screwdriver A couple of things to think about

Sort is normally alphabetic - thus a field with a 12 will be before a 2 or 3. You will need the 'n' option on the field to make numeric.

To help determine where your script has gone astray, I would suggest you verify that your logic is fine for just one key. Then, add a second key; and so on until you discover the problem.

And from more investigation, I do not see any records improperly sorted.

Code:
> sort file200 -k3n -k4.1,4.3 -k9.1,9.10 -k29n -k8.1,8.2 -k28.1,28.10 | cut -d":" -f2-4,8-9,28-29

{data records here --- removed once issue solved --- not part of issue}

>


Last edited by joeyg; 10-16-2008 at 09:39 AM.. Reason: Added sample output from a test
# 7  
Old 10-16-2008
Quote:
Originally Posted by joeyg
Sort is normally alphabetic - thus a field with a 12 will be before a 2 or 3. You will need the 'n' option on the field to make numeric.
Thanks for this explanation, i think the 'n' option was the key to fixing my sorting problem.

Quote:
Originally Posted by joeyg
And from more investigation, I do not see any records improperly sorted.
Sorry i think my ". . ." between the records did not properly indicate that there were 49 other records between the two sets of records that actually needed to be together.

Anyhow, thanks for all your help, you steered me to a solution.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Group/Sort logs by thread number

Hello Experts, For the following sample log, can you tell me how I can sort it or group it by thread number. To be specific, I just want to group them together so that its easy to analyize. Thanks. 2012-08-17 00:00:06,369 INFO ExecuteThread: '33' for queue:... (2 Replies)
Discussion started by: samjna
2 Replies

2. Shell Programming and Scripting

Perl open(CMD, "cmd |"); buffering problem..

Hello, There's a third-party application's command that shows the application's status like "tail -f verybusy.log". When use the command, the output comes every 1-sec. but when it goes in a script below the output comes every 8-sec...What is the problem and how can I fix it? open(CMD,... (2 Replies)
Discussion started by: Shawn, Lee
2 Replies

3. Shell Programming and Scripting

Sort Data by Group !

Hello, I have a file and i want to sort by third column and extract the three top lines of each group, it is determined by the second column (144, 89, 55, etc). Could you please help me with the appropiate awk shell script XLY-XLP 144 0.592772 XLY-XLE 144 0.798121 ... (3 Replies)
Discussion started by: csierra
3 Replies

4. Shell Programming and Scripting

Unix cmd prompt how to get old cmd run?

Hi, I am using SunOS I want to serch my previous command from unix prompt (like on AIX we can search by ESC -k) how to get in SunOs urgent help require. (10 Replies)
Discussion started by: RahulJoshi
10 Replies

5. Shell Programming and Scripting

Sort the file contents in each group....print the group title as well

I've this file and need to sort the data in each group File would look like this ... cat file1.txt Reason : ABC 12345-0023 32123-5400 32442-5333 Reason : DEF 42523-3453 23345-3311 Reason : HIJ 454553-0001 I would like to sort each group on the last 4 fileds and print them... (11 Replies)
Discussion started by: prash184u
11 Replies

6. Shell Programming and Scripting

AWK or KSH : Sort, Group and extract from 3 files

Hi, I've the following two CSV files: File1.csv File2.csv Class,Student# Student#,Marks 1001,6001 6002,50 1001,6002 6001,60 1002,7000 ... (3 Replies)
Discussion started by: Matrix2682
3 Replies

7. UNIX for Dummies Questions & Answers

sort group of n lines base on pattern in first line

I have record having 10 fields and each field being printed on a new line, first line cotains name of exchange, 2nd line stock name, third line stock price, etc etc... now i want to retrieve data only for a particular exchanged and that too only 2nd and 3rd row info... NSE RNRL 70 12 1... (1 Reply)
Discussion started by: manishma71
1 Replies

8. Shell Programming and Scripting

awk - sort, then print the high value for each group

Hi @ all I'm trying to achive to this problem, I've a 2-column composed file as the following: 192.168.1.2 2 192.168.1.3 12 192.168.1.2 4 192.168.1.4 3 cpc1-swan1-2-3-cust123.swan.cable.ntl.com 4 192.168.1.3 5 192.168.1.2 10 192.168.1.4 8... (8 Replies)
Discussion started by: m4rco-
8 Replies

9. Shell Programming and Scripting

Sort, group rows

I wrote script in bash which generates this report: User1,admin,rep,User2,shell,path1,x1,r1 User2,admin,rep,User7,shell,path1,x1,r1 User3,admin,rep,User4,shell,path1,x1,r1 User4,admin,rep,User3,shell,path1,x1,r1 User5,admin,rep,User1,shell,path1,x1,r1 User6,admin,rep,User5,shell,path1,x1,r1... (6 Replies)
Discussion started by: patrykxes
6 Replies

10. UNIX for Dummies Questions & Answers

man <cmd> >> cmd.txt

I've noticed most of my postings here are because of syntax errors. So I want to begin compiling a large txt file that contains all the "man <cmd>" of the commands I most have problems with. I ran a "man nawk >> nawk.txt" but it included a header/footer on each "page". Anyone know how I'd be... (6 Replies)
Discussion started by: yongho
6 Replies
Login or Register to Ask a Question