Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to Sort Records Uniquely? Post 302168484 by earnstaf on Monday 18th of February 2008 02:01:57 PM
Old 02-18-2008
Quote:
Originally Posted by otheus
It's not clear what key should be used for the sort. But if it were the string as a whole, we could try:

Code:
perl -0045 -ne'$/="%";$X{$_}++ } END { foreach (sort keys %X) { chop; print $_,"Count: ", $X{$_},"\n%"; ' filename

[/code]

You can change "sort keys" to "sort { EXPR } keys" where EXPR is an expression that compares $a and $b and returns 0, -1, or 1. The implied default is "$a <=> $b", but you can customize it to sort on a subfield, like:

Code:
foreach (sort {
   ($a =~ /dstport: (\d+)/m) <=> ($b =~ /dstport:  (\d+)/m); 
} keys %X) ....

otheus,

This looks good and I see you have an output for "count" but I'm a little confused on how to use it in practice. The keys, in particular, is something I'm not familiar at all with. If we have it sort on the subfield dstport, and then match somewhere, but srcip is different within the same record, what is the output?

Also, do I need to write a perl script or can this be done from the command line?

I ran the command you have in your original block of code, and it gives the output that radoulov's code gives with a Count at the bottom of each record (which is blank, meaning it just reads "Count:" with nothing filling it.) I would guess that has something to do with the foreach in your next block of code that I'm not really sure how to implement.

Last edited by earnstaf; 02-18-2008 at 03:07 PM..
 

10 More Discussions You Might Find Interesting

1. Solaris

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (2 Replies)
Discussion started by: svenkatareddy
2 Replies

2. Shell Programming and Scripting

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (19 Replies)
Discussion started by: svenkatareddy
19 Replies

3. Shell Programming and Scripting

Sort & Split records in a file

Hi, I am new to scripting. I need a script to sort and the records in a file and then split them into different files. For example, the file is: H1...................... H2...................... D2.................... D2.................... H1........................... (15 Replies)
Discussion started by: Sunitha_edi82
15 Replies

4. Shell Programming and Scripting

Based on num of records in file1 need to check records in file2 to set some condns

Hi All, I have two files say file1 and file2. I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions . Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies

5. Shell Programming and Scripting

sort a file which has 3.7 million records

hi, I'm trying to sort a file which has 3.7 million records an gettign the following error...any help is appreciated... sort: Write error while merging. Thanks (6 Replies)
Discussion started by: greenworld
6 Replies

6. Shell Programming and Scripting

Unix sort for fixed length columns and records

I was trying to use the AIX 6.1 sort command to sort fixed-length data records, sorting by specific columns only. It took some time to figure out how to get it to work, so I wanted to share the solution. The sort man page wasn't much help, because it talks about field delimeters (default space... (1 Reply)
Discussion started by: CheeseHead1
1 Replies

7. UNIX for Dummies Questions & Answers

Alphabetical sort for multi line records contains in a single file

Hi all, I So, I've got a monster text document comprising a list of various company names and associated info just in a long list one after another. I need to sort them alphabetically by name... The text document looks like this: Company Name: the_first_company's_name_here Address:... (2 Replies)
Discussion started by: quee1763
2 Replies

8. UNIX for Advanced & Expert Users

How to uniquely distinguish between two USB ports??

Hi all, I am facing a problem while writing a shell script. My machine has two USB ports- left port and right port. whenever I connect USBS to both the ports, entry is generated as /sys/block/sdc and /sys/block/sdd and I mount the USBs to a particular directory. But I need to know... (3 Replies)
Discussion started by: Pkumar Sachin
3 Replies

9. Shell Programming and Scripting

How to read records in a file and sort it?

I have a file which has number of pipe delimited records. I am able to read the records....but I want to sort it after reading. i=0 while IFS="|" read -r usrId dataOwn expire email group secProf startDt endDt smhRole RoleCat DataProf SysRole MesgRole SearchProf do print $usrId $dataOwn... (4 Replies)
Discussion started by: harish468
4 Replies

10. UNIX for Beginners Questions & Answers

Extract delta records using with "comm" and "sort" commands combination

Hi All, I have 2 pipe delimited files viz., file_old and file_new. I'm trying to compare these 2 files, and extract all the different rows between them into a new_file. comm -3 < sort file_old < sort file_new > new_file I am getting the below error: -ksh: sort: cannot open But if I do... (7 Replies)
Discussion started by: njny
7 Replies
USORT(3)								 1								  USORT(3)

usort - Sort an array by values using a user-defined comparison function

SYNOPSIS
bool usort (array &$array, callable $value_compare_func) DESCRIPTION
This function will sort an array by its values using a user-supplied comparison function. If the array you wish to sort needs to be sorted by some non-trivial criteria, you should use this function. Note If two members compare as equal, their relative order in the sorted array is undefined. Note This function assigns new keys to the elements in $array. It will remove any existing keys that may have been assigned, rather than just reordering the keys. PARAMETERS
o $array - The input array. o $value_compare_func - The comparison function must return an integer less than, equal to, or greater than zero if the first argument is considered to be respectively less than, equal to, or greater than the second. int callback (mixed $a, mixed $b) Caution Returning non-integer values from the comparison function, such as float, will result in an internal cast to integer of the callback's return value. So values such as 0.99 and 0.1 will both be cast to an integer value of 0, which will compare such values as equal. RETURN VALUES
Returns TRUE on success or FALSE on failure. EXAMPLES
Example #1 usort(3) example <?php function cmp($a, $b) { if ($a == $b) { return 0; } return ($a < $b) ? -1 : 1; } $a = array(3, 2, 5, 6, 1); usort($a, "cmp"); foreach ($a as $key => $value) { echo "$key: $value "; } ?> The above example will output: 0: 1 1: 2 2: 3 3: 5 4: 6 Note Obviously in this trivial case the sort(3) function would be more appropriate. Example #2 usort(3) example using multi-dimensional array <?php function cmp($a, $b) { return strcmp($a["fruit"], $b["fruit"]); } $fruits[0]["fruit"] = "lemons"; $fruits[1]["fruit"] = "apples"; $fruits[2]["fruit"] = "grapes"; usort($fruits, "cmp"); while (list($key, $value) = each($fruits)) { echo "$fruits[$key]: " . $value["fruit"] . " "; } ?> When sorting a multi-dimensional array, $a and $b contain references to the first index of the array. The above example will output: $fruits[0]: apples $fruits[1]: grapes $fruits[2]: lemons Example #3 usort(3) example using a member function of an object <?php class TestObj { var $name; function TestObj($name) { $this->name = $name; } /* This is the static comparing function: */ static function cmp_obj($a, $b) { $al = strtolower($a->name); $bl = strtolower($b->name); if ($al == $bl) { return 0; } return ($al > $bl) ? +1 : -1; } } $a[] = new TestObj("c"); $a[] = new TestObj("b"); $a[] = new TestObj("d"); usort($a, array("TestObj", "cmp_obj")); foreach ($a as $item) { echo $item->name . " "; } ?> The above example will output: b c d Example #4 usort(3) example using a closure to sort a multi-dimensional array <?php $array[0] = array('key_a' => 'z', 'key_b' => 'c'); $array[1] = array('key_a' => 'x', 'key_b' => 'b'); $array[2] = array('key_a' => 'y', 'key_b' => 'a'); function build_sorter($key) { return function ($a, $b) use ($key) { return strnatcmp($a[$key], $b[$key]); }; } usort($array, build_sorter('key_b')); foreach ($array as $item) { echo $item['key_a'] . ', ' . $item['key_b'] . " "; } ?> The above example will output: y, a x, b z, c SEE ALSO
uasort(3), The comparison of array sorting functions. PHP Documentation Group USORT(3)
All times are GMT -4. The time now is 06:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy