Sponsored Content
Full Discussion: sort/uniq
Top Forums UNIX for Dummies Questions & Answers sort/uniq Post 30118 by jimmyflip on Thursday 17th of October 2002 04:22:02 AM
Old 10-17-2002
sort/uniq

I have a file:

Fred
Fred
Fred
Jim
Fred
Jim
Jim

If sort is executed on the listed file, shouldn't the output be?:

Fred
Fred
Fred
Fred
Jim
Jim
Jim

Then if uniq is executed on the sorted file, shouldn't the output be:

4 Fred
3 Jim

I can't get it to work. Any help on this would be appreciated. Thanks.
jimmyflip
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with Last,uniq, sort and cut

Using the last, uniq, sort and cut commands, determine how many times the different users have logged in. I know how to use the last command and cut command... i came up with last | cut -f1 -d" " | uniq i dont know if this is right, can someone please help me... thanks (1 Reply)
Discussion started by: jay1228
1 Replies

2. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

3. Shell Programming and Scripting

Sort, Uniq, Duplicates

Input File is : ------------- 25060008,0040,03, 25136437,0030,03, 25069457,0040,02, 80303438,0014,03,1st 80321837,0009,03,1st 80321977,0009,03,1st 80341345,0007,03,1st 84176527,0047,03,1st 84176527,0047,03, 20000735,0018,03,1st 25060008,0040,03, I am using the following in the script... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies

4. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

5. Shell Programming and Scripting

sort | uniq question

Hello, I have a large data file: 1234 8888 bbb 2745 8888 bbb 9489 8888 bbb 1234 8888 aaa 4838 8888 aaa 3977 8888 aaa I need to remove duplicate lines (where the first column is the duplicate). I have been using: sort file.txt | uniq -w4 > newfile.txt However, it seems to keep the... (11 Replies)
Discussion started by: palex
11 Replies

6. Shell Programming and Scripting

Sort and uniq after comparision

Hi All, I have a text file with the format shown below. Some of the records are duplicated with the only exception being date (Field 15). I want to compare all duplicate records using subscriber number (field 7) and keep only those records with greater date. ... (1 Reply)
Discussion started by: nua7
1 Replies

7. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

8. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

9. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies

10. Shell Programming and Scripting

Sort & Uniq -u

Hi All, Below the actual file which i like to sort and Uniq -u /opt/oracle/work/Antony/Shell_Script> cat emp.1st 2233|a.k. shukula |g.m. |sales |12/12/52 |6000 1006|chanchal singhvi |director |sales |03/09/38 |6700... (8 Replies)
Discussion started by: Antony Ankrose
8 Replies
Boulder::Stream(3pm)					User Contributed Perl Documentation				      Boulder::Stream(3pm)

NAME
Boulder::Stream - Read and write tag/value data from an input stream SYNOPSIS
#!/bin/perl # Read a series of People records from STDIN. # Add an "Eligible" tag to all those whose # Age >= 35 and Friends list includes "Fred" use Boulder::Stream; # filestream way: my $stream = Boulder::Stream->newFh; while ( my $record = <$stream> ) { next unless $record->Age >= 35; my @friends = $record->Friends; next unless grep {$_ eq 'Fred'} @friends; $record->insert(Eligible => 'yes'); print $stream $record; } # object oriented way: my $stream = Boulder::Stream->new; while (my $record = $stream->get ) { next unless $record->Age >= 35; my @friends = $record->Friends; next unless grep {$_ eq 'Fred'} @friends; $record->insert(Eligible => 'yes'); print $stream $record; } DESCRIPTION
Boulder::Stream provides stream-oriented access to Boulder IO hierarchical tag/value data. It can be used in a magic tied filehandle mode, as shown in the synopsis, or in object-oriented mode. Using tied filehandles, Stone objects are read from input using the standard <> operator. Stone objects printed to the tied filehandle appear on the output stream in Boulder format. By default, data is read from the magic ARGV filehandle (STDIN or a list of files provided on the command line) and written to STDOUT. This can be changed to the filehandles of your choice. Pass through behavior When using the object-oriented form of Boulder::Stream, tags which aren't specifically requested by the get() method are passed through to output unchanged. This allows pipes of programs to be constructed easily. Most programs will want to put the tags back into the boulder stream once they're finished, potentially adding their own. Of course some programs will want to behave differently. For example, a database query program will generate but not read a boulderio stream, while a report generator will read but not write the stream. This convention allows the following type of pipe to be set up: query_database | find_vector | find_dups | | blast_sequence | pick_primer | mail_report If all the programs in the pipe follow the conventions, then it will be possible to interpose other programs, such as a repetitive element finder, in the middle of the pipe without disturbing other components. SKELETON BOULDER PROGRAM
Here is a skeleton example. #!/bin/perl use Boulder::Stream; my $stream = Boulder::Stream->newFh; while ( my $record = <$stream> ) { next unless $record->Age >= 35; my @friends = $record->Friends; next unless grep {$_ eq 'Fred'} @friends; $record->insert(Eligible => 'yes'); print $stream $record; } The code starts by creating a Boulder::Stream object to handle the I/O. It reads from the stream one record at a time, returning a Stone object. We recover the Age and Friends tags, and continue looping unless the Age is greater or equal to 35, and the list of Friends contains "Fred". If these criteria match, then we insert a new tag named Eligible and print the record to the stream. The output may look like this: Name=Janice Age=36 Eligible=yes Friends=Susan Friends=Fred Friends=Ralph = Name=Ralph Age=42 Eligible=yes Friends=Janice Friends=Fred = Name=Susan Age=35 Eligible=yes Friends=Susan Friends=Fred = Note that in this case only records that meet the criteria are echoed to standard output. The object-oriented version of the program looks like this: #!/bin/perl use Boulder::Stream; my $stream = Boulder::Stream->new; while ( my $record = $stream->get('Age','Friends') ) { next unless $record->Age >= 35; my @friends = $record->Friends; next unless grep {$_ eq 'Fred'} @friends; $record->insert(Eligible => 'yes'); $stream->put($record); } The get() method is used to fetch Stones containing one or more of the indicated tags. The put() method is used to send the result to standard output. The pass-through behavior might produce a set of records like this one: Name=Janice Age=36 Eligible=yes Friends=Susan Friends=Fred Friends=Ralph = Name=Phillip Age=30 = Name=Ralph Age=42 Eligible=yes Friends=Janice Friends=Fred = Name=Barbara Friends=Agatha Friends=Janice = Name=Susan Age=35 Eligible=yes Friends=Susan Friends=Fred = Notice that there are now two records ("Phillip" and "Barbara") that do not contain the Eligible tag. Boulder::Stream METHODS $stream = Boulder::Stream->new(*IN,*OUT) $stream = Boulder::Stream->new(-in=>*IN,-out=>*OUT) The new() method creates a new Boulder::Stream object. You can provide input and output filehandles. If you leave one or both undefined new() will default to standard input or standard output. You are free to use files, pipes, sockets, and other types of file handles. You may provide the filehandle arguments as bare words, globs, or glob refs. You are also free to use the named argument style shown in the second heading. $fh = Boulder::Stream->newFh(-in=>*IN, -out=>*OUT) Returns a filehandle object tied to a Boulder::Stream object. Reads on the filehandle perform a get(). Writes invoke a put(). To retrieve the underlying Boulder::Stream object, call Perl's built-in tied() function: $stream = tied $fh; $stone = $stream->get(@taglist) @stones = $stream->get(@taglist) Every time get() is called, it will return a new Stone object. The Stone will be created from the input stream, using just the tags provided in the argument list. Pass no tags to receive whatever tags are present in the input stream. If none of the tags that you specify are in the current boulder record, you will receive an empty Stone. At the end of the input stream, you will receive undef. If called in an array context, get() returns a list of all stones from the input stream that contain one or more of the specified tags. $stone = $stream->read_record(@taglist) Identical to get(>, but the name is longer. $stream->put($stone) Write a Stone to the output filehandle. $stream->write_record($stone) Identical to put(), but the name is longer. Useful State Variables in a Boulder::Stream Every Boulder::Stream has several state variables that you can adjust. Fix them in this fashion: $a = new Boulder::Stream; $a->{delim}=':'; $a->{record_start}='['; $a->{record_end}=']'; $a->{passthru}=undef; o delim This is the delimiter character between tags and values, "=" by default. o record_start This is the start of nested record character, "{" by default. o record_end This is the end of nested record character, "}" by default. o passthru This determines whether unrecognized tags should be passed through from the input stream to the output stream. This is 'true' by default. Set it to undef to override this behavior. BUGS
Because the delim, record_start and record_end characters in the Boulder::Stream object are used in optimized (once-compiled) pattern matching, you cannot change these values once get() has once been called. To change the defaults, you must create the Boulder::Stream, set the characters, and only then begin reading from the input stream. For the same reason, different Boulder::Stream objects cannot use different delimiters. AUTHOR
Lincoln D. Stein <lstein@cshl.org>, Cold Spring Harbor Laboratory, Cold Spring Harbor, NY. This module can be used and distributed on the same terms as Perl itself. SEE ALSO
Boulder, Boulder::Blast, Boulder::Genbank, Boulder::Medline, Boulder::Unigene, Boulder::Omim, Boulder::SwissProt perl v5.10.1 2001-06-11 Boulder::Stream(3pm)
All times are GMT -4. The time now is 07:45 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy