optimizing - to find the number of occurrence Post: 302293552

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers optimizing - to find the number of occurrence Post 302293552 by e3r1ck_ETT on Tuesday 3rd of March 2009 09:56:22 AM

03-03-2009

Registered User

I found the unix command to be useful.

cat filename | sort | uniq -c | sort -nr

What if I am using a tab-delimited text file with multiple columns?

Example:

S100A16 hsa-miR-125a-3p S100A16 0.0011959 0.768059
PBXIP1 hsa-miR-125a-3p PBXIP1 0.0199898 0.700326
CYB5R3 hsa-miR-125a-3p CYB5R3 0.0081174 0.748953
BEST3 hsa-miR-125a-3p BEST3 0.00148927 0.756234
FAM101A hsa-miR-125a-3p FAM101A 0.0196212 0.783555
KIAA0195 hsa-miR-125a-3p KIAA0195 0.0019755 0.747427
LLGL2 hsa-miR-125a-3p LLGL2 0.0248212 0.876563
FBLN5 hsa-miR-125a-3p FBLN5 0.0162988 0.776446
IFITM3 hsa-miR-125a-3p IFITM3 0.00896808 0.478704
SSH3 hsa-miR-125a-3p SSH3 0.0301693 0.836054
EXTERNAL_NAME SEQ EXTERNAL_NAME p-value(1 vs. 2) Ratio(1 vs. 2)
EMILIN1 hsa-miR-369-5p EMILIN1 0.0254294 0.720094
ADD3 hsa-miR-369-5p ADD3 0.0184075 0.742096
AIFM2 hsa-miR-369-5p AIFM2 0.00646348 0.829228
GPT2 hsa-miR-369-5p GPT2 0.00473291 0.706895

and I want the output to read

10 hsa-miR-125a-3p
4 hsa-miR-369-5p
1 SEQ

Initially, I made a new file, composing only the column of interest, using perl (column 1 here), named it filenam_list; then applied the

cat filenam_list | sort | uniq -c | sort -nr > filename_counts
rm filenam_list

Is there a more efficient way of doing this? I'm sure there has to be. I repeated the procedure on 7 files and I have to it again.

e3r1ck_ETT

View Public Profile for e3r1ck_ETT

Find all posts by e3r1ck_ETT

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Line number of an occurrence using grep

Hi All, is there a way to extract the line number of an occurrence using grep? I know that with the -n option it prints out the line number as well. I would like to assign the line number to a variable. Thanks, Sarah

2. UNIX for Dummies Questions & Answers

line number of the i-th occurrence of a pattern

Hi all, is there a simple way to obtain the line number of the i-th occurrence of a pattern? I have OCCURRENCE=`grep -io "${STRING_NAME}" ${1}-${8}${EXT}.out_bis| wc -l` which tells me how many occurency I have. I would like to go through them and determine the line number and assign...

3. Shell Programming and Scripting

Uniq adresses with number of occurrence

Alo I have a file with a lot of addresses where I want to list unique addresses and the number of theirs occurrence. I have this input file: 0011bd09 ea 01 0b 04 ea 01 0b 38-bd 11 00 98 15 cb 01 00 .......8........ 0011bd09 ea 11 00 98 15 cb 01 00-00 00 00 d8 3d 8d 01 94 ...

4. Shell Programming and Scripting

find string nth occurrence in file and print line number

Hi I have requirement to find nth occurrence in a file and capture data from with in lines (between lines) Data in File. <QUOTE> <SESSION> <ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/> <ATTRIBUTE NAME='Service Name' VALUE='None'/> </SESSION> <SESSION> <ATTRIBUTE...

5. Shell Programming and Scripting

Count number of occurrence at each line

Hi I have the following file ENST001 ENST002 4 4 4 88 9 9 ENST004 3 3 3 99 8 8 ENST009 ENST010 ENST006 8 8 8 77 8 8 Basically I want to count how many times ENST* is repeated in each line so the expected results is 2 1 3 Any suggestion please ?

6. Shell Programming and Scripting

number of occurrence

: i need a bash script to convert the displayed output 12 14 15 12 15 13 to 12 * 2 ,13 * 1,14*1,15*1 Thanks, nevil

7. Shell Programming and Scripting

How to count the occurrence of a number?

Hi, I have a file which contained a set of numbers like Col1 col2 col3 col4 1 sa 13 0 2 sb 14 0 3 sc 15 9 4 sd 16 -9 5 sd 20 -2 6 sd 20 4 Here in last column I need to count the zeros, positive values and negative values, please help me to do that.

8. Shell Programming and Scripting

awk to find the number of occurrence

My file contains like this on 10 th line NM1*IL*1* awk '/NM1/{print NR}' *.dat output is 10 awk '/NM1*IL*1*/{print NR}' *.dat output is Nothing but im expecting 10 on second code as well .

9. Shell Programming and Scripting

awk or other way to find out number of occurrence of 7th character

Hi all, I am looking for to filter out based on 7th character and list the number of occurrence based on the 7th character if p , d , o or m 1. if 7th character is p , Output should be: p_hosts = N 2. if 7th character is d , Output should be: d_hosts = N 3. if 7th character is o , Output...

10. Shell Programming and Scripting

Optimizing find with many replacements

Hello, I'm looking for advice on how to optimize this bash script, currently i use the shotgun approach to avoid file io/buffering problems of forks trying to write simultaneously to the same file. i'd like to keep this as a fairly portable bash script rather than writing a C routine. in a...

LEARN ABOUT NETBSD

comm

COMM(1) 						    BSD General Commands Manual 						   COMM(1)

NAME

     comm -- select or reject lines common to two files

SYNOPSIS

     comm [-123f] file1 file2

DESCRIPTION

     The comm utility reads file1 and file2, which should be sorted lexically, and produces three text columns as output: lines only in file1;
     lines only in file2; and lines in both files.

     The filename ``-'' means the standard input.

     The following options are available:

     -1      Suppress printing of column 1.

     -2      Suppress printing of column 2.

     -3      Suppress printing of column 3.

     -f      Fold case in line comparisons.

     Each column will have a number of tab characters prepended to it equal to the number of lower numbered columns that are being printed.  For
     example, if column number two is being suppressed, lines printed in column number one will not have any tabs preceding them, and lines
     printed in column number three will have one.

     comm assumes that the files are lexically sorted; all characters participate in line comparisons.

EXIT STATUS

     comm exits 0 on success, >0 if an error occurred.

SEE ALSO

     cmp(1), diff(1), sort(1), uniq(1)

STANDARDS

     The comm utility conforms to IEEE Std 1003.2-1992 (``POSIX.2'').

BSD
								   June 6, 1993 							       BSD