Extracting column if above certain values and repeated over a number of times continuously


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extracting column if above certain values and repeated over a number of times continuously
# 8  
Old 09-23-2010
This is what I get for the 3th column:
Code:
$ cat file
id.1 A01 A01 A01 A68 A68
id.2 A5 A5 A5 A3 A3
1001 0.01 0 abc 0.136 0.136
1002 0.02 0 abc 0.262 0.183
1003 0.03 0 abc 0.662 0.662
1004 0.04 0 def 0.758 0.607
1005 0.05 0 def 0.855 0.855
1006 0.06 0 efg 0.867 0.867
1007 0 0 hij 0.872 0.692
1008 0.04 0 def 0.902 0.902
1009 0 0.06 def 0.51 0.906
1010 0 0 efg 0.51 0.906
1011 0 0.04 hij 0.51 0.969
1013 0.05 0 abc 0.07 0.743
1014 0.06 0 def 0.971 0.971
1015 0.07 0 def 0.971 0.743
1021 0 0.51 efg 0.996 0.996
1022 0 0.51 hij 0.996 0.996
1023 0 0.51 abc 0 0.995
1026 0.51 0 def 0 0.996
1027 0.51 0 poi 0 0.996
1028 0 0 sdf 0.05 0.996
$
$ awk 'NR<3{print $1, $3; next} $3 > 0.5{print $1, $3}' file
id.1 A01
id.2 A5
1021 0.51
1022 0.51
1023 0.51

Isn't this what you expected?
This User Gave Thanks to Franklin52 For This Post:
# 9  
Old 09-23-2010
Actually I do not have access to unix box. for this code can you give the trace of it by set -x. On basis of this I will look into it.

Code:
 
cols_to_print="1,2"

while col_no in 3 4 5 6 7 8 9 10 ##type all columns to include in search
do
y=`cut -f ${field_no}  -d " " filename | uniq -c | sort -b -n -r | \
sed -n -e '1s/[^0-9]*\([0-9][0-9]*\).*/\1/p'`

[ $y -ge 50 ] && cols_to_print=${cols_to_print}",${col_no}"

done

cut -d " " -f ${cols_to_print}



---------- Post updated at 01:02 PM ---------- Previous update was at 12:52 PM ----------

Code:
cols_to_print="1,2"

while col_no in 3 4 5 6 7 8 9 10 ##type all columns to include in search
do
y=`cut -f ${field_no}  -d " " filename | uniq -c | sort -b -n -r | \
sed -n -e '1s/[^0-9]*\([0-9][0-9]*\).*/\1/p'`

[ $y -ge 50 ] && cols_to_print=${cols_to_print}",${col_no}"

done

cut -d " " -f ${cols_to_print} filename ## i missed the filename here.

Please give a trace.. maybe there is a typo.. I will be able tel then.
# 10  
Old 09-23-2010
Hi Franklin52,
Thanks
I got it correct this time. I may have make typo in my last attempt.
Can you help in explaining the part NR<3{print $1, $3; next} $3 > 0.5

newbeeuk

---------- Post updated at 04:43 PM ---------- Previous update was at 04:38 PM ----------

Hi Franklin52
i see the logic now
It is print the first 2 row and then check the value in the following rows.

---------- Post updated at 04:57 PM ---------- Previous update was at 04:43 PM ----------

Hi 116@434

I run your code in script file
nothing run this time. When -x is added, then it is just like cat file

When i replace while with for in line 2 of code, then it return
cut: invalid byte, character or field list
and line 8 -ge: unary operator expected

newbeeuk
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting values based on line-column numbers from multiple text files

Dear All, I have to solve the following problems with multiple tab-separated text file but I don't know how. Any help would be greatly appreciated. I have access to Linux mint (but not as a professional). I have multiple tab-delimited files with the following structure: file1: 1 44 2 ... (5 Replies)
Discussion started by: Bastami
5 Replies

2. UNIX for Dummies Questions & Answers

Append no of times a column is repeated at the end

Hi folks, Iam working on a bash script, i need to print how many times column 2 repeated at the end of each line. Input.txt COL1 COL2 COL3 COL4 1 XX 45 N 2 YY 34 y 3 ZZ 44 N 4 XX 89 Y 5 XX 45 N 6 YY 84 D 7 ZZ 22 S Output.txt COL1 COL2 COL3 COL4 COL5 1 XX 45 N 3 2 YY 34... (6 Replies)
Discussion started by: tech_frk
6 Replies

3. Shell Programming and Scripting

Extracting unique values of a column from a feed file

Hi Folks, I have the below feed file named abc1.txt in which you can see there is a title and below is the respective values in the rows and it is completely pipe delimited file ,. ... (4 Replies)
Discussion started by: punpun66
4 Replies

4. Homework & Coursework Questions

Accepting a phrase and counting the number of times that it is repeated in a specific website

1. The problem statement, all variables and given/known data: Develop a shell script that accepts a phrase and counts the number of times that it is repeated in a specific website. Note: Im not sure if it's the whole website, or just a specific page but im guessing its thewhole website. ... (2 Replies)
Discussion started by: Zakerii
2 Replies

5. Shell Programming and Scripting

Choosing between repeated entries based on the "absolute values" of a column

Hello, I was looking for a way to select between the repeated entries (column1) based on the values of absolute values of column 3 (larger value). For example if the same gene id has FC value -2 and 1, I should get the output as -2. Kindly help. GeneID Description FC ... (2 Replies)
Discussion started by: Sanchari
2 Replies

6. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

7. UNIX for Dummies Questions & Answers

Extracting rows from a space delimited text file based on the values of a column

I have a space delimited text file. I want to extract rows where the third column has 0 as a value and write those rows into a new space delimited text file. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

8. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on numerical values of a column

I have a text file where the second column is a list of numbers going from small to large. I want to extract the rows where the second column is smaller than or equal to 0.0001. My input: rs10082730 9e-08 12 46002702 rs2544081 1e-07 12 46015487 rs1425136 1e-06 7 35396742 rs2712590... (1 Reply)
Discussion started by: evelibertine
1 Replies

9. UNIX for Dummies Questions & Answers

To get the total of how many times the pattern is repeated.

Hi, I need the total of how many times the pattern is being repeated in a particular file. I tried with grep as below, but no go. grep -c best sample Contents of Sample file: ----------------------- This is the best site I have never come across. The best one indeed. Very best... (8 Replies)
Discussion started by: venkatesht
8 Replies

10. UNIX for Dummies Questions & Answers

how many times does this repeated sequence exist

need a script to determine daily how many times does the below repeated sequence exist in a log file, and if it shows failure to connect in the same file 200 PORT Command successful. 150 Opening BINARY mode data connection for rgr016.daily.0305. 226 Transfer complete. local: rgr016.daily.0306... (3 Replies)
Discussion started by: lichento
3 Replies
Login or Register to Ask a Question