Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Cannot subset ranges from another range set

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Cannot subset ranges from another range set
# 1  
Old 01-03-2020
Cannot subset ranges from another range set
Code:
Ca21chr2_C_albicans_SC5314	2159343	2228327	Ca21chr2_C_albicans_SC5314	636587	638608
Ca21chr2_C_albicans_SC5314	5286	50509	Ca21chr2_C_albicans_SC5314	634021	636276
Ca21chr2_C_albicans_SC5314	1886545	1900975	Ca21chr2_C_albicans_SC5314	610758	613544
Ca21chr2_C_albicans_SC5314	1919115	1930649	Ca21chr2_C_albicans_SC5314	606248	608308
Ca21chr2_C_albicans_SC5314	590278	603163	Ca21chr2_C_albicans_SC5314	1554724	1556511
Ca21chr2_C_albicans_SC5314	267403	279993	Ca21chr2_C_albicans_SC5314	1547799	1548998
Ca21chr2_C_albicans_SC5314	1611869	1622753	Ca21chr2_C_albicans_SC5314	1519257	1520960
Ca21chr2_C_albicans_SC5314	1479229	1490747	Ca21chr2_C_albicans_SC5314	1514712	1516178
Ca21chr2_C_albicans_SC5314	157814	166956	Ca21chr2_C_albicans_SC5314	897896	900774
Ca21chr2_C_albicans_SC5314	2148223	2149627	Ca21chr2_C_albicans_SC5314	890821	892818
Ca21chr2_C_albicans_SC5314	1041578	1051493	Ca21chr2_C_albicans_SC5314	588237	589598
Ca21chr2_C_albicans_SC5314	736894	745664	Ca21chr2_C_albicans_SC5314	557079	558713
Ca21chr2_C_albicans_SC5314	618550	627903	Ca21chr2_C_albicans_SC5314	7510	8043
Ca21chr2_C_albicans_SC5314	1116919	1125425	Ca21chr2_C_albicans_SC5314	922654	924717
Ca21chr2_C_albicans_SC5314	1262940	1271939	Ca21chr2_C_albicans_SC5314	1778986	1779687
Ca21chr2_C_albicans_SC5314	288630	296284	Ca21chr2_C_albicans_SC5314	795730	798201
Ca21chr2_C_albicans_SC5314	1250513	1258731	Ca21chr2_C_albicans_SC5314	766651	768309
Ca21chr2_C_albicans_SC5314	1499806	1508334	Ca21chr2_C_albicans_SC5314	763501	765159
Ca21chr2_C_albicans_SC5314	98269	105803	Ca21chr2_C_albicans_SC5314	758203	758733
Ca21chr2_C_albicans_SC5314	1604362	1611315	Ca21chr2_C_albicans_SC5314	700893	702539

This is a snippet of my data. What I want to do is to find out if the range of column 5 and column 6 is a subset of the range between column 2 and column 3. The data in column 2 and 3 are longer than data in column 5 and 6. A script has to scan through columns 2 and 3 in totality for every range defined by column 5 and 6. How do I do it. Any awk scripts? I am sorry if I did not follow the forum's rules, this is my first time using it.
Last edited by vbe; 01-03-2020 at 05:25 AM.. Reason: code tags - not quotes please
# 2  
Old 01-03-2020
You would need to compare $5 and $6 to all the ranges in $2, $3 so you would need to put them in memory first, so a way to do it would be to read the inputfile twice, the first time to put $2, $3 in memory, the second time to compare $5 and $6 to the ranges.

A simple first approach, assuming that $1 is always he same value could look something like this:

Code:
awk '
  NR==FNR {                                                    # When reading the file for the first time
    R[$2 FS $3]                                                # Store the ranges $2 to $3 in array R, using the standard field separator
    next
  }
  {                                                            # When reading the file for the second time
    for(i in R) {                                              # For every line, for every range
      split(i,F)                                               # Split the stored range in minimum and maximum using the standard field separator
      if(F[1]<=$5 && $5<=F[2])                                 # F[1] will contain the minimum, F[2] the maximum, so if $5 , $6 are inside it.
        print $0, "range " $5 "-" $6 " inside " F[1] "-" F[2]  # print the result
    }
  }
' infile infile                                                # read the file twice

With your data this should produce:
Code:
Ca21chr2_C_albicans_SC5314	618550	627903	Ca21chr2_C_albicans_SC5314	7510	8043 range 7510-8043 inside 5286-50509

Last edited by Scrutinizer; 01-03-2020 at 06:31 AM..
This User Gave Thanks to Scrutinizer For This Post:
cryptodice
# 3  
Old 01-03-2020
Hi
I'm a little confused. Is that necessary?
cat file
Code:
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	4	7
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	1	7
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	4	13
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	0	17
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	0	3
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	10	17
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	0	2
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	11	17

Code:
awk '
($5 >= $2 && $5 <= $3) ||
($6 <= $3 && $6 >= $2) ||
($5 < $2 && $6 > $3)    {print $0 RS ($2>$5?$2:$5) FS ($3>$6?$6:$3)}
' file

Code:
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	4	7
4 7
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	1	7
3 7
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	4	13
4 10
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	0	17
3 10
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	0	3
3 3
Ca21chr2_C_albicans_SC5314	3	10	Ca21chr2_C_albicans_SC5314	10	17
10 10

# 4  
Old 01-03-2020
Thank you so much for your solution. But I used the following code:
Code:
awk '
  NR==FNR {                                                    # When reading the file for the first time
    R[$2 FS $3]                                                # Store the ranges $2 to $3 in array R, using the standard field separator
    next
  }
  {                                                            # When reading the file for the second time
    for(i in R) {                                              # For every line, for every range
      split(i,F)                                               # Split the stored range in minimum and maximum using the standard field separator
      if(F[1]<=$5 && $5<=F[2])                                 # F[1] will contain the minimum, F[2] the maximum, so if $5 , $6 are inside it.
        print $0, "range " $5 "-" $6 " inside " F[1] "-" F[2]  # print the result
    }
  }
' h1.txt h1.txt

But it is returning me the following, NOT what you wrote above:
Code:
inside 5286-50509s_SC5314      618550  627903  Ca21chr2_C_albicans_SC5314      7510    8043

Last edited by Scrutinizer; 01-03-2020 at 04:19 PM.. Reason: quote tags -> code tags
# 5  
Old 01-03-2020
Try converting the input file to UNIX format first.
It appears to be in Windows format
Code:
tr -d '\r' <file >newfile


--
or you can maybe use
Code:
dos2unix file

if available on your OS.
# 6  
Old 01-03-2020
In awk you can strip the \r as follows
Code:
awk '
{ sub(/\r$/, "") }
...

# 7  
Old 01-03-2020
Similar approach to Scrutinizer's, but opens/reads file but once and keeps data in memory:
Code:
awk '
        {LN[NR]   = $0
         MIN2[NR] = $2
         MAX3[NR] = $3
         MIN5[NR] = $5
         MAX6[NR] = $6
        }
END     {for (i=1; i<=NR; i++)
           for (j=1; j<=NR; j++) if ((MIN5[i] >= MIN2[j]) && (MAX6[i] <= MAX3[j]))  print LN[i], "range", MIN5[i], "-", MAX6[i], "within", MIN2[j], "-", MAX3[j], "boundaries."
        }
' file
Ca21chr2_C_albicans_SC5314    618550    627903    Ca21chr2_C_albicans_SC5314    7510    8043 range 7510 - 8043 within 5286 - 50509 boundaries.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with sum range of data set together

Input File: 2000 3 1998 2 1997 2 1994 1 1991 1 1989 1 1987 2 1986 2 1985 1 1984 1 . . 10 277256 9 278274 8 282507 7 284837 6 287066 5 292967 (4 Replies)
Discussion started by: perl_beginner
4 Replies

2. Red Hat

Which is the effective ephemeral port range in Linux 2.6 for this set up?

In my Linux system ephemeral port range is showing different ranges as follows $ cat /proc/sys/net/ipv4/ip_local_port_range 32768 61000  cat /etc/sysctl.conf | grep net.ipv4.ip_local_port_range net.ipv4.ip_local_port_range = 9000 65500 Which will be the effective ephemeral port... (5 Replies)
Discussion started by: steephen
5 Replies

3. Shell Programming and Scripting

How to set end limit while copying files of a range??

I have files being generated in format A20140326.00........ to A20140326.24............. I need to copy these hourly basis from one location to another. Eg. If i copy from 14 to 19 the hour, I use wildcard as A201403226.1*. Requirement is : I need to copy from 06 hour and wil run the script... (1 Reply)
Discussion started by: Saidul
1 Replies

4. UNIX for Dummies Questions & Answers

How to subset data?

Hi. I have a large data file. the first column has unique identifiers. I have approximately 5 of these files and they have varying number of columns in their rows. I need to extract ~300 of the rows in to a separate file. I'm not looking for something that would do all 5 files at once, but... (7 Replies)
Discussion started by: kadm
7 Replies

5. Shell Programming and Scripting

Generate Regex numeric range with specific sub-ranges

hi all, Say i have a range like 0 - 1000 and i need to split into diffrent files the lines which are within a specific fixed sub-range. I can achieve this manually but is not scalable if the range increase. E.g cat file1.txt Response time 2 ms Response time 15 ms Response time 101... (12 Replies)
Discussion started by: varu0612
12 Replies

6. Shell Programming and Scripting

sed filtering lines by range fails 1-line-ranges

The following is part of a larger project and sed is (right now) a given. I am working on a recursive Korn shell function to "peel off" XML tags from a larger text. Just for context i will show the complete function (not working right now) here: function pGetXML { typeset chTag="$1" typeset... (5 Replies)
Discussion started by: bakunin
5 Replies

7. UNIX for Dummies Questions & Answers

how to get a subset of such a file

Dear all, I have a file lik below: n of row=420, n of letters in each row=100000 like below: there is no space between the letters. what I want is: the 75000th letter to the 85000th letter in each row. how to do that? thanks a lot! ... (2 Replies)
Discussion started by: forevertl
2 Replies

8. Shell Programming and Scripting

use variable to set the range of a for loop

Hi; For sure there's an easy answer to this one that I am not finding.. I first set a variable, say b1a:] max=5 then I want to use max to set the range for a for loop like so (it should run for i in 1:5) b1a:] for i in {1..$max}; do echo $i; done {1..5} I would like the output... (2 Replies)
Discussion started by: jbr950
2 Replies

9. UNIX for Dummies Questions & Answers

Help with subset and if-then statements

Hello everyone. I'm new to the boards, I hope I can get and possibly give some help through these forums. I need some help. I have two CSV files, let's call them File A and File B. This is the structure for File A: ID, VAR1, VAR2, VAR3 - VAR50 (where the VAR 1-VAR50 are either 0 or 1) ... (1 Reply)
Discussion started by: JWill
1 Replies

10. Shell Programming and Scripting

print range between two patterns if it contains a pattern within the range

I want to print between the range two patterns if a particular pattern is present in between the two patterns. I am new to Unix. Any help would be greatly appreciated. e.g. Pattern1 Bombay Calcutta Delhi Pattern2 Pattern1 Patna Madras Gwalior Delhi Pattern2 Pattern1... (2 Replies)
Discussion started by: joyan321
2 Replies
Login or Register to Ask a Question