Sponsored Content
Top Forums Shell Programming and Scripting awk to print unique text in field Post 302957261 by cmccabe on Thursday 8th of October 2015 01:17:49 PM
Old 10-08-2015
awk to print unique text in field

I am trying to use awk to print the unique entries in $2

So in the example below there are 3 lines but 2 of the lines match in $2 so only one is used in the output.


File.txt
Code:
chr17:29667512-29667673 NF1:exon.1;NF1:exon.2;NF1:exon.38;NF1:exon.4;NF1:exon.46;NF1:exon.47 703.807
chr16:89877104-89877220 FANCA:exon.4;FANCA:exon.5 159.284
chr16:89838075-89838232 FANCA:exon.23;FANCA:exon.4 583.497

Desired output
Code:
NF1
FANCA

Code:
awk '!seen[$2]++ {lines[i++]=$2}      END {for (i in lines) if (seen[lines[i]]==1) print lines[i]}' file.txt > output.txt

is close but uses the entire line not the unique text

Current output
Code:
NF1:exon.14;NF1:exon.16;NF1:exon.8
NF1:exon.13;NF1:exon.22;NF1:exon.28;NF1:exon.30;NF1:exon.4
CTC1:exon.1;CTC1:exon.16;CTC1:exon.2;CTC1:exon.3;CTC1:exon.5
CTC1:exon.1;CTC1:exon.10;CTC1:exon.2;CTC1:exon.3

Desired output
Code:
NF1
CTC1

Thank you Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

2. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the... (1 Reply)
Discussion started by: rocket_dog
1 Replies

3. Shell Programming and Scripting

Print unique records in 2 columns using awk

Is it possible to print the records that has only 1 value in 2nd column. Ex: input awex1 1 awex1 2 awex1 3 assww 1 ader34 1 ader34 2 output assww 1 (5 Replies)
Discussion started by: quincyjones
5 Replies

4. Shell Programming and Scripting

Print unique names in a specific column using awk

Is it possible to modify file like this. 1. Remove all the duplicate names in a define column i.e 4th col 2. Count the no.of unique names separated by ";" and print as a 5th col thanx in advance!! Q input c1 30 3 Eh2 c10 96 3 Frp c41 396 3 Ua5;Lop;Kol;Kol c62 2 30 Fmp;Fmp;Fmp ... (5 Replies)
Discussion started by: quincyjones
5 Replies

5. Shell Programming and Scripting

awk to parse field and include the text of 1 pipe in field 4

I am trying to parse the input in awk to include the |gc= in $4 but am not able to. The below is close: awk so far: awk '{sub(/\|]+]++/, ""); print }' input.txt Input chr1 955543 955763 AGRN-6|pr=2|gc=75 0 + chr1 957571 957852 AGRN-7|pr=3|gc=61.2 0 + chr1 970621 ... (7 Replies)
Discussion started by: cmccabe
7 Replies

6. Shell Programming and Scripting

awk to print unique text in field before hyphen

Trying to print the unique values in $2 before the -, currently the count is displayed. Hopefully, the below is close. Thank you :). file chr2:46603668-46603902 EPAS1-902|gc=54.3 253.1 chr2:211471445-211471675 CPS1-1205|gc=48.3 264.7 chr19:15291762-15291983 NOTCH3-1003|gc=68.8 195.8... (3 Replies)
Discussion started by: cmccabe
3 Replies

7. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than... (0 Replies)
Discussion started by: cmccabe
0 Replies

8. UNIX for Beginners Questions & Answers

Print lines based upon unique values in Nth field

For some reason I am having difficulty performing what should be a fairly easy task. I would like to print lines of a file that have a unique value in the first field. For example, I have a large data-set with the following excerpt: PS003,001 MZMWR/ L-DWD// * PS003,001... (4 Replies)
Discussion started by: jvoot
4 Replies

9. Shell Programming and Scripting

awk to print text in field if match and range is met

In the awk below I am trying to match the value in $4 of file1 with the split value from $4 in file2. I store the value of $4 in file1 in A and the split value (using the _ for the split) in array. I then strore the value in $2 as min, the value in $3 as max, and the value in $1 as chr. If A is... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. UNIX for Beginners Questions & Answers

Awk: count unique elements in a field and sum their occurence across the entire file

Hi, Sure it's an easy one, but it drives me insane. input ("|" separated): 1|A,B,C,A 2|A,D,D 3|A,B,B I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1. I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies
SIBsim4(1)							   User Manuals 							SIBsim4(1)

NAME
SIBsim4 - align RNA sequences with a DNA sequence, allowing for introns SYNOPSIS
SIBsim4 [ options ] dna rna_db DESCRIPTION
SIBsim4 is a similarity-based tool for aligning a collection of expressed sequences (EST, mRNA) with a genomic DNA sequence. Launching SIBsim4 without any arguments will print the options list, along with their default values. SIBsim4 employs a blast-based technique to first determine the basic matching blocks representing the "exon cores". In this first stage, it detects all possible exact matches of W-mers (i.e., DNA words of size W) between the two sequences and extends them to maximal scoring gap-free segments. In the second stage, the exon cores are extended into the adjacent as-yet-unmatched fragments using greedy alignment algorithms, and heuristics are used to favor configurations that conform to the splice-site recognition signals (e.g., GT-AG). If neces- sary, the process is repeated with less stringent parameters on the unmatched fragments. By default, SIBsim4 searches both strands and reports the best matches, measured by the number of matching nucleotides found in the align- ment. The R command line option can be used to restrict the search to one orientation (strand) only. Currently, four major alignment display options are supported, controlled by the A option. By default, only the endpoints, overall similar- ity, and orientation of the introns are reported. An arrow sign ('->' or '<-') indicates the orientation of the intron. The sign `==' marks the absence from the alignment of a cDNA fragment starting at that position. In the description below, the term MSP denotes a maximal scoring pair, that is, a pair of highly similar fragments in the two sequences, obtained during the blast-like procedure by extending a W-mer hit by matches and perhaps a few mismatches. OPTIONS
-A <int> output format 0: exon endpoints only 1: alignment text 3: both exon endpoints and alignment text 4: both exon endpoints and alignment text with polyA info Note that 2 is unimplemented. Default value is 0. -C <int> MSP score threshold for the second pass. Default value is 12. -c <int> minimum score cutoff value. Alignments which have scores below this value are not reported. Default value is 50. -E <int> cutoff value. Default value is 3. -f <int> score filter in percent. When multiple hits are detected for the same RNA element, only those having a score within this percentage of the maximal score for that RNA element are reported. Setting this value to 0 disables filtering and all hits will be reported, provided their score is above the cutoff value specified through the c option. Default value is 75. -g <int> join exons when gap on genomic and RNA have lengths which differ at most by this percentage. Default value is 10. -H <int> report chimeric transcripts when the best score is lower than this percentage of the overall RNA coverage and the chimera score is greater than this percentage of the RNA length (0 disables this report) Default value is 75. -I <int> window width in which to search for intron splicing. Default value is 6. -K <int> MSP score threshold for the first pass. Default value is 16. -L <str> a comma separated list of forward splice-types. Default value is "GTAG,GCAG,GTAC,ATAC". -M <int> scoring splice sites, evaluate match within M nucleotides. Default value is 10. -o <int> when printing results, offset nt positions in dna sequence by this amount. Default value is 0. -q <int> penalty for a nucleotide mismatch. Default value is -5. -R <int> direction of search 0: search the '+' (direct) strand only 1: search the '-' strand only 2: search both strands Default value is 2. -r <int> reward for a nucleotide match. Default value is 1. -S <int> splice site indels search breadth. While determining the best position of a splice site, SIBsim4 will evaluate adding at most this number of insertions and deletions on the DNA strand on each side of the splice junction. Default value is 2. -s <int> split score in percent. While linking MSP, if two consecutive group of exons appear like they could be part of two different copies of the same gene, they will be tested to see if the score of each individual group relative to the best overall score is greater than this value. If both groups have a relative score above this threshold they will be split. Default value is 75. -W <int> word size. Default value is 12. -X <int> value for terminating word extensions. Default value is 12. Bioinformatics April 2007 SIBsim4(1)
All times are GMT -4. The time now is 06:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy