Getting non unique lines from concatenated files Post: 302506712

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Getting non unique lines from concatenated files Post 302506712 by pawannoel on Monday 21st of March 2011 06:15:32 PM

03-21-2011

Registered User

Hi Bartus11,

I'm not sure of this code is working !! ... I get the following result .... just showing u the first few lines ...

Code:

pawan-noels-computer:/Volumes/USB/test noel$ perl -lne '$h{$.}=$_;END{@o=sort{(split "=",(split ";",(split " +",$h{$a})[8])[2])[1]<=>(split "=",(split ";",(split " +",$h{$b})[8])[2])[1]}keys %h; for $i (@o){print $h{$i}}}' file_1
chr15   levure5 SNP     30924   30924   0.000000        .       .       genotype=R;reference=A;coverage=43;refAlleleCounts=23;refAlleleStarts=16;refAlleleMeanQV=17;novelAlleleCounts=19;novelAlleleStarts=11;novelAlleleMeanQV=19;diColor1=01;diColor2=23;het=1;flag=
chr02   levure7 SNP     792879  792879  0.010875        .       .       genotype=M;reference=A;coverage=24;refAlleleCounts=19;refAlleleStarts=14;refAlleleMeanQV=15;novelAlleleCounts=4;novelAlleleStarts=3;novelAlleleMeanQV=26;diColor1=23;diColor2=32;het=1;flag=;gene;ID=YBR298C;Name=YBR298C;gene=MAL31;Alias=MAL31,MALT,MAL3T
chr02   levure7 SNP     459336  459336  0.000000        .       .       genotype=Y;reference=T;coverage=15;refAlleleCounts=9;refAlleleStarts=7;refAlleleMeanQV=25;novelAlleleCounts=6;novelAlleleStarts=6;novelAlleleMeanQV=22;diColor1=12;diColor2=30;het=1;flag=;gene;ID=YBR115C;Name=YBR115C;gene=LYS2;Alias=LYS2
chr10   levure6 SNP     39633   39633   0.000000        .       .       genotype=T;reference=A;coverage=9;refAlleleCounts=0;refAlleleStarts=0;refAlleleMeanQV=0;novelAlleleCounts=9;novelAlleleStarts=5;novelAlleleMeanQV=18;diColor1=23;diColor2=23;het=0;flag=h4,h10,;gene;ID=YJL211C;Name=YJL211C
chr15   levure6 SNP     30924   30924   0.000000        .       .       genotype=R;reference=A;coverage=36;refAlleleCounts=26;refAlleleStarts=19;refAlleleMeanQV=14;novelAlleleCounts=10;novelAlleleStarts=8;novelAlleleMeanQV=20;diColor1=01;diColor2=23;het=1;flag=

pawannoel

View Public Profile for pawannoel

Find all posts by pawannoel

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Lines Concatenated with awk

Hello, I have a bash shell script and I use awk to print certain columns of one file and direct the output to another file. If I do a less or cat on the file it looks correct, but if I email the file and open it with Outlook the lines outputted by awk are concatenated. Here is my awk line:...

2. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ********

3. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to...

4. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100...

5. UNIX for Dummies Questions & Answers

getting unique lines from 2 files

hi i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command? my...

6. Shell Programming and Scripting

compare 2 files and return unique lines in each file (based on condition)

hi my problem is little complicated one. i have 2 files which appear like this file 1 abbsss:aa:22:34:as akl abc 1234 mkilll:as:ss:23:qs asc abc 0987 mlopii:cd:wq:24:as asd abc 7866 file2 lkoaa:as:24:32:sa alk abc 3245 lkmo:as:34:43:qs qsa abc 0987 kloia:ds:45:56:sa acq abc 7805 i...

7. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1...

8. Shell Programming and Scripting

Look up 2 files and print the concatenated output

file 1 Sun Mar 17 00:01:33 2013 submit , Name="1234" Sun Mar 17 00:01:33 2013 submit , Name="1344" Sun Mar 17 00:01:33 2013 submit , Name="1124" .. .. .. .. Sun Mar 17 00:01:33 2013 submit , Name="8901" file 2 Sun Mar 17 00:02:47 2013 1234 execute SUCCEEDED Sun Mar 17...

9. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks.

10. UNIX for Beginners Questions & Answers

Print number of lines for files in directory, also print number of unique lines

I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with: wc -l *|sort 15263 Image.txt 16401 reference.txt 40459 richtexteditor.txt How can I also print the number of unique lines in each file? 15263 1401 Image.txt 16401...

LEARN ABOUT DEBIAN

paps

PAPS(1) 						      General Commands Manual							   PAPS(1)

NAME

       paps - UTF-8 to PostScript converter using Pango

SYNOPSIS

       paps [options] files...

DESCRIPTION

       paps reads a UTF-8 encoded file and generates a PostScript language rendering of the file. The rendering is done by creating outline curves
       through the pango ft2 backend.

OPTIONS

       These programs follow the usual GNU command line syntax, with long options starting with  two  dashes  (`-').   A  summary  of  options	is
       included below.

       --landscape
	      Landscape output. Default is portrait.

       --columns=cl
	      Number of columns output. Default is 1.
	      Please notice this option isn't related to the terminal length as in a "80 culums terminal".

       --font=desc
	      Set the font description. Default is Monospace 12.

       --rtl  Do right to left (RTL) layout.

       --paper ps
	      Choose paper size. Known paper sizes are legal, letter and A4. Default is A4.

       Postscript points
	      Each postscript point equals to 1/72 of an inch. 36 points are 1/2 of an inch.

       --bottom-margin=bm
	      Set bottom margin. Default is 36 postscript points.

       --top-margin=tm
	      Set top margin. Default is 36 postscript points.

       --left-margin=lm
	      Set left margin. Default is 36 postscript points.

       --right-margin=rm
	      Set right margin. Default is 36 postscript points.

       --gutter-width=gw
	      Set gutter width. Default is 40 postscript points.

       --help Show summary of options.

       --header
	      Draw page header for each page.

       --markup
	      Interpret the text as pango markup.

       --lpi  Set the lines per inch. This determines the line spacing.

       --cpi  Set the characters per inch. This is an alternative method of specifying the font size.

       --stretch-chars
	      Indicates  that  characters  should be stretched in the y-direction to fill up their vertical space. This is similar to the texttops
	      behaviour.

AUTHOR

       paps was written by Dov Grobgeld <dov.grobgeld@gmail.com>.

       This manual page was written by Lior Kaplan <kaplan@debian.org>, for the Debian project (but may be used by others).

								  April  17, 2006							   PAPS(1)