03-18-2011
Yeah infact you are right, I made a mistake about 3 being unique to file2, was meant to be 2.
Thanks a lot for the useful input, however my original .gff files to be analysed have huge/long lines and thus the output of your code is not recognisable as tab delimited file specific entires. Is there a way to arrange them in long format where file spepcific entries are separated by multiple new lines / hashes / dots / undescores for the sake of easy reading ?
Thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I have a bash shell script and I use awk to print certain columns of one file and direct the output to another file. If I do a less or cat on the file it looks correct, but if I email the file and open it with Outlook the lines outputted by awk are concatenated.
Here is my awk line:... (6 Replies)
Discussion started by: xadamz23
6 Replies
2. Shell Programming and Scripting
Hi,
I have 2 files
file1
********
01-05-09|java.xls|
02-05-08|c.txt|
08-01-09|perl.txt|
01-01-09|oracle.txt|
********
file2
********
01-02-09|windows.xls|
02-05-08|c.txt|
01-05-09|java.xls|
08-02-09|perl.txt|
01-01-09|oracle.txt|
******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies
3. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
4. Shell Programming and Scripting
Hi friends,
I have multiple files. For now, let's say I have two of the following style
cat 1.txt
cat 2.txt
output.txt
Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies
5. UNIX for Dummies Questions & Answers
hi
i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command?
my... (1 Reply)
Discussion started by: anurupa777
1 Replies
6. Shell Programming and Scripting
hi
my problem is little complicated one. i have 2 files which appear like this
file 1
abbsss:aa:22:34:as akl abc 1234
mkilll:as:ss:23:qs asc abc 0987
mlopii:cd:wq:24:as asd abc 7866
file2
lkoaa:as:24:32:sa alk abc 3245
lkmo:as:34:43:qs qsa abc 0987
kloia:ds:45:56:sa acq abc 7805
i... (5 Replies)
Discussion started by: anurupa777
5 Replies
7. Shell Programming and Scripting
Hello everyone,
Maybe somebody could help me with an awk script.
I have this input (field separator is comma ","):
547894982,M|N|J,U|Q|P,98,101,0,1,1
234900027,M|N|J,U|Q|P,98,101,0,1,1
234900023,M|N|J,U|Q|P,98,54,3,1,1
234900028,M|H|J,S|Q|P,98,101,0,1,1
234900030,M|N|J,U|F|P,98,101,0,1,1... (2 Replies)
Discussion started by: Ophiuchus
2 Replies
8. Shell Programming and Scripting
file 1
Sun Mar 17 00:01:33 2013 submit , Name="1234"
Sun Mar 17 00:01:33 2013 submit , Name="1344"
Sun Mar 17 00:01:33 2013 submit , Name="1124"
..
..
..
..
Sun Mar 17 00:01:33 2013 submit , Name="8901"
file 2
Sun Mar 17 00:02:47 2013 1234 execute SUCCEEDED
Sun Mar 17... (24 Replies)
Discussion started by: aravindj80
24 Replies
9. UNIX for Dummies Questions & Answers
I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies
10. UNIX for Beginners Questions & Answers
I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with:
wc -l *|sort
15263 Image.txt
16401 reference.txt
40459 richtexteditor.txt
How can I also print the number of unique lines in each file?
15263 1401 Image.txt
16401... (15 Replies)
Discussion started by: spacegoose
15 Replies
tabix(1) Bioinformatics tools tabix(1)
NAME
bgzip - Block compression/decompression utility
tabix - Generic indexer for TAB-delimited genome position files
SYNOPSIS
bgzip [-cdhB] [-b virtualOffset] [-s size] [file]
tabix [-0lf] [-p gff|bed|sam|vcf] [-s seqCol] [-b begCol] [-e endCol] [-S lineSkip] [-c metaChar] in.tab.bgz [region1 [region2 [...]]]
DESCRIPTION
Tabix indexes a TAB-delimited genome position file in.tab.bgz and creates an index file in.tab.bgz.tbi when region is absent from the com-
mand-line. The input data file must be position sorted and compressed by bgzip which has a gzip(1) like interface. After indexing, tabix is
able to quickly retrieve data lines overlapping regions specified in the format "chr:beginPos-endPos". Fast data retrieval also works over
network if URI is given as a file name and in this case the index file will be downloaded if it is not present locally.
OPTIONS OF TABIX
-p STR Input format for indexing. Valid values are: gff, bed, sam, vcf and psltab. This option should not be applied together with any
of -s, -b, -e, -c and -0; it is not used for data retrieval because this setting is stored in the index file. [gff]
-s INT Column of sequence name. Option -s, -b, -e, -S, -c and -0 are all stored in the index file and thus not used in data retrieval.
[1]
-b INT Column of start chromosomal position. [4]
-e INT Column of end chromosomal position. The end column can be the same as the start column. [5]
-S INT Skip first INT lines in the data file. [0]
-c CHAR Skip lines started with character CHAR. [#]
-0 Specify that the position in the data file is 0-based (e.g. UCSC files) rather than 1-based.
-h Print the header/meta lines.
-B The second argument is a BED file. When this option is in use, the input file may not be sorted or indexed. The entire input will
be read sequentially. Nonetheless, with this option, the format of the input must be specificed correctly on the command line.
-f Force to overwrite the index file if it is present.
-l List the sequence names stored in the index file.
EXAMPLE
(grep ^"#" in.gff; grep -v ^"#" in.gff | sort -k1,1 -k4,4n) | bgzip > sorted.gff.gz;
tabix -p gff sorted.gff.gz;
tabix sorted.gff.gz chr1:10,000,000-20,000,000;
NOTES
It is straightforward to achieve overlap queries using the standard B-tree index (with or without binning) implemented in all SQL data-
bases, or the R-tree index in PostgreSQL and Oracle. But there are still many reasons to use tabix. Firstly, tabix directly works with a
lot of widely used TAB-delimited formats such as GFF/GTF and BED. We do not need to design database schema or specialized binary formats.
Data do not need to be duplicated in different formats, either. Secondly, tabix works on compressed data files while most SQL databases do
not. The GenCode annotation GTF can be compressed down to 4%. Thirdly, tabix is fast. The same indexing algorithm is known to work effi-
ciently for an alignment with a few billion short reads. SQL databases probably cannot easily handle data at this scale. Last but not the
least, tabix supports remote data retrieval. One can put the data file and the index at an FTP or HTTP server, and other users or even web
services will be able to get a slice without downloading the entire file.
AUTHOR
Tabix was written by Heng Li. The BGZF library was originally implemented by Bob Handsaker and modified by Heng Li for remote file access
and in-memory caching.
SEE ALSO
samtools(1)
tabix-0.2.0 11 May 2010 tabix(1)