Here is the top of the pattern file:

Chr1 711
Chr1 892
Chr1 956
Chr1 10904
Chr1 32210
Chr1 37388
Chr1 49438
Chr1 71326
Chr1 71348
Chr1 88300
Chr1 90571
Chr1 90606
Chr1 90809
Chr1 90864
Chr1 96770
Chr1 97473

Here is the top of the Master file I'm trying to grep out of

Chr1 3658 A 1 ^M. C
Chr1 3659 A 1 . C
Chr1 3660 A 1 . C
Chr1 3661 C 2 .^M. F@
Chr1 3662 A 2 .. F@
Chr1 3663 A 2 .. F@
Chr1 3664 A 2 .. F@
Chr1 3665 T 2 .. FD
Chr1 3666 A 2 .. HD
Chr1 3667 C 2 .. HD
Chr1 3668 A 4 ..^K,^K. H:.$
Chr1 3669 T 4 ..,. GA1*
Chr1 3670 A 4 ..,. HCD9
Chr1 3671 A 4 ..,. GDDJ
Chr1 3672 T 4 ..,. JFBI
Chr1 3673 C 4 ..,. JBDJ
Chr1 3674 G 6 ..,.^:,^:. IBDJ?=
Chr1 3675 G 6 ..,.,. J8DHDJ



So I have a 12GB file and I want to pull out the lines that correspond to about 350,000 specific positions on the chromosomes. Each is a separate line.

I'm using Ubuntu. The Master file is a .bcf made with samtools and the coordinates (pattern file) was made from a table in a paper and is tab delimited. I added a third column with just a tab so as not to pull out multiple entries beginning with the search pattern eg only want Chr1 3675 pulling not other entries like Chr1 367567 for example.

This starts to ouput the lines with the correct patterns to the terminal within a few seconds:

Code:

cat C2L4_all.bcf | grep -f lertest.txt

But this just greps indefinately, creates the output file but it stays empty. After about 10 min I kill the command.

Code:

cat C2L4_all.bcf | grep -f lertest.txt > plop.txt

I tried the script command to copy the terminal to a file but it has all kinds if formatting info in it as well. Must be an easier way.

---------- Post updated at 01:29 AM ---------- Previous update was at 01:21 AM ----------

Should I let it run overnight? Its just the non-write to file version only takes a few seconds!!

I'm wondering if it scans the whole 12 GB file before writing to file whereas if it finds something it may instantly display it in the terminal?

Seems odd if this is the case though!