12-14-2015
Extract spread columns from large file
Dear all,
I want to extract around 300 columns from a very large file with almost 2million columns. There are no headers, but I can find out which column numbers I want. I know I can extract with the function 'cut -f2' for example just the second column but how do I do this for such a large number of columns which are spread around the whole document?
Thank you in advance for your help.
Best regards,
Nicole
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All!!
I have a large file containing millions of record. My purpose is to extract 7 characters immediately after text '19' from this file (including text '19') and save the result in new file.
So, my OUTPUT would be as under :
191234561
194567894
192789005
198839408
and so on.....
... (7 Replies)
Discussion started by: parshant_bvcoe
7 Replies
2. Shell Programming and Scripting
Hello,
I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file.
What will be the besat and fastest way to extract the ne file.
sample file format :--... (2 Replies)
Discussion started by: learner16s
2 Replies
3. Shell Programming and Scripting
Hi All,
I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like
12345- , i tried with egrep ,eg : egrep "+" text.txt
but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies
4. Shell Programming and Scripting
I have done a couple of searches on this and have found many threads but I don't think I've found one that is useful to me - probably because I have very basic comprehension of perl and beginners shell so trying to manipulate a script already posted maybe beyond my capabilities....
Anyway - I... (26 Replies)
Discussion started by: Myrona
26 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I have a data file that contains 61 columns. I want to delete all the columns except columns, 3,6 and 8. The columns are tab de-limited. How would I achieve this on the terminal?
Thanks (2 Replies)
Discussion started by: lost.identity
2 Replies
6. Shell Programming and Scripting
Hi i have a php script that works 100% however i don't want this to run on php because of server limits etc. Ideally if i could convert this simple php script to a shell script i can set it up to run on a cron. My mac server has curl on it. So i am assuming i should be using this to download the... (3 Replies)
Discussion started by: timgolding
3 Replies
7. UNIX for Dummies Questions & Answers
Hi everyone. I have a file with ~500 columns and I would like to perform a simple calculation on every two columns. The file looks like this:
$cat input
id A B C D E F.....X
1 2 4 2 3 4 1 n
2 4 6 4 6 4 5 n
3 4 7 5 2 2 3 n
4 ... (5 Replies)
Discussion started by: torchij
5 Replies
8. Shell Programming and Scripting
Kindly help merging information from two files with the following data structure.
I want to match for the CHR-SNP in Foo and get the columns that match from CHROM-rsID
Fields 1 & 2 of foo may have duplicates, however, a joint key of Fields $1$2$3$4 is unique.
Also would be helpful to clean up... (4 Replies)
Discussion started by: genehunter
4 Replies
9. Shell Programming and Scripting
Hi All!!
I have a large file containing millions of records. My purpose is to extract 8 characters immediately from the given file.
222222222|ZRF|2008.pdf|2008|01/29/2009|001|B|C|C
222222222|ZRF|2009.pdf|2009|01/29/2010|001|B|C|C
222222222|ZRF|2010.pdf|2010|01/29/2011|001|B|C|C... (5 Replies)
Discussion started by: pavand
5 Replies
10. UNIX for Beginners Questions & Answers
Hi All,
I have records in unix file like below. In this file, we have empty fields from 4th Column to 22nd Column. I have some 200000 records in a file. I want to extract records only which have empty fields from 4th field to 22nd filed. This file is comma separated file. what is the unix... (2 Replies)
Discussion started by: rakeshp
2 Replies
COLRM(1) BSD General Commands Manual COLRM(1)
NAME
colrm -- remove columns from a file
SYNOPSIS
colrm [start [stop]]
DESCRIPTION
The colrm utility removes selected columns from the lines of a file. A column is defined as a single character in a line. Input is read
from the standard input. Output is written to the standard output.
If only the start column is specified, columns numbered less than the start column will be written. If both start and stop columns are spec-
ified, columns numbered less than the start column or greater than the stop column will be written. Column numbering starts with one, not
zero.
Tab characters increment the column count to the next multiple of eight. Backspace characters decrement the column count by one.
ENVIRONMENT
The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7).
EXIT STATUS
The colrm utility exits 0 on success, and >0 if an error occurs.
SEE ALSO
awk(1), column(1), cut(1), paste(1)
HISTORY
The colrm command appeared in 3.0BSD.
BSD
August 4, 2004 BSD