12-14-2015
Extract spread columns from large file
Dear all,
I want to extract around 300 columns from a very large file with almost 2million columns. There are no headers, but I can find out which column numbers I want. I know I can extract with the function 'cut -f2' for example just the second column but how do I do this for such a large number of columns which are spread around the whole document?
Thank you in advance for your help.
Best regards,
Nicole
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All!!
I have a large file containing millions of record. My purpose is to extract 7 characters immediately after text '19' from this file (including text '19') and save the result in new file.
So, my OUTPUT would be as under :
191234561
194567894
192789005
198839408
and so on.....
... (7 Replies)
Discussion started by: parshant_bvcoe
7 Replies
2. Shell Programming and Scripting
Hello,
I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file.
What will be the besat and fastest way to extract the ne file.
sample file format :--... (2 Replies)
Discussion started by: learner16s
2 Replies
3. Shell Programming and Scripting
Hi All,
I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like
12345- , i tried with egrep ,eg : egrep "+" text.txt
but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies
4. Shell Programming and Scripting
I have done a couple of searches on this and have found many threads but I don't think I've found one that is useful to me - probably because I have very basic comprehension of perl and beginners shell so trying to manipulate a script already posted maybe beyond my capabilities....
Anyway - I... (26 Replies)
Discussion started by: Myrona
26 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I have a data file that contains 61 columns. I want to delete all the columns except columns, 3,6 and 8. The columns are tab de-limited. How would I achieve this on the terminal?
Thanks (2 Replies)
Discussion started by: lost.identity
2 Replies
6. Shell Programming and Scripting
Hi i have a php script that works 100% however i don't want this to run on php because of server limits etc. Ideally if i could convert this simple php script to a shell script i can set it up to run on a cron. My mac server has curl on it. So i am assuming i should be using this to download the... (3 Replies)
Discussion started by: timgolding
3 Replies
7. UNIX for Dummies Questions & Answers
Hi everyone. I have a file with ~500 columns and I would like to perform a simple calculation on every two columns. The file looks like this:
$cat input
id A B C D E F.....X
1 2 4 2 3 4 1 n
2 4 6 4 6 4 5 n
3 4 7 5 2 2 3 n
4 ... (5 Replies)
Discussion started by: torchij
5 Replies
8. Shell Programming and Scripting
Kindly help merging information from two files with the following data structure.
I want to match for the CHR-SNP in Foo and get the columns that match from CHROM-rsID
Fields 1 & 2 of foo may have duplicates, however, a joint key of Fields $1$2$3$4 is unique.
Also would be helpful to clean up... (4 Replies)
Discussion started by: genehunter
4 Replies
9. Shell Programming and Scripting
Hi All!!
I have a large file containing millions of records. My purpose is to extract 8 characters immediately from the given file.
222222222|ZRF|2008.pdf|2008|01/29/2009|001|B|C|C
222222222|ZRF|2009.pdf|2009|01/29/2010|001|B|C|C
222222222|ZRF|2010.pdf|2010|01/29/2011|001|B|C|C... (5 Replies)
Discussion started by: pavand
5 Replies
10. UNIX for Beginners Questions & Answers
Hi All,
I have records in unix file like below. In this file, we have empty fields from 4th Column to 22nd Column. I have some 200000 records in a file. I want to extract records only which have empty fields from 4th field to 22nd filed. This file is comma separated file. what is the unix... (2 Replies)
Discussion started by: rakeshp
2 Replies
COLUMN(1) BSD General Commands Manual COLUMN(1)
NAME
column -- columnate lists
SYNOPSIS
column [-tx] [-c columns] [-s sep] [file ...]
DESCRIPTION
The column utility formats its input into multiple columns. Rows are filled before columns. Input is taken from file operands, or, by
default, from the standard input. Empty lines are ignored.
The options are as follows:
-c Output is formatted for a display columns wide.
-s Specify a set of characters to be used to delimit columns for the -t option.
-t Determine the number of columns the input contains and create a table. Columns are delimited with whitespace, by default, or with
the characters supplied using the -s option. Useful for pretty-printing displays.
-x Fill columns before filling rows.
Column exits 0 on success, >0 if an error occurred.
ENVIRONMENT
COLUMNS The environment variable COLUMNS is used to determine the size of the screen if no other information is available.
EXAMPLES
(printf "PERM LINKS OWNER GROUP SIZE MONTH DAY HH:MM/YEAR NAME
"
; ls -l | sed 1d) | column -t
SEE ALSO
colrm(1), ls(1), paste(1), sort(1)
HISTORY
The column command appeared in 4.3BSD-Reno.
AVAILABILITY
The column command is part of the util-linux-ng package and is available from ftp://ftp.kernel.org/pub/linux/utils/util-linux-ng/.
BSD
June 6, 1993 BSD