Extracting columns from a matrix and storing each column in a separate file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extracting columns from a matrix and storing each column in a separate file
# 1  
Old 04-21-2011
Extracting columns from a matrix and storing each column in a separate file

Hi All,

I have a huge matrix file consisting some some millions rows and 6000 columns. The contents are just floating point numbers in the matrix. I want to extract each column (i.e. 6000 of them) and store each column in a separate file.
For example, 1.dat will consist of elements from column 1. 2.dat will have the numbers from the 2nd column and so on. I have tried this but it does not work:
Code:
awk '{print $c}{c++}{print > c."dat"}' tdm.mtx

I am using Linux.
# 2  
Old 04-21-2011
What is the field separator of your file?
This User Gave Thanks to Shell_Life For This Post:
# 3  
Old 04-21-2011
Sorry, totally forgot...separated by spaces...the matrix is something like this

Code:
1 2 3
4 .005 6
6 7 .08

# 4  
Old 04-21-2011
try this
Code:
#/bin/ksh 
set -A row `head -1 file.txt`
num_col=`echo ${#row[*]}`
i=1
while (( i <= ${num_col} ))
do
 cut -d ' ' -f${i} file.txt  > ${i}.dat
(( i += 1 ))
done

This User Gave Thanks to max_hammer For This Post:
# 5  
Old 04-21-2011
Here is an awk :
Code:
awk '{ for (i=1;i <= NF;i++) print $i > i".dat" }' file.txt

This User Gave Thanks to Peasant For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Transpose matrix, and rearrange columns common with another file

This is my first post, I apologize if I have broken rules. Some assistance with the following will be very helpful. I have a couple of files, both should ultimately have common columns only, arranged in the same order. This file needs to be transposed, to bring the rows to columns ... (2 Replies)
Discussion started by: abh.kumar
2 Replies

2. Shell Programming and Scripting

How to parse parts of 1 column into two separate columns?

I have a shell script that is currently transferring a csv file from a server into a Teradata database table. One of the 30 or so columns is called "destination_url". In that URL there are parameters, and it is possible for those parameters to be repeated because of referring companies copying... (3 Replies)
Discussion started by: craigwg
3 Replies

3. Shell Programming and Scripting

Help w/ Reading Matrix & Storing in dynamic array

First of I would just like to state that I am not looking for you guys to just do my work for me, I do want to learn and actually understand everything that is happening. Hey all, I am having trouble on this. What I need to do is... Write an executable C file that will take a text file (not a... (8 Replies)
Discussion started by: innvert
8 Replies

4. UNIX for Dummies Questions & Answers

How to generate one long column by merging two separate two columns in a single file?

Dear all, I have a simple question. I have a file like below (separated by tab): col1 col2 col3 col4 col5 col6 col7 21 66745 rs1234 21 rs5678 23334 0.89 21 66745 rs2334 21 rs9978 23334 0.89 21 66745 ... (4 Replies)
Discussion started by: forevertl
4 Replies

5. Shell Programming and Scripting

Need help with awk statement to break nth column in csv file into 3 separate columns

Hello Members, I have a csv file in the format below. Need help with awk statement to break nth column into 3 separate columns and export the changes to new file. input file --> file.csv cat file.csv|less "product/fruit/mango","location/asia/india","type/alphonso" need output in... (2 Replies)
Discussion started by: awk-admirer
2 Replies

6. Shell Programming and Scripting

Extracting rows and columns in a matrix based on condition

Hi I have a matrix with n rows and m columns like below example. i want to extract all the pairs with values <200. Input A B C D A 100 206 51 300 B 206 100 72 48 C 351 22 100 198 D 13 989 150 100 Output format A,A:200 A,C:51 B,B:100... (2 Replies)
Discussion started by: anurupa777
2 Replies

7. Shell Programming and Scripting

Convert a 3 column tab delimited file to a matrix

Hi all, I have a 3 columns input file like this: CPLX9PC-4943 CPLX9PC-4943 1 CPLX9PC-4943 CpxID123 0 CPLX9PC-4943 CpxID126 0 CPLX9PC-4943 CPLX9PC-5763 0.5 CPLX9PC-4943 CpxID13 0 CPLX9PC-4943 CPLX9PC-6163 0 CPLX9PC-4943 CPLX9PC-6164 0.04... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

8. Shell Programming and Scripting

Perl- creating a matrix from a 3 column file

Dear all, I'm new in perl scripting and I'm trying to creating a matrix from a 3 column file sorting data in a particular manner. In the final matrix I need to have the first column "IDs" on the header of the columns and the second column values on the header of each row. And the value fo the... (2 Replies)
Discussion started by: gabrysfe
2 Replies

9. UNIX Desktop Questions & Answers

Extracting data from tables and storing in a file

Hi I am trying to write a script to extract information from a database and save that info to a csv file. I am using sql, an oracle database I have no idea how to even begin this. Can somebody please help. (8 Replies)
Discussion started by: ladyAnne
8 Replies

10. Shell Programming and Scripting

Extracting particular string in a file and storing matched string in output file

Hi , I have input file and i want to extract below strings <msisdn xmlns="">0492001956</ msisdn> => numaber inside brackets <resCode>3000</resCode> => 3000 needs to be extracted <resMessage>Request time getBalances_PSM.c(37): d out</resMessage></ns2:getBalancesResponse> => the word... (14 Replies)
Discussion started by: sushmab82
14 Replies
Login or Register to Ask a Question