Sponsored Content
Top Forums Shell Programming and Scripting Combining certain columns of multiple files into one file Post 303003068 by jgt on Thursday 7th of September 2017 10:05:40 AM
Old 09-07-2017
Quote:
Originally Posted by ksennin
Hello Unix gurus,

I have a large number of files (say X) each containing two columns of data and the same number of rows.

I would like to combine these files to create a unique merged file containing X columns corresponding to the second column of each file (with a bonus of having the first column of the first file only as a first column of the merged file).

I have only started recently to use bash scripting to try and do this, so I am quite new to this, but I have been trying "paste", or even "pr" as below:
Code:
Not the actual code ("..." signify a long list of file names or columns):

pr -m -t -s\  file1 file2 file3 [...] fileX | gawk '{print $1,$2,$4,$6,$8,[...],$X }' > merged.file

I don't really know what I am doing wrong, but when I don't get an error message ("pr: page width too narrow") it is insanely slow, even on a machine with loads of RAM. Would there be a better way to do this basic transformation?

Many thanks for your help and your time!

KS
read the man pages for the cut and paste commands.

---------- Post updated at 10:05 AM ---------- Previous update was at 10:01 AM ----------

Quote:
Originally Posted by jgt
read the man pages for the cut and paste commands.
use the cut command to retrieve only the second field in all but the first file. then paste all the resultant files to the first.
This User Gave Thanks to jgt For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

combining columns from different files

Hi all, I would be very grateful for some advice on the following. I have several text files. The files are experiment results with columns of data separated by white space. The files begin with several lines of header which are all preceeded by a comment character '#'. Each file has a... (10 Replies)
Discussion started by: iomaire
10 Replies

2. Shell Programming and Scripting

Combining columns from different files

I have two files I need to combine. The problem I'm having is I need to only combine data from the second file in the empty spaces of the first. For example: file1 Data Field Data Field Data Field Data Field file2 a - Insert Data b - Insert Data c - Insert Data d - Insert Data... (10 Replies)
Discussion started by: handband2
10 Replies

3. UNIX for Dummies Questions & Answers

Combining two text files as columns?

I have one space delimited file with multiple columns and one tab delimited file with multiple columns (They have the same number of rows). I want to basically combine these two text files into a new text file by column. How would I go about doing that? (1 Reply)
Discussion started by: evelibertine
1 Replies

4. Shell Programming and Scripting

Combining columns from multiple files to one file

I'm trying to combine colums from multiple file to a single file but having some issues, appreciate your help. The filenames are the same except for the extension, path1.m0 --------- a b c d e f g h i path1.m1 --------- m n o p q r s t u File names are path1.m The... (3 Replies)
Discussion started by: rkmca
3 Replies

5. Shell Programming and Scripting

Combining multiple column files into one with file name as first row

Hello All, I have several column files like this $cat a_b_s1.xls 1wert 2tg 3asd 4asdf 5asdf $cat c_d_s2.xls 1wert 2tg 3asd 4asdf 5asdf desired put put $cat combined.txt s1 s2 (2 Replies)
Discussion started by: avatar_007
2 Replies

6. Shell Programming and Scripting

Combining multiple files

I have 2 files. each having 3 coloums 1st field date as 20130322 2nd field time as 05:55 3rd field numberic value File 2 has entries missing for some date time. FILE1 20130322 05:35 2219 20130322 05:40 1809 20130322 05:45 1617 20130322 05:50 ... (2 Replies)
Discussion started by: sandeepkmehra
2 Replies

7. Shell Programming and Scripting

Combining columns from multiple files into one single output file

Hi, I have 3 files with one column value as shown File: a.txt ------------ Data_a1 Data_a2 File2: b.txt ------------ Data_b1 Data_b2 Data_b3 Data_b4 File3: c.txt ------------ Data_c1 Data_c2 Data_c3 Data_c4 Data_c5 (6 Replies)
Discussion started by: vfrg
6 Replies

8. Linux

[Solved] Combining columns from different files

Hey Guys & Gals, I am stuck with the following ; I have 2 text files, each containing 2 columns. My goal is to have a column from the 2nd file placed inbetween the columns in the first file. Basically the idea is, each address has a different name (but 1 name per address) but 1 address... (6 Replies)
Discussion started by: TAPE
6 Replies

9. Shell Programming and Scripting

Join two files combining multiple columns and produce mix and match output

I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns. Like I have file A 1234,ABCD,23,JOHN,NJ,USA 2345,ABCD,24,SAM,NY,USA 5678,GHIJ,24,TOM,NY,USA 5678,WXYZ,27,MAT,NJ,USA and file B ... (2 Replies)
Discussion started by: mady135
2 Replies

10. UNIX for Beginners Questions & Answers

Combining multiple files into one

Hello Everyone, I have 4 different files (one column in each) that I'm trying to combine into 1 file with four columns. Having issues trying to get the columns to format properly. I have tried the following: paste file1 file2 file3 file4 | column -s $'\t' -t > results.txt paste file1 file2... (1 Reply)
Discussion started by: malk71
1 Replies
MPSMatrixDescriptor(3)					 MetalPerformanceShaders.framework				    MPSMatrixDescriptor(3)

NAME
MPSMatrixDescriptor SYNOPSIS
#import <MPSMatrixTypes.h> Inherits NSObject. Class Methods (__nonnull instancetype) + matrixDescriptorWithDimensions:columns:rowBytes:dataType: (__nonnull instancetype) + matrixDescriptorWithRows:columns:rowBytes:dataType: (__nonnull instancetype) + matrixDescriptorWithRows:columns:matrices:rowBytes:matrixBytes:dataType: (size_t) + rowBytesFromColumns:dataType: (size_t) + rowBytesForColumns:dataType: Properties NSUInteger rows NSUInteger columns NSUInteger matrices MPSDataType dataType NSUInteger rowBytes NSUInteger matrixBytes Detailed Description This depends on Metal.framework A MPSMatrixDescriptor describes the sizes, strides, and data type of a an array of 2-dimensional matrices. All storage is assumed to be in 'matrix-major'. See the description for MPSMatrix for further details. Method Documentation + (__nonnull instancetype) matrixDescriptorWithDimensions: (NSUInteger) rows(NSUInteger) columns(NSUInteger) rowBytes(MPSDataType) dataType Create a MPSMatrixDescriptor with the specified dimensions and data type. Parameters: rows The number of rows of the matrix. columns The number of columns of the matrix. rowBytes The number of bytes between starting elements of consecutive rows. Must be a multiple of the element size. dataType The type of the data to be stored in the matrix. For performance considerations the optimal row stride may not necessarily be equal to the number of columns in the matrix. The MPSMatrix class provides a method which may be used to determine this value, see the rowBytesForColumns API in the MPSMatrix class. The number of matrices described is initialized to 1. + (__nonnull instancetype) matrixDescriptorWithRows: (NSUInteger) rows(NSUInteger) columns(NSUInteger) matrices(NSUInteger) rowBytes(NSUInteger) matrixBytes(MPSDataType) dataType Create a MPSMatrixDescriptor with the specified dimensions and data type. Parameters: rows The number of rows of a single matrix. columns The number of columns of a single matrix. matrices The number of matrices in the MPSMatrix object. rowBytes The number of bytes between starting elements of consecutive rows. Must be a multiple of the element size. matrixBytes The number of bytes between starting elements of consecutive matrices. Must be a multiple of rowBytes. dataType The type of the data to be stored in the matrix. For performance considerations the optimal row stride may not necessarily be equal to the number of columns in the matrix. The MPSMatrix class provides a method which may be used to determine this value, see the rowBytesForColumns API in the MPSMatrix class. + (__nonnull instancetype) matrixDescriptorWithRows: (NSUInteger) rows(NSUInteger) columns(NSUInteger) rowBytes(MPSDataType) dataType + (size_t) rowBytesForColumns: (NSUInteger) columns(MPSDataType) dataType + (size_t) rowBytesFromColumns: (NSUInteger) columns(MPSDataType) dataType Return the recommended row stride, in bytes, for a given number of columns. Parameters: columns The number of columns in the matrix for which the recommended row stride, in bytes, is to be determined. dataType The type of matrix data values. To achieve best performance the optimal stride between rows of a matrix is not necessarily equivalent to the number of columns. This method returns the row stride, in bytes, which gives best performance for a given number of columns. Using this row stride to construct your array is recommended, but not required (provided that the stride used is still large enough to allocate a full row of data). Property Documentation - columns [read], [write], [nonatomic], [assign] The number of columns in a matrix. - dataType [read], [write], [nonatomic], [assign] The type of the data which makes up the values of the matrix. - matrices [read], [nonatomic], [assign] The number of matrices. - matrixBytes [read], [nonatomic], [assign] The stride, in bytes, between corresponding elements of consecutive matrices. Must be a multiple of rowBytes. - rowBytes [read], [write], [nonatomic], [assign] The stride, in bytes, between corresponding elements of consecutive rows. Must be a multiple of the element size. - rows [read], [write], [nonatomic], [assign] The number of rows in a matrix. Author Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code. Version MetalPerformanceShaders-100 Thu Feb 8 2018 MPSMatrixDescriptor(3)
All times are GMT -4. The time now is 05:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy