Sponsored Content
Top Forums Shell Programming and Scripting combining columns from different files Post 82899 by iomaire on Wednesday 7th of September 2005 01:57:24 PM
Old 09-07-2005
combining columns from different files

Hi all,
I would be very grateful for some advice on the following.

I have several text files. The files are experiment results with columns of data separated by white space.

The files begin with several lines of header which are all preceeded by a comment character '#'.

Each file has a record number column as its first column and then several columns of values relating to that record. The record numbers go from 1-N in each experiment file

I would like to compare a particular column across several experiment files.

How to I take two of these experiment files and create a new file such that the new file contains:
- none of the header information
- the first column is the record column
- the next column is ,say column 2 from the first file
- the next column is the same column 2 from the second file

I've looked into awk and join but am not sure how to use them in combination to achieve what I want.

Thanks,
Enda

Sample file
File1.dat:
################
# Header info
################
1 10 20 30
2 13 50 13
3 ....
etc
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ... (4 Replies)
Discussion started by: manneni prakash
4 Replies

2. Shell Programming and Scripting

Combining columns from different files

I have two files I need to combine. The problem I'm having is I need to only combine data from the second file in the empty spaces of the first. For example: file1 Data Field Data Field Data Field Data Field file2 a - Insert Data b - Insert Data c - Insert Data d - Insert Data... (10 Replies)
Discussion started by: handband2
10 Replies

3. UNIX for Dummies Questions & Answers

Combining two text files as columns?

I have one space delimited file with multiple columns and one tab delimited file with multiple columns (They have the same number of rows). I want to basically combine these two text files into a new text file by column. How would I go about doing that? (1 Reply)
Discussion started by: evelibertine
1 Replies

4. Shell Programming and Scripting

Combining columns from multiple files to one file

I'm trying to combine colums from multiple file to a single file but having some issues, appreciate your help. The filenames are the same except for the extension, path1.m0 --------- a b c d e f g h i path1.m1 --------- m n o p q r s t u File names are path1.m The... (3 Replies)
Discussion started by: rkmca
3 Replies

5. UNIX for Dummies Questions & Answers

Need Help in reading N days files from a Directory & combining the files

Hi All, Request your expertise in tackling one requirement in my project,(i dont have much expertise in Shell Scripting). The requirement is as below, 1) We store the last run date of a process in a file. When the batch run the next time, it should read this file, get the last run date from... (1 Reply)
Discussion started by: dsfreddie
1 Replies

6. Shell Programming and Scripting

Combining columns from multiple files into one single output file

Hi, I have 3 files with one column value as shown File: a.txt ------------ Data_a1 Data_a2 File2: b.txt ------------ Data_b1 Data_b2 Data_b3 Data_b4 File3: c.txt ------------ Data_c1 Data_c2 Data_c3 Data_c4 Data_c5 (6 Replies)
Discussion started by: vfrg
6 Replies

7. Linux

[Solved] Combining columns from different files

Hey Guys & Gals, I am stuck with the following ; I have 2 text files, each containing 2 columns. My goal is to have a column from the 2nd file placed inbetween the columns in the first file. Basically the idea is, each address has a different name (but 1 name per address) but 1 address... (6 Replies)
Discussion started by: TAPE
6 Replies

8. Shell Programming and Scripting

Combining rows into columns

hi experts, I have a flat file with below contents Database1 Table1 column1 Database1 Table1 column2 Database1 Table1 column3 Database1 Table1 column4 Database1 Table2 Column1 Database1 Table2 Column2 Database2 Table1 Column1 Database2 Table1 Column2 Database2 Table1 Column3... (9 Replies)
Discussion started by: Selva_2507
9 Replies

9. Shell Programming and Scripting

Join two files combining multiple columns and produce mix and match output

I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns. Like I have file A 1234,ABCD,23,JOHN,NJ,USA 2345,ABCD,24,SAM,NY,USA 5678,GHIJ,24,TOM,NY,USA 5678,WXYZ,27,MAT,NJ,USA and file B ... (2 Replies)
Discussion started by: mady135
2 Replies

10. Shell Programming and Scripting

Combining certain columns of multiple files into one file

Hello Unix gurus, I have a large number of files (say X) each containing two columns of data and the same number of rows. I would like to combine these files to create a unique merged file containing X columns corresponding to the second column of each file (with a bonus of having the first... (3 Replies)
Discussion started by: ksennin
3 Replies
SDIFF(1)							     GNU Tools								  SDIFF(1)

NAME
sdiff - find differences between two files and merge interactively SYNOPSIS
sdiff -o outfile [options] from-file to-file DESCRIPTION
The sdiff command merges two files and interactively outputs the results to outfile. If from-file is a directory and to-file is not, sdiff compares the file in from-file whose file name is that of to-file, and vice versa. from-file and to-file may not both be directories. sdiff options begin with -, so normally from-file and to-file may not begin with -. However, -- as an argument by itself treats the remaining arguments as file names even if they begin with -. You may not use - as an input file. sdiff without -o (or --output) produces a side-by-side difference. This usage is obsolete; use diff --side-by-side instead. Options Below is a summary of all of the options that GNU sdiff accepts. Each option has two equivalent names, one of which is a single letter preceded by -, and the other of which is a long name preceded by --. Multiple single letter options (unless they take an argument) can be combined into a single command line argument. Long named options can be abbreviated to any unique prefix of their name. -a Treat all files as text and compare them line-by-line, even if they do not appear to be text. -b Ignore changes in amount of white space. -B Ignore changes that just insert or delete blank lines. -d Change the algorithm to perhaps find a smaller set of changes. This makes sdiff slower (sometimes much slower). -H Use heuristics to speed handling of large files that have numerous scattered small changes. --expand-tabs Expand tabs to spaces in the output, to preserve the alignment of tabs in the input files. -i Ignore changes in case; consider upper- and lower-case to be the same. -I regexp Ignore changes that just insert or delete lines that match regexp. --ignore-all-space Ignore white space when comparing lines. --ignore-blank-lines Ignore changes that just insert or delete blank lines. --ignore-case Ignore changes in case; consider upper- and lower-case to be the same. --ignore-matching-lines=regexp Ignore changes that just insert or delete lines that match regexp. --ignore-space-change Ignore changes in amount of white space. -l --left-column Print only the left column of two common lines. --minimal Change the algorithm to perhaps find a smaller set of changes. This makes sdiff slower (sometimes much slower). -o file --output=file Put merged output into file. This option is required for merging. -s --suppress-common-lines Do not print common lines. --speed-large-files Use heuristics to speed handling of large files that have numerous scattered small changes. -t Expand tabs to spaces in the output, to preserve the alignment of tabs in the input files. --text Treat all files as text and compare them line-by-line, even if they do not appear to be text. -v --version Output the version number of sdiff. -w columns --width=columns Use an output width of columns. Note that for historical reasons, this option is -W in diff, -w in sdiff. -W Ignore horizontal white space when comparing lines. Note that for historical reasons, this option is -w in diff, -W in sdiff. SEE ALSO
cmp(1), comm(1), diff(1), diff3(1). DIAGNOSTICS
An exit status of 0 means no differences were found, 1 means some differences were found, and 2 means trouble. GNU Tools 22sep1993 SDIFF(1)
All times are GMT -4. The time now is 05:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy