09-07-2005
combining columns from different files
Hi all,
I would be very grateful for some advice on the following.
I have several text files. The files are experiment results with columns of data separated by white space.
The files begin with several lines of header which are all preceeded by a comment character '#'.
Each file has a record number column as its first column and then several columns of values relating to that record. The record numbers go from 1-N in each experiment file
I would like to compare a particular column across several experiment files.
How to I take two of these experiment files and create a new file such that the new file contains:
- none of the header information
- the first column is the record column
- the next column is ,say column 2 from the first file
- the next column is the same column 2 from the second file
I've looked into awk and join but am not sure how to use them in combination to achieve what I want.
Thanks,
Enda
Sample file
File1.dat:
################
# Header info
################
1 10 20 30
2 13 50 13
3 ....
etc
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi, I have two files.
File1:
File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length.
ID Name
1-43<<11 spaces>>Swapna<<94 spaces>>
1-234<<10 spaces>>Mani<<96 spaces>>
1-3456<<9 spaces>>Kapil<<95 spaces>>
File2:
... (4 Replies)
Discussion started by: manneni prakash
4 Replies
2. Shell Programming and Scripting
I have two files I need to combine. The problem I'm having is I need to only combine data from the second file in the empty spaces of the first. For example:
file1
Data Field
Data Field
Data Field
Data Field
file2
a - Insert Data
b - Insert Data
c - Insert Data
d - Insert Data... (10 Replies)
Discussion started by: handband2
10 Replies
3. UNIX for Dummies Questions & Answers
I have one space delimited file with multiple columns and one tab delimited file with multiple columns (They have the same number of rows). I want to basically combine these two text files into a new text file by column. How would I go about doing that? (1 Reply)
Discussion started by: evelibertine
1 Replies
4. Shell Programming and Scripting
I'm trying to combine colums from multiple file to a single file but having some issues, appreciate your help.
The filenames are the same except for the extension,
path1.m0
---------
a b c
d e f
g h i
path1.m1
---------
m n o
p q r
s t u
File names are path1.m
The... (3 Replies)
Discussion started by: rkmca
3 Replies
5. UNIX for Dummies Questions & Answers
Hi All,
Request your expertise in tackling one requirement in my project,(i dont have much expertise in Shell Scripting). The requirement is as below,
1) We store the last run date of a process in a file. When the batch run the next time, it should read this file, get the last run date from... (1 Reply)
Discussion started by: dsfreddie
1 Replies
6. Shell Programming and Scripting
Hi,
I have 3 files with one column value as shown
File: a.txt
------------
Data_a1
Data_a2
File2: b.txt
------------
Data_b1
Data_b2
Data_b3
Data_b4
File3: c.txt
------------
Data_c1
Data_c2
Data_c3
Data_c4
Data_c5 (6 Replies)
Discussion started by: vfrg
6 Replies
7. Linux
Hey Guys & Gals,
I am stuck with the following ;
I have 2 text files, each containing 2 columns.
My goal is to have a column from the 2nd file placed inbetween the columns in the first file.
Basically the idea is, each address has a different name (but 1 name per address) but 1 address... (6 Replies)
Discussion started by: TAPE
6 Replies
8. Shell Programming and Scripting
hi experts,
I have a flat file with below contents
Database1 Table1 column1
Database1 Table1 column2
Database1 Table1 column3
Database1 Table1 column4
Database1 Table2 Column1
Database1 Table2 Column2
Database2 Table1 Column1
Database2 Table1 Column2
Database2 Table1 Column3... (9 Replies)
Discussion started by: Selva_2507
9 Replies
9. Shell Programming and Scripting
I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns.
Like I have file A
1234,ABCD,23,JOHN,NJ,USA
2345,ABCD,24,SAM,NY,USA
5678,GHIJ,24,TOM,NY,USA
5678,WXYZ,27,MAT,NJ,USA
and file B
... (2 Replies)
Discussion started by: mady135
2 Replies
10. Shell Programming and Scripting
Hello Unix gurus,
I have a large number of files (say X) each containing two columns of data and the same number of rows.
I would like to combine these files to create a unique merged file containing X columns corresponding to the second column of each file (with a bonus of having the first... (3 Replies)
Discussion started by: ksennin
3 Replies
SDIFF(1) GNU Tools SDIFF(1)
NAME
sdiff - find differences between two files and merge interactively
SYNOPSIS
sdiff -o outfile [options] from-file to-file
DESCRIPTION
The sdiff command merges two files and interactively outputs the results to outfile.
If from-file is a directory and to-file is not, sdiff compares the file in from-file whose file name is that of to-file, and vice versa.
from-file and to-file may not both be directories.
sdiff options begin with -, so normally from-file and to-file may not begin with -. However, -- as an argument by itself treats the
remaining arguments as file names even if they begin with -. You may not use - as an input file.
sdiff without -o (or --output) produces a side-by-side difference. This usage is obsolete; use diff --side-by-side instead.
Options
Below is a summary of all of the options that GNU sdiff accepts. Each option has two equivalent names, one of which is a single letter
preceded by -, and the other of which is a long name preceded by --. Multiple single letter options (unless they take an argument) can be
combined into a single command line argument. Long named options can be abbreviated to any unique prefix of their name.
-a Treat all files as text and compare them line-by-line, even if they do not appear to be text.
-b Ignore changes in amount of white space.
-B Ignore changes that just insert or delete blank lines.
-d Change the algorithm to perhaps find a smaller set of changes. This makes sdiff slower (sometimes much slower).
-H Use heuristics to speed handling of large files that have numerous scattered small changes.
--expand-tabs
Expand tabs to spaces in the output, to preserve the alignment of tabs in the input files.
-i Ignore changes in case; consider upper- and lower-case to be the same.
-I regexp
Ignore changes that just insert or delete lines that match regexp.
--ignore-all-space
Ignore white space when comparing lines.
--ignore-blank-lines
Ignore changes that just insert or delete blank lines.
--ignore-case
Ignore changes in case; consider upper- and lower-case to be the same.
--ignore-matching-lines=regexp
Ignore changes that just insert or delete lines that match regexp.
--ignore-space-change
Ignore changes in amount of white space.
-l
--left-column
Print only the left column of two common lines.
--minimal
Change the algorithm to perhaps find a smaller set of changes. This makes sdiff slower (sometimes much slower).
-o file
--output=file
Put merged output into file. This option is required for merging.
-s
--suppress-common-lines
Do not print common lines.
--speed-large-files
Use heuristics to speed handling of large files that have numerous scattered small changes.
-t Expand tabs to spaces in the output, to preserve the alignment of tabs in the input files.
--text Treat all files as text and compare them line-by-line, even if they do not appear to be text.
-v
--version
Output the version number of sdiff.
-w columns
--width=columns
Use an output width of columns. Note that for historical reasons, this option is -W in diff, -w in sdiff.
-W Ignore horizontal white space when comparing lines. Note that for historical reasons, this option is -w in diff, -W in sdiff.
SEE ALSO
cmp(1), comm(1), diff(1), diff3(1).
DIAGNOSTICS
An exit status of 0 means no differences were found, 1 means some differences were found, and 2 means trouble.
GNU Tools 22sep1993 SDIFF(1)