Sponsored Content
Top Forums Shell Programming and Scripting combining columns from different files Post 82981 by iomaire on Friday 9th of September 2005 03:58:07 AM
Old 09-09-2005
Hi again,

I'm sorry I'm not being clear. I've attached a bash script that does what I need but it's messy.
- it assumes that all files with a certain name pattern in a directory are to be included
- the column to extract is hard-coded as I couldn't pass it to awk within the script

Scenario: each file is the result of an experiment. Each column is a particular value measured in the experiment. Each row is the index of the measurement taken.
I want to a create summary file across all experiment result files such that a summary file joins a given colum from each experiment file.

Example problem:
File_A.dat
#################
# Comments
# Header
#################
1 val_A12 val_A13
2 val_A22 val_A23
3 val_A32 val_A33

File_B.dat
#################
# Comments
# Header
#################
1 val_B12 val_B13
2 val_B22 val_B23
3 val_B32 val_B33

Files might have up to 2500 rows and 30 columns.

Want to pass to a script 'Give me all column 2's from File_A.dat File_B.dat ... File_30.dat

Desired result file (for column 2's):
Result.dat:
1 val_A12 val_B12
2 val_A22 val_B22
2 val_A32 val_B32
etc

Like I say, my script does the job but hopefully you can think of a neater way to do this.

Regards,
Éanna
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combining Two fixed width columns to a variable length file

Hi, I have two files. File1: File1 contains two fixed width columns ID of 15 characters length and Name is of 100 characters length. ID Name 1-43<<11 spaces>>Swapna<<94 spaces>> 1-234<<10 spaces>>Mani<<96 spaces>> 1-3456<<9 spaces>>Kapil<<95 spaces>> File2: ... (4 Replies)
Discussion started by: manneni prakash
4 Replies

2. Shell Programming and Scripting

Combining columns from different files

I have two files I need to combine. The problem I'm having is I need to only combine data from the second file in the empty spaces of the first. For example: file1 Data Field Data Field Data Field Data Field file2 a - Insert Data b - Insert Data c - Insert Data d - Insert Data... (10 Replies)
Discussion started by: handband2
10 Replies

3. UNIX for Dummies Questions & Answers

Combining two text files as columns?

I have one space delimited file with multiple columns and one tab delimited file with multiple columns (They have the same number of rows). I want to basically combine these two text files into a new text file by column. How would I go about doing that? (1 Reply)
Discussion started by: evelibertine
1 Replies

4. Shell Programming and Scripting

Combining columns from multiple files to one file

I'm trying to combine colums from multiple file to a single file but having some issues, appreciate your help. The filenames are the same except for the extension, path1.m0 --------- a b c d e f g h i path1.m1 --------- m n o p q r s t u File names are path1.m The... (3 Replies)
Discussion started by: rkmca
3 Replies

5. UNIX for Dummies Questions & Answers

Need Help in reading N days files from a Directory & combining the files

Hi All, Request your expertise in tackling one requirement in my project,(i dont have much expertise in Shell Scripting). The requirement is as below, 1) We store the last run date of a process in a file. When the batch run the next time, it should read this file, get the last run date from... (1 Reply)
Discussion started by: dsfreddie
1 Replies

6. Shell Programming and Scripting

Combining columns from multiple files into one single output file

Hi, I have 3 files with one column value as shown File: a.txt ------------ Data_a1 Data_a2 File2: b.txt ------------ Data_b1 Data_b2 Data_b3 Data_b4 File3: c.txt ------------ Data_c1 Data_c2 Data_c3 Data_c4 Data_c5 (6 Replies)
Discussion started by: vfrg
6 Replies

7. Linux

[Solved] Combining columns from different files

Hey Guys & Gals, I am stuck with the following ; I have 2 text files, each containing 2 columns. My goal is to have a column from the 2nd file placed inbetween the columns in the first file. Basically the idea is, each address has a different name (but 1 name per address) but 1 address... (6 Replies)
Discussion started by: TAPE
6 Replies

8. Shell Programming and Scripting

Combining rows into columns

hi experts, I have a flat file with below contents Database1 Table1 column1 Database1 Table1 column2 Database1 Table1 column3 Database1 Table1 column4 Database1 Table2 Column1 Database1 Table2 Column2 Database2 Table1 Column1 Database2 Table1 Column2 Database2 Table1 Column3... (9 Replies)
Discussion started by: Selva_2507
9 Replies

9. Shell Programming and Scripting

Join two files combining multiple columns and produce mix and match output

I would like to join two files when two columns in each file matches with each other and then produce an output when taking multiple columns. Like I have file A 1234,ABCD,23,JOHN,NJ,USA 2345,ABCD,24,SAM,NY,USA 5678,GHIJ,24,TOM,NY,USA 5678,WXYZ,27,MAT,NJ,USA and file B ... (2 Replies)
Discussion started by: mady135
2 Replies

10. Shell Programming and Scripting

Combining certain columns of multiple files into one file

Hello Unix gurus, I have a large number of files (say X) each containing two columns of data and the same number of rows. I would like to combine these files to create a unique merged file containing X columns corresponding to the second column of each file (with a bonus of having the first... (3 Replies)
Discussion started by: ksennin
3 Replies
TRACE-CMD-RESTORE(1)													      TRACE-CMD-RESTORE(1)

NAME
trace-cmd-restore - restore a failed trace record SYNOPSIS
trace-cmd restore [OPTIONS] [command] cpu-file [cpu-file ...] DESCRIPTION
The trace-cmd(1) restore command will restore a crashed trace-cmd-record(1) file. If for some reason a trace-cmd record fails, it will leave a the per-cpu data files and not create the final trace.dat file. The trace-cmd restore will append the files to create a working trace.dat file that can be read with trace-cmd-report(1). When trace-cmd record runs, it spawns off a process per CPU and writes to a per cpu file usually called trace.dat.cpuX, where X represents the CPU number that it is tracing. If the -o option was used in the trace-cmd record, then the CPU data files will have that name instead of the trace.dat name. If a unexpected crash occurs before the tracing is finished, then the per CPU files will still exist but there will not be any trace.dat file to read from. trace-cmd restore will allow you to create a trace.dat file with the existing data files. OPTIONS
-c Create a partial trace.dat file from the machine, to be used with a full trace-cmd restore at another time. This option is useful for embedded devices. If a server contains the cpu files of a crashed trace-cmd record (or trace-cmd listen), trace-cmd restore can be executed on the embedded device with the -c option to get all the stored information of that embedded device. Then the file created could be copied to the server to run the trace-cmd restore there with the cpu files. If *-o* is not specified, then the file created will be called 'trace-partial.dat'. This is because the file is not a full version of something that trace-cmd-report(1) could use. -t tracing_dir Used with -c, it overrides the location to read the events from. By default, tracing information is read from the debugfs/tracing directory. -t will use that location instead. This can be useful if the trace.dat file to create is from another machine. Just tar -cvf events.tar debugfs/tracing and copy and untar that file locally, and use that directory instead. -k kallsyms Used with -c, it overrides where to read the kallsyms file from. By default, /proc/kallsyms is used. -k will override the file to read the kallsyms from. This can be useful if the trace.dat file to create is from another machine. Just copy the /proc/kallsyms file locally, and use -k to point to that file. -o output' By default, trace-cmd restore will create a trace.dat file (or trace-partial.dat if -c is specified). You can specify a different file to write to with the -o option. -i input By default, trace-cmd restore will read the information of the current system to create the initial data stored in the trace.dat file. If the crash was on another machine, then that machine should have the trace-cmd restore run with the -c option to create the trace.dat partial file. Then that file can be copied to the current machine where trace-cmd restore will use -i to load that file instead of reading from the current system. EXAMPLES
If a crash happened on another box, you could run: $ trace-cmd restore -c -o box-partial.dat Then on the server that has the cpu files: $ trace-cmd restore -i box-partial.dat trace.dat.cpu0 trace.dat.cpu1 This would create a trace.dat file for the embedded box. SEE ALSO
trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-start(1), trace-cmd-stop(1), trace-cmd-extract(1), trace-cmd-reset(1), trace-cmd-split(1), trace-cmd-list(1), trace-cmd-listen(1) AUTHOR
Written by Steven Rostedt, <rostedt@goodmis.org[1]> RESOURCES
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git COPYING
Copyright (C) 2010 Red Hat, Inc. Free use of this software is granted under the terms of the GNU Public License (GPL). NOTES
1. rostedt@goodmis.org mailto:rostedt@goodmis.org 06/11/2014 TRACE-CMD-RESTORE(1)
All times are GMT -4. The time now is 03:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy