Sponsored Content
Top Forums Shell Programming and Scripting Merge values from multiple directories into one file in awk or bash Post 302983004 by cmccabe on Wednesday 5th of October 2016 02:04:28 PM
Old 10-05-2016
Merge values from multiple directories into one file in awk or bash

I am trying to merge or combine all $1 values in validation.txt from multiple directories into one new file and output it here tab-delimited:/home/cmccabe/Desktop/20x/total/total.txt. Each $2 value and the header would then be a new field in total.txt. I am not sure how to go about this as cat is not the right tool. My real data has 15 paths and 70 .txt files. Thank you Smilie.

/home/cmccabe/Desktop/20x/panel/cardiomyopathy/coverage/validation.txt path1
Code:
Sample    20xCARDIOMYOPATHY
NA00449    98.8
NA02782    98.9

/home/cmccabe/Desktop/20x/panel/idp/coverage/validation.txt path2
Code:
Sample    20xIDP
NA00449    98.6
NA02782    98.7

/home/cmccabe/Desktop/20x/panel/pid/coverage/validation.txt path3
Code:
Sample    20xPID
NA00449    98.9
NA02782    99.0

desired output of total.txt (tab-delimited)
Code:
Sample    20xCARDIOMYOPATHY 20xIDP 20xPID
NA00449    98.8              98.6   98.9
NA02782    98.9              98.7   99.0


Last edited by cmccabe; 10-05-2016 at 03:10 PM.. Reason: added details, fixed format
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

bash script to rename multiple directories

Hello I have a directory structure with year in format 4 digits, e.g 2009, below which is month format 1 or 2 digits, e.g 1 or 12, blow which is day format 1 or 2 digits, e.g 1 or 31. I want to change the names of lots of directories to the be Year - 4 digits , e.g 2009 - No change here... (4 Replies)
Discussion started by: garethsays
4 Replies

2. Shell Programming and Scripting

BASH: extracting values from multiple lines after a match

Hi there, I have the following output, # raidctl -l RAID Volume RAID RAID Disk Volume Type Status Disk Status ------------------------------------------------------ c0t1d0 IM OK c0t1d0 OK ... (4 Replies)
Discussion started by: rethink
4 Replies

3. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of... (20 Replies)
Discussion started by: amarn
20 Replies

4. UNIX for Dummies Questions & Answers

Create multiple directories with awk

Hello all. Newbie here. In a directory, I have 50 files and one additional file that is a list of the names of the 50 files. I would like to create a directory for each of the 50 files, and I need the 50 directory names to correspond to the 50 file names. I know this can be done by running... (6 Replies)
Discussion started by: Zeckendorff
6 Replies

5. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

6. Shell Programming and Scripting

Read record from the text file contain multiple separated values & assign those values to variables

I have a file containing multiple values, some of them are pipe separated which are to be read as separate values and some of them are single value all are these need to store in variables. I need to read this file which is an input to my script Config.txt file name, first path, second... (7 Replies)
Discussion started by: ketanraut
7 Replies

7. Shell Programming and Scripting

Get multiple values from an xml file using one of the following commands or together awk/perl/script

Hello, I have a requirement to extract the value from multiple xml node and print out the values to new file to compare. Would be done using either awk/perl or some unix script. For example sample input file: ..... ..... <factories xmi:type="resources.jdbc:DataSource"... (2 Replies)
Discussion started by: slbmind
2 Replies

8. Shell Programming and Scripting

[bash] wanted: function with a clean way for multiple return values

Hi, I have a small part of a project which is done as a bash script. bash was selected as an portability issue that works out of the box. In this script I have an exec shell-function, a wrapper around arbitrary commands. I want to have STDOUT, as an addon STDERR and the EXIT-CODE of a specified... (5 Replies)
Discussion started by: stomp
5 Replies

9. Shell Programming and Scripting

awk to extract multiple values from file and add two additional fields

In the attached file I am trying to use awk to extract multiple values and create the tab-delimited desired output. In the output R_Index is a the sequential # and Pre_Enrichment is defaulted to .. I can extract from the values to the side of the keywords, but most are above and I can not... (2 Replies)
Discussion started by: cmccabe
2 Replies
All times are GMT -4. The time now is 04:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy