Sponsored Content
Top Forums UNIX for Beginners Questions & Answers How to copy a column of multiple files and paste into new excel file (next to column)? Post 303033844 by drl on Friday 12th of April 2019 09:22:35 AM
Old 04-12-2019
Hi.

Here is a solution that uses a single "master" paste with a created list of cut process substitution commands:
Code:
#!/usr/bin/env bash

# @(#) s1       Demonstrate creation and execution of arbitrary number of commands.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C cut paste

pl " Basic structure of data file data1, showing invisible characters:"
cat -A data1

pl " Input data files:" data?
head data?

pl " Results, process substitution:"
paste <(cut -f1,5 data1) <(cut -f1,5 data2)

pl " Create a list of process substitution commands:"
command=""
for i in data*
do
  pe " Working on file $i"
  command="$command <(cut -f1,5 ${i})"
done
pe " command = :$command:"

pl " Use the created command string in a master paste command:"
pe " ( noting that command \"eval\" can be dangerous. )"
eval paste $command

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-7-amd64, x86_64
Distribution        : Debian 8.11 (jessie) 
bash GNU bash 4.3.30
cut (GNU coreutils) 8.23
paste (GNU coreutils) 8.23

-----
 Basic structure of data file data1, showing invisible characters:
org1_1^I1^I1^I2.5^I100$
org1_2^I1^I2^I5.5^I98^I$
org1_3^I1^I3^I7.2^I88$

-----
 Input data files: data1 data2 data3
==> data1 <==
org1_1  1       1       2.5     100
org1_2  1       2       5.5     98
org1_3  1       3       7.2     88

==> data2 <==
org2_1  1       1       2.5     100
org2_2  1       2       5.5     56
org2_3  1       3       7.2     70

==> data3 <==
org3_1  1       1       2.5     100
org3_2  1       2       5.5     35
org3_3  1       3       7.2     44

-----
 Results, process substitution:
org1_1  100     org2_1  100
org1_2  98      org2_2  56
org1_3  88      org2_3  70

-----
 Create a list of process substitution commands:
 Working on file data1
 Working on file data2
 Working on file data3
 command = : <(cut -f1,5 data1) <(cut -f1,5 data2) <(cut -f1,5 data3):

-----
 Use the created command string in a master paste command:
 ( noting that command "eval" can be dangerous. )
org1_1  100     org2_1  100     org3_1  100
org1_2  98      org2_2  56      org3_2  35
org1_3  88      org2_3  70      org3_3  44

Best wishes ... cheers, drl
These 3 Users Gave Thanks to drl For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

paste each 10 lines of single column to several column

Hi, I need to paste each 10 lines of single column to several columns. Please, can anyone tell me how to write in awk? Input File: 22 34 36 12 17 19 15 11 89 99 56 38 29 (4 Replies)
Discussion started by: nica
4 Replies

2. Shell Programming and Scripting

Copy a column and paste to other file question

Please help me. This is simple, but urgent problem for me. :( I have a two files file1 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 ..... file2 11 12 13 14 15 11 12 13 14 15 11 12 13 14 15 ..... 1) I hope to make a new file, file 3, that consists of 2nd... (2 Replies)
Discussion started by: exsonic
2 Replies

3. UNIX for Advanced & Expert Users

Copy a column to another column in UNIX fixedwidth file

Hi All, I have a fixedwidth file of length 3000. Now i want to copy a column of 4 chars i.e( length 1678-1681) to column 1127 – 1171 to the same file. Please let me know how can i achive using a single command in fixed width file. Also source column length is 4 chars and target column length... (4 Replies)
Discussion started by: kiranparsha
4 Replies

4. UNIX for Dummies Questions & Answers

Paste column from one file as column of

Any shortcuts for doing this? I need to cut the column 4 values from File1 and paste them as column4 values of File2, but only for the (first) same number of lines as File1 . All rows in File1 are contained in File2 in the exact same order, so the cut paste should work. File1 (with header and 3... (4 Replies)
Discussion started by: senhia83
4 Replies

5. Shell Programming and Scripting

Problems with awk (fatal error) and paste (two variables into one column-by-column)

Hello, I have a script extracting columns of useful numbers from a data file, and manipulating the numbers with awk commands. I have problems with my script... 1. There are two lines assigning numbers to $BaseForAveraging. If I use the commented line (the first one) and let the second one... (9 Replies)
Discussion started by: vgbraymond
9 Replies

6. Shell Programming and Scripting

Paste columns based on common column: multiple files

Hi all, I've multiple files. In this case 5. Space separated columns. Each file has 12 columns. Each file has 300-400K lines. I want to get the output such that if a value in column 2 is present in all the files then get all the columns of that value and print it side by side. Desired output... (15 Replies)
Discussion started by: genome
15 Replies

7. Shell Programming and Scripting

Locate the files in the first column and copy the files in 2nd column

#cat data.txt file1 folder1 file2 thisforfile2 file3 thisfolderforfile3 lata4 folder4 step 1: create the folder first in column 2 for i in `awk '{print $2}' data.txt` do mkdir /home/data/$i done step 2: locate the files in column1 and stored them into a file for i in... (17 Replies)
Discussion started by: kenshinhimura
17 Replies

8. UNIX for Beginners Questions & Answers

How to insert data into black column( Secound Column ) in excel (.XLSX) file using shell script?

Source Code of the original script is down below please run the script and try to solve this problem this is my data and I want it column wise 2019-03-20 13:00:00:000 2019-03-20 15:00:00:000 1 Operating System LAB 0 1 1 1 1 1 1 1 1 1 0 1 (5 Replies)
Discussion started by: Shubham1182
5 Replies

9. UNIX for Beginners Questions & Answers

How to copy particular files from a multiple directories and paste in a new directory?

Dear all I have a multiple directories, say for example org1, org2, org3 ..... org100 and each directory having a file namely dnaG.fasta. I need to copy all the dnaG.fasta file from each directory and paste in another directory fastconcatg. Therefore, my script has to copy dnaG.fasta file from... (5 Replies)
Discussion started by: dineshkumarsrk
5 Replies

10. UNIX for Beginners Questions & Answers

Copy files if column 5 in a file contains ā€œVā€

I have number of csv files (like tmo_2019*). In these files some files have 5th column value as V. I want to copy those files having 5th column value as V to specific directory /test/V_files/. I tried to extract file names by below but not able to complete command for copy. find -type f -iname... (4 Replies)
Discussion started by: Bops
4 Replies
All times are GMT -4. The time now is 11:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy