Sponsored Content
Top Forums Shell Programming and Scripting How to combine data files using for loop Post 302219131 by aigles on Monday 28th of July 2008 07:17:54 AM
Old 07-28-2008
Another way :

Code:
for dir in A B C D E
do
   echo $dir/file1.txt
done | 
xargs paste |
awk '{out = $1 OFS $2; for (i=3; i<=NF; i+=3) out = out OFS $i; print out}'

Execution (step by step):
Code:
> for dir in A B C D E
do
    echo $dir/file1.txt
done
A/file1.txt
B/file1.txt
C/file1.txt
D/file1.txt
E/file1.txt
>

Code:
> for dir in A B C D E
do
    echo $dir/file1.txt
done |
xargs echo paste  # echo for debug purpose
paste A/file1.txt B/file1.txt C/file1.txt D/file1.txt E/file1.txt
>

Code:
> for dir in A B C D E
do
    echo $dir/file1.txt
done |
xargs paste
1 1 A11 1 1 B11 1 1 C11 1 1 D11 1 1 E11
1 2 A12 1 2 B12 1 2 C12 1 2 D12 1 2 E12
>

Code:
> for dir in A B C D E
do
    echo $dir/file1.txt
done |
xargs paste |
awk '{out = $1 OFS $2; for (i=3; i<=NF; i+=3) out = out OFS $i; print out}'
1 1 A11 B11 C11 D11 E11
1 2 A12 B12 C12 D12 E12
>

Jean-Pierre.

Last edited by aigles; 07-28-2008 at 09:06 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

i want to combine two awk scripts which is having same loop and filelist

hi, to all give me some idea below scripts file=`cat filelist` for filename in $file do awk '{ if ($0 ~ /system/ && flag == 0) { flag=1; print $0; } else if ($0 ~ /system/ && flag == 1) { printf("%s SYSLAY %s %s %s\n",$1,$3, $4, $5); } else print $0; }' $filename >... (6 Replies)
Discussion started by: LAKSHMI NARAYAN
6 Replies

2. UNIX for Dummies Questions & Answers

Recover data from 2 files then combine

Using dd or similar tools to recover data from 2 damaged cdroms, I need a way to then combine the 2 files, 1 from each cd, and make a good file: this all result from finding that certain cd's tops scratch easily even when using the "proper" cd markers, hence making the file useless, however the... (1 Reply)
Discussion started by: saint65
1 Replies

3. Shell Programming and Scripting

combine data of 2 files by variable

my first post ... please be gentle. I have been working on a script to get info out of mysql. Its a support ticket system database OTRS. I can write the subject of open tickets to a text file with a unique user id. I also have a text file with the unique user id, username and email adres. I... (11 Replies)
Discussion started by: dicenl
11 Replies

4. Shell Programming and Scripting

get data from files combine them to a file

hi people; this is my file1.txt:192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 ... this is my file2.txt:portnames usernames maxusercap ... i want to write to file3.txt:l ./getports 192.168.1.1 'get all;l+;get . portnames;l-' l ./getports 192.168.1.1 'get all;l+;get . usernames;l-'... (4 Replies)
Discussion started by: gc_sw
4 Replies

5. Shell Programming and Scripting

How to combine data from 2 file ? How to print ?

Dear friends, I am just trying to print data from 2 file,namely file_1.txt and file_1.dat (specific column of ROW 1) file_1.txt 12 13 14 15 99 AMC 69 36 89 12 13 14 15 99 AMC 69 84 -12 12 13 14 ... (6 Replies)
Discussion started by: nex_asp
6 Replies

6. Shell Programming and Scripting

How to combine the data of files?

I have a main file as follows aaa 3/2 = 1.5 aba 55+6 = 61 aca 67+8 = 75 hjk 3+3 = 67 ghd 66+30 = 96 ghj 99-3 = 96 ffg 67+3 = 70 I have 4 sub files named sub1, sub2, sub3, sub4 content of sub1 aaa 23+5 = 28 hjk 45+6 = 51 ghd 40-20 = 20 ... (2 Replies)
Discussion started by: jackevan
2 Replies

7. Shell Programming and Scripting

Combine/omit data from 2 files

i made a script on my own. this is for the inventory to all of my AWS servers, and i run it to all of my servers to get the hostname, please look at file2. Then i need some data in file3 as well,. i need to combine them #cat file1 192.10.1.41 server.age.com ###### 192.10.0.40 ssh cant... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

8. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

9. Shell Programming and Scripting

Combine data out of 3 files into one new file

Hi, How can I combine the data of of three files into one new file? I try to give as much informations as possible. The three existing files are called file1 file2 and file3 the new file should named output_combined. The size of the files will be around 900 words/lines each .. but always... (5 Replies)
Discussion started by: MyMemberName
5 Replies

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies
paste(1)						      General Commands Manual							  paste(1)

Name
       paste - merge file data

Syntax
       paste file1 file2...
       paste -dlist file1 file2...
       paste -s [-dlist] file1 file2...

Description
       In  the	first  two forms, concatenates corresponding lines of the given input files file1, file2, etc.	It treats each file as a column or
       columns of a table and pastes them together horizontally (parallel merging).

       In the last form, the command combines subsequent lines of the input file (serial merging).

       In all cases, lines are glued together with the tab character, or with characters from an optionally specified  list.   Output  is  to  the
       standard output, so it can be used as the start of a pipe, or as a filter, if - is used in place of a file name.

Options
       -       Used in place of any file name, to read a line from the standard input.	(There is no prompting).

       -dlist  Replaces  characters  of  all but last file with nontabs characters (default tab).  One or more characters immediately following -d
	       replace the default tab as the line concatenation character.  The list is used circularly, i. e. when exhausted, it is reused.	In
	       parallel  merging  (i. e. no -s option), the lines from the last file are always terminated with a new-line character, not from the
	       list.  The list may contain the special escape sequences: 
 (new-line), 	 (tab), \ (backslash), and  (empty string, not a null
	       character).   Quoting  may  be  necessary,  if characters have special meaning to the shell (for example, to get one backslash, use
	       -d"\\" ).
	       Without this option, the new-line characters of each but the last file (or last line in case of the -s option) are  replaced  by  a
	       tab character.  This option allows replacing the tab character by one or more alternate characters (see below).

       -s      Merges  subsequent  lines  rather  than	one  from  each input file.  Use tab for concatenation, unless a list is specified with -d
	       option.	Regardless of the list, the very last character of the file is forced to be a new-line.

Examples
       ls | paste -d" " -
       list directory in one column
       ls | paste - - - -
       list directory in four columns
       paste -s -d"	
" file
       combine pairs of lines into lines

Diagnostics
       line too long
		 Output lines are restricted to 511 characters.

       too many files
		 Except for -s option, no more than 12 input files may be specified.

See Also
       cut(1), grep(1), pr(1)

																	  paste(1)
All times are GMT -4. The time now is 07:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy