Using GAWK to combine files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using GAWK to combine files
# 8  
Old 11-14-2009
I is unclear to me what you are after. Suppose you have a couple of files with just one column and e.g. 4 rows then danmero's paste command should just work...
example:
Code:
$> ls *.txt
a.txt  b.txt  c.txt  d.txt  e.txt  f.txt  g.txt

$> cat a.txt
a1
a2
a3
a4

$> cat g.txt
g1
g2
g3
g4

$> paste -d\: *.txt
a1:b1:c1:d1:e1:f1:g1
a2:b2:c2:d2:e2:f2:g2
a3:b3:c3:d3:e3:f3:g3
a4:b4:c4:d4:e4:f4:g4



---------- Post updated at 01:08 AM ---------- Previous update was at 12:44 AM ----------

-or-

Do you need it transposed, like so:

Code:
$> for i in *.txt ; do xargs < $i | tr ' ' ':' ; done
a1:a2:a3:a4
b1:b2:b3:b4
c1:c2:c3:c4
d1:d2:d3:d4
e1:e2:e3:e4
f1:f2:f3:f4
g1:g2:g3:g4

Code:
$> paste -s -d\: *.txt
a1:a2:a3:a4
b1:b2:b3:b4
c1:c2:c3:c4
d1:d2:d3:d4
e1:e2:e3:e4
f1:f2:f3:f4
g1:g2:g3:g4


Last edited by Scrutinizer; 11-14-2009 at 04:58 AM..
# 9  
Old 11-14-2009
Ok fixed the issue...

I was actually combining windows file format files which had windows specific line break characters. As a result of which columns were not visible on same row...

Hence had to convert the result file to unix file as shown below:

Quote:
paste -d\# Columns/*.out > Table.windows
tr -d '\15\32' < Table.windows > Table.unix
I have few more questions:

1. What is the significance of black slash (\) in -d\

2. If I want to use tab as the delimiter then I guess following should be sufficient but somehow its not giving the desired results:
Quote:
paste -d Columns/*.out > Table.windows
Any pointers?

3. Is there a way I can use a long single delimiter like '##--##' or may be '&+&+&'
I tried following but it doesn't seem to work.
Quote:
paste -d '##--##' Columns/*.out > Table.windows
Any idea?
# 10  
Old 11-14-2009
1. In the paste example it is used as an escape character but it is not really necessary there:
Code:
paste -d: *.txt

should work too. in your tr -d example it means ascii character with octal value 15 followed by ascii character oct 32 (carriage return and substitute character)

2. try paste without the -d option
3. paste can only use single character separators (or a list thereof, which it will then cycle through and use one character at a time)
You could try this;
Code:
paste Columns/*.out|sed 's/\t/##--##/g' > Table.windows

See man paste for further details

Last edited by Scrutinizer; 11-14-2009 at 07:45 AM..
# 11  
Old 11-14-2009
Quote:
paste Columns/*.out|sed 's/\t/##--##/g' > Table.windows

It will also remove the tab charactes if any present in the data.

I think following hack will do the trick....

Quote:
for file in `ls *.out`; do gawk 'BEGIN{FS = "##--##" }{print $1"##--#"}' $file > ../Mod_Columns/$file ; done
Changing FS as I am interested in the space characters after the columns otherwise GAWK will trim the white spaces...

Quote:
paste -d# Mod_Columns/*.out > Table.windows
Advantage of this is that there no need use 'tr' to remove line break characters - GAWK takes of care of it...
# 12  
Old 11-14-2009
Quote:
Originally Posted by paragkalra
It will also remove the tab charactes if any present in the data.
OK, then pick a character that is not present and replace that again by the string you want, like e.g.
Code:
paste -d:@ Columns/*.out|sed 's/@/##--##/g' > Table.windows

or some other character.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Windows & DOS: Issues & Discussions

gawk: Merge N files into one

hi all! i'm an awk newbie and have been trying in vain to merge N files together. ie. file1.txt: a b c 1 1 1 2 2 2 3 3 3 4 4 4 file2.txt: a b c 5 5 5 6 6 6 7 7 7 8 8 8 (13 Replies)
Discussion started by: sameeribraimo
13 Replies

3. Shell Programming and Scripting

combine two files...

Hi, i have two files. i want to combine records from these two files in below manner :- first line from first file(1st line) 2nd line from 2nd file(1st line) 3rd line from 1st file(2nd line) 4th line from 2nd file(2nd line) so on.... (1 Reply)
Discussion started by: deepakiniimt
1 Replies

4. Shell Programming and Scripting

Reading in all files from parent directory (GAWK)

Hi all, I'm very, very new to scripting (let alone SHELL) and was wondering if anyone could help me out as I seem to be in a spot of bother. I collect data (.dat files) which are automatically seperated into several sub directories, so the file paths I'm reading in at the moment would be... (11 Replies)
Discussion started by: gd9629
11 Replies

5. Shell Programming and Scripting

combine multiple files by column into one files already sorted!

I have multiple files; each file contains a certain data in a column view simply i want to combine all those files into one file in columns example file1: a b c d file 2: 1 2 3 4 file 3: G (4 Replies)
Discussion started by: ahmedamro
4 Replies

6. Shell Programming and Scripting

gawk - reading two files & re arrange the columns

Hi, I am trying to read 2 files and writing to the 3rd file if I find the same elements in 2 files. my first file is 1 0 kb12124819 766409 1.586e-01 1 0 kb17160939 773886 8.674e-01 1 0 kb4475691 836671 8.142e-01 1 0 ... (2 Replies)
Discussion started by: ezhil01
2 Replies

7. Shell Programming and Scripting

Combine files with same name

I need a script that combines files with the same name. These files are on a windows directory but the PC has Cygwin so i have a limited unix command set. What I've got; WebData_9_2007-09-20.txt WebData_9_2007-09-20.txt WebData_9_2007-09-21.txt WebData_9_2007-09-20.txt... (4 Replies)
Discussion started by: jmwhitford
4 Replies

8. Shell Programming and Scripting

combine two files

I have two files: file1 and file2 the content of files1 is: 13 22 333 42 56 55 ... the content of file2 is: aa dd cc ee ff gg ... (1 Reply)
Discussion started by: fredao1
1 Replies

9. Shell Programming and Scripting

How to combine 2 files

hi all i have 2 files f1 and f2 i have to combine these 2 files and make a new file f3 when i use paste f1 f2 >f3 its pasting vertically but i want to paste horizontally How to do .. pls let me know (2 Replies)
Discussion started by: ravi.sadani19
2 Replies

10. UNIX for Dummies Questions & Answers

how to combine two files

i need to combine two file. These two files have the same line number, and i need to combine each corresponding line. I tried the paste, but i need coma as the delimeter. are there anyway to do it? thanks. (4 Replies)
Discussion started by: tao
4 Replies
Login or Register to Ask a Question