Merge multiple columns into one using cat


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Merge multiple columns into one using cat
# 8  
Old 05-29-2018
Thanks Don Cragun,
My objective is to have a single column data i.e. <new-line> line terminators on every line with no <carriage-return> as follow:
file1.txt:
Code:
1234
1234
1234

file2.txt:
Code:
aaaa
bbbb
cccc
dddd

Code:
cat file1.txt file2.txt > file3.txt

file3.txt:
Code:
1234
1234
1234
aaaa
bbbb
cccc
dddd

Thanks.




Moderator's Comments:
Mod Comment Please use CODE tags (for data as well!) as required by forum rules!

Last edited by RudiC; 05-29-2018 at 01:28 PM.. Reason: Added CODE tags.
# 9  
Old 05-29-2018
Hmmm - repeating your unmodified problem description again doesn't really help, esp. when the error seems to have been tracked down by Scrutinizer - the (first?) file doesn't have the necessary line terminator on the last line. You'll have to repair that, esp. as his proposal doesn't work on your system. What would be the result of
Code:
{ cat file1; echo; cat file2; } > file3

This User Gave Thanks to RudiC For This Post:
# 10  
Old 05-29-2018
You can use the following to directly process DOS format text files without manually stripping out the <carriage-return> characters from your input files...

Create a file named merge containing:
Code:
for file in "$@"
do	tr -d '\r' "$file"
	echo
done

Then you can issue the command:
Code:
sh merge file1.txt file2.txt > file3.txt

to get what you want as long as the file pathnames you pass to merge as operands are DOS format text files.

The file1.txt and file2.txt are sample file pathname operands gleaned from post #1 in this thread. This script will work with one or more file pathname operands.
This User Gave Thanks to Don Cragun For This Post:
# 11  
Old 05-29-2018
Hi geomarine...

Think of your problem and how to get round it.
You want a single file in UNIX format.

Here is a DEMO of your two files:
Code:
#!/bin/sh
# All longhand to see the sequence of events.

echo "Create files as per OP."
printf "1234\r\n1234\r\n1234" > /tmp/file1.txt
printf "aaaa\r\nbbbb\r\ncccc\r\ndddd" > /tmp/file2.txt
echo "Join together to test."
cat /tmp/file1.txt /tmp/file2.txt > /tmp/file3.txt
cat -v /tmp/file3.txt
echo ""
echo "Remove ^M, <CR> character and create temp files 4 and 5..."
tr -d '\r' < /tmp/file1.txt > /tmp/file4.txt
tr -d '\r' < /tmp/file2.txt > /tmp/file5.txt
cat /tmp/file4.txt /tmp/file5.txt > /tmp/file3.txt
echo "...and check the error exists."
cat -v /tmp/file3.txt
echo ""
echo "Now add the newlines in the correct place, longhand."
cat /tmp/file4.txt > /tmp/file3.txt
echo "" >> /tmp/file3.txt
cat /tmp/file5.txt >> /tmp/file3.txt
echo "" >> /tmp/file3.txt
echo "Finalisation."
cat /tmp/file3.txt
echo "Check file validity using 'hexdump'."
hexdump -C /tmp/file3.txt

Result: OSX 10.13.4, default bash terminal calling OSX 'sh'.
Code:
Last login: Tue May 29 17:17:20 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> chmod 755 CATFiles.sh
AMIGA:barrywalker~/Desktop/Code/Shell> ./CATFiles.sh
Create files as per OP.
Join together to test.
1234^M
1234^M
1234aaaa^M
bbbb^M
cccc^M
dddd
Remove ^M, <CR> character and create temp files 4 and 5...
...and check the error exists.
1234
1234
1234aaaa
bbbb
cccc
dddd
Now add the newlines in the correct place, longhand.
Finalisation.
1234
1234
1234
aaaa
bbbb
cccc
dddd
Check file validity using 'hexdump'.
00000000  31 32 33 34 0a 31 32 33  34 0a 31 32 33 34 0a 61  |1234.1234.1234.a|
00000010  61 61 61 0a 62 62 62 62  0a 63 63 63 63 0a 64 64  |aaa.bbbb.cccc.dd|
00000020  64 64 0a                                          |dd.|
00000023
AMIGA:barrywalker~/Desktop/Code/Shell> _


Last edited by wisecracker; 05-29-2018 at 02:37 PM.. Reason: Put ':' in place of '/' in 'Here is a DEMO of your two files:'
This User Gave Thanks to wisecracker For This Post:
# 12  
Old 05-29-2018
Please don't forget the requestor is using Win 7 Pr., probably with a cygwin or busybox setup, both with a (limited?) set of not-necessarily-*nix-compatible tools. I'm not sure that s/he will really benefit from a pure *nix result file without \r line terminators on the Win 7 system.
These 2 Users Gave Thanks to RudiC For This Post:
# 13  
Old 05-29-2018
Quote:
Originally Posted by RudiC
Please don't forget the requestor is using Win 7 Pr., probably with a cygwin or busybox setup, both with a (limited?) set of not-necessarily-*nix-compatible tools. I'm not sure that s/he will really benefit from a pure *nix result file without \r line terminators on the Win 7 system.
I was well aware of the situation but he did stress that he wanted it in UNIX format in an earlier post.
Good point though, but it is just as easy to put '\r\n's back if that becomes a necessity.
This User Gave Thanks to wisecracker For This Post:
# 14  
Old 06-05-2018
My Cygwin crashed and I have to re-install it so I couldn't test the recommendations. I have tested the 'cat' on a windows 10 OS and it works fine. So, the problem is with my computer though am not sure what could be wrong while such a simple command to fail. I decided to move my data to another system for the operation. Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join and merge multiple files with duplicate key and fill void columns

Join and merge multiple files with duplicate key and fill void columns Hi guys, I have many files that I want to merge: file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: (5 Replies)
Discussion started by: yjacknewton
5 Replies

2. Shell Programming and Scripting

Merge columns from multiple files

Hello and Good day I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files . I need to merge $2,$3,$6 from first file and $6 from another files. File1: $1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies

3. Shell Programming and Scripting

Merge records based on multiple columns

Hi, I have a file with 16 columns and out of these 16 columns 14 are key columns, 15 th is order column and 16th column is having information. I need to concate the 16th column based on value of 1-14th column as key in order of 15th column. Here are the example file Input File (multiple... (3 Replies)
Discussion started by: Ravi Agrawal
3 Replies

4. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

5. UNIX for Dummies Questions & Answers

Merge columns from multiple files

Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files. I know paste will append columns like so: paste file1 file2 file3 file4 file5 ... But this becomes inconvenient when you want to append a large number of files into a single file. ... (2 Replies)
Discussion started by: torchij
2 Replies

6. UNIX for Dummies Questions & Answers

How do I merge multiple columns into one column?

Hi all, I'm looking for a way to merge multiple columns (from one file) into a single column in an output file. The file I have looks somewhat like this: @HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII @HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies

7. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

8. Shell Programming and Scripting

Merge columns

Hi all - I have a file like below: A: A1,A2,A3,A4 B: 1,2,3,4 C: z,y,x,w .... This format repeats The output should come in a single line merging the first line field with the other two rows: A1_1 A1_z A2_2 A2_y A3_3 A3_x A4_4 A4_w Could anyone help with some directions ... (4 Replies)
Discussion started by: deepakgang
4 Replies

9. Shell Programming and Scripting

How to merge rows into columns ????

Hi guz I want to merge multiple rows into a multiple columns based on the first column. The file has symbol // I want to break the symbool // and I nedd exactlynew column at that point the output will be like this please guyz help in this isssue!!!!! merging rows into columns ... (4 Replies)
Discussion started by: bogu0001
4 Replies

10. UNIX for Advanced & Expert Users

use of sed over cat to merge files

I have three files, basically: file 1 - one line header file 2 - big data (approx 80GB) file 3 - a one line trailer the existing process cats these together i.e cat file 1 file 2 file 3 however... I was thinking, surely it could be more efficient to insert the header (file 1) on the... (2 Replies)
Discussion started by: miwinter
2 Replies
Login or Register to Ask a Question