Error in concatenating multiple files from subdirectories


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Error in concatenating multiple files from subdirectories
# 1  
Old 02-13-2011
Error in concatenating multiple files from subdirectories

I want to concatenate multiple files recursively from sub-directories intoone file in Linux.

I saved the following script as script.sh in $HOME/testing1 where I have several subdirectories and .txt files into them. I ran script.sh from the command prompt of $HOME/testing1 as ./script.sh. But it deleted all the files from my sub-directories. Can anybody tell me what was wrong and write the correct script for me?

The only messages I got:


Code:
cat: file11.txt: input file is output filecat: file22.txt: input file is output file............
Here is the script I have used:


Code:
ogdir=$HOME/testing1 ## adjust to tastefor dir in "$logdir"/*/do(cd "$dir"files=( *.txt )cat "${files[@]}" > "${PWD##*/}.txt"rm "${files[@]}")done

---------- Post updated at 01:31 PM ---------- Previous update was at 01:29 PM ----------

I want to concatenate multiple files recursively from sub-directories into one file in Linux.

I saved the following script as script.sh in $HOME/testing1 where I have several subdirectories and .txt files into them. I ran script.sh from the command prompt of $HOME/testing1 as ./script.sh. But it deleted all the files from my sub-directories. Can anybody tell me what was wrong and write the correct script for me?

The only messages I got:

Code:
cat: file11.txt: input file is output file
cat: file22.txt: input file is output file

Here is the script I have used:

l
Code:
ogdir=$HOME/testing1 ## adjust to taste
for dir in "$logdir"/*/
do
(
cd "$dir"
files=( *.txt )
cat "${files[@]}" > "${PWD##*/}.txt"
rm "${files[@]}"
)
done

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 02-13-2011 at 02:16 PM.. Reason: code tags, please!
# 2  
Old 02-13-2011
Have you noticed the mismatched variables?
$logdir
ogdir

If you post a listing of the content of $logdir, it might help to know why cat is trying to use file11.txt and file22.txt as the source of input and output.
# 3  
Old 02-13-2011
The "ogdir" and "logdir" is just a misplaced code tag.

We definitely need to see a reasonable sample of the directory tree and contents. Also, what you expect "concatonate" to do in the context of your files in your directory tree?
This is not so much about correcting some script you found in a post on the Internet, but about finding a solution to your problem in your environment.

The script posted has many design flaws were it to be used as a general tool. It was posted by cfajohnson in response to a specific post a month ago.
I assume you have bash because this line is a syntax error in my unix Posix shells:
Quote:
files=( *.txt )
One of the more obvious issues is that it canot be run twice or you could end up with all your ".txt" files deleted (including the concatonated one). There is no way to unconcatonate after this script has been run once.
# 4  
Old 02-13-2011
If I have to guess I would say that [s]he has a directory name file11 and file22, as children of testing1, which have themselves a file11.txt and file22.txt
When it hits "${PWD##*/}.txt" the directory names are used to send output to a file name as such, conflicting with the file already open.

But this is only a guess.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

2. UNIX for Dummies Questions & Answers

Help with untarring multiple files from tarred directories and subdirectories

Hi, I want to untar all log files from following tarred directory hierarchy Log_files.tar.gz/subject*.tar.gz/project*/*.log It means there are subject1.tar.gz to subject9.tar.gz and in those tarred subect directories there are project1 - project5 directories and in those directories there... (2 Replies)
Discussion started by: rv_trojan
2 Replies

3. Shell Programming and Scripting

Problem in concatenating multiple files from subdirectories

I want to concatenate multiple files recursively from sub-directories intoone file in Linux. I saved the following script as script.sh in $HOME/testing1 where I have several subdirectories and .txt files into them. I ran script.sh from the command prompt of $HOME/testing1 as ./script.sh. But it... (3 Replies)
Discussion started by: raj284
3 Replies

4. Shell Programming and Scripting

Compress files as per timestamp in multiple subdirectories

I'd really appreciate if anyone could assist me with this code A directory with multiple subdirectories has multiple files which are timestamp'ed. We need to - compress files as per timestamp - save compressed file/s in the respective folder - delete the source files ============... (2 Replies)
Discussion started by: sreewin7
2 Replies

5. Shell Programming and Scripting

concatenating selected lines of multiple files

Hi, I would like a shell script that reads all files in a directory and concatenate them. It is not a simple concatenation. The first few lines of the files should not be included. The lines to be included are the lines from where 'START HERE' appears up to the end of the file. For example, I... (4 Replies)
Discussion started by: laiko
4 Replies

6. Shell Programming and Scripting

Concatenating two files

HI I need to concatenate two files which are having headers. the result file should contain only the header from first file only and the header in second file have to be skipped. file1: name age sriram 23 file2 name age prabu 25 result file should be name age sriram 23 prabu ... (6 Replies)
Discussion started by: Sriramprabu
6 Replies

7. Shell Programming and Scripting

negatively concatenating files

That subject might sound weird. What I have is two files. One looks like: rf17 pts/59 Jul 10 08:43 (10.72.11.22) 27718 pts/59 0:00 b rf17 pts/3 Jul 10 10:03 (10.72.11.22) 32278 pts/3 1:43 b rf58 pts/29 Jul 10 10:09 (10.72.11.51) 44220 pts/29 0:06 b rf58 pts/61 Jul 10 08:45 (10.72.11.51)... (2 Replies)
Discussion started by: raidzero
2 Replies

8. Shell Programming and Scripting

Concatenating multiple lines to one line if match pattern

Hi all, I've been working on a script which I have hit a road block now. I have written a script using sed to extract the below data and pumped into another file: Severity............: MAJORWARNING Summary: System temperature is out of normal range. Severity............: MAJORWARNING... (13 Replies)
Discussion started by: phixsius
13 Replies

9. UNIX for Dummies Questions & Answers

sftp multiple files and subdirectories

Hi, I'm new to using the sftp command prompt within the unix shell (usually I just use an FTP windows program). I've successfully connected to a remote server from within ssh. I've also been able to copy multiple files using the mget command e.g. mget *.html How do I copy all files,... (3 Replies)
Discussion started by: robbieg
3 Replies

10. UNIX for Dummies Questions & Answers

concatenating x files into a one...

... i have 4 files to concatenate but in a certain order and i wanted to do it in a shorter one line command , if possible ! 4 files : file , file0 , file1 and file2 file1 into file2 file0 into the result file into the result thanks in advance Christian (1 Reply)
Discussion started by: Nicol
1 Replies
Login or Register to Ask a Question