Shell Programming and Scripting

Hello!

I am a real beginner in scripting, so I am struggling with a really easy task!

I want to write a script to concatenate several text files onto each other and generate a new file. I wanted the first argument to be the name of the new file, so:

Code:

./my_script.sh new_file file1.txt file2.txt file3.txt

would concatenate file1-3 and generate a file new_file.txt

Here is what I did so far:

Code:

#!/bin/bash

var=0
for i in $*
   do
	   var=$((var+1))
	   if [ $var -eq 1 ]
	   then
		   filename=$i
		   echo $name
		   touch $filename.txt
	   else
		   filename=$i
		   cat $filename | while read line
		   do
			   name=$line
			   size=${#name}
			   if [ $size -ne 0 ];then
				   echo "$name" >> $filename.txt
			   fi
		   done
	   fi
   done

But the problem is, it generates the file but the echo $name does not write anything into my file and I don't know what is the problem here :S

Redirection is your friend. Consider cat as well, not echo. This is the heart of the logic you need:

Code:

!/bin/bash
# usage example: mycat.sh  new1 old1 old2 old3
# step one create a new empty file, just in case there is one with the name new1
> $1.
cat $2 >> $1   # append
cat $3 >> $1   # append
cat $4 >> $1

You can make a loop, you do not have to check if a parameter is zero length unless the parameter is "" but if you require file checking:

Code:

for file in "$2" "$3" "$4"
do
  [ -f "$file" ] &&  cat  $file >> $1
done

Or make a one-liner using:

Code:

cat $2 $3 $4 > $1

The cat command is short for concatenate, meaning to string together. So cat is what I would use to do what you ask. This does not check for existence of the input file or a missing or bad filename.

None of the above is checking return codes, the $? variable which you should check in case you have an error other than bad file name. Like permissions.

Yes but it is important to me, that the generated file has an ending, like .txt

And if I write like you proposed, so:

Code:

>$1.txt

and then try to concatenate with whatever, cat or echo,

Code:

cat $2 >> $1.txt

It is not concatenating :S, also removed the .txt and wrote

Code:

cat $2 >> $1

but still nothing :S

Use

Code:

 ${1}.txt  not $1.txt

That's a bit surprising. Works for me exactly as anticipated:

Code:

>$1.txt
cat file1 >> $1.txt
cat file2 >> $1.txt
cat $1.txt

shows the contents of file[12] concatenated together.

First a few comments on the above code. The commands marked in red seem strange, dangerous, or wrong and dangerous.

In the first case, if any of your command line arguments contain whitespace, you won't get what you want. Use:

Code:

for i in "$@"

instead.

In the second case, you set a variable named filename and then print the contents of a variable named name. The 1st time through the loop, name is definitely not set. On subsequent times through the loop, name is still not set as long as you're using bash. (If you were using ksh it would be defined, but probably wouldn't be what you want. See later comments.) This is why, this echo prints nothing (but it would not affect what actually gets written into your destination file).

The third line marked in red will do what you want as long as there are no whitespace characters in any of the names of the files you're processing. You should ALWAYS quote strings that you get from a user to protect against there being a <space>, <tab>, or <newline> where you wouldn't expect one.

The fourth line marked in red could do some very strange things if there are any whitespace characters (other than <newline>) in the current line in your file. If the first character on a line is a <space> or a <tab> name will be set to an empty string as the rest of the line will be interpreted as a command to be executed with name set to an empty string. I presume what you wanted here was:

Code:

			   name="$line"

but I have no idea why you aren't just using line instead of making a copy of $line in name.

Furthermore, all expansions of the variables $i, $filename, and $name should be quoted.

Now for the fun part. Why didn't you tell us that in addition to creating new_file.txt your script also created file1.txt.txt, file2.txt.txt, and file3.txt.txt. The then side of your first if statement saves the name of the file you want to use as your destination file in the variable filename. The else side of that if statement replaces the destination saved in filename with the name of the current source file!!! There is nothing in the above script that writes anything to $1.txt. The touch command just updates the timestamp of that file if it already existed or creates it if it didn't exist; it doesn't alter its contents.

But, none of the above explains what you say has been happening with all of the suggested code that others have provided you in this thread. Have you verified that your early testing didn't empty out your source files? Please show us the output of the commands:

Code:

id
ls -l

This will tell us if my guess about creating file*.txt.txt is correct, tell us if there might be an permission problems, and lets us verify that your input files aren't empty.

Have any of the commands that have been suggested produced any diagnostic messages?

You said you want to concatenate files, but it looks like your code is trying to remove empty lines, remove blank lines, or maybe remove lines starting with any whitespace characters and maybe to remove all trailing whitespace from every line that is in the source files.

Please give us a precise definition of exactly what you want this script to do!

If you are just trying to create a file with the name specified by the 1st operand (with .txt added if the name didn't already end with that string) and concatenate all of the files named by the remaining operands into it, a much simpler solution would be something like:

Code:

#!/bin/ksh
dest=${1%.txt}.txt
shift
cat "$@" > "$dest"

If you're trying to delete empty lines while copying data to the destination file, change the last line to:

Code:

grep -v '^$' "$@" > "$dest"

I used the Korn shell, but this will also work with bash or any shell that performs the basic variable expansions required by the POSIX standards. (Of course this script should verify that it was invoked with at least two operands before trying the cat or grep, but I'll leave that as an exercise for the reader.