Difficulty with CAT redirection in script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Difficulty with CAT redirection in script
# 1  
Old 01-23-2017
Linux Difficulty with CAT redirection in script

I have not been able to append the contents of many files into one file. I have executed the CAT command shown below separately substituting an actual path and file name for the array variable to verify that I have the syntax correct. The bottom line - nothing is happening with CAT. I am running the script with sudo. Please excuse all the debug statements - just verifying that the variables contain valid values. I am running this on Ubuntu 14.04.

Code:
#!/bin/bash
echo ""
echo "==============================================================================="
echo "This script copies all of the LocalConfig.pri files contents into another file."
echo "==============================================================================="
echo ""
cd /
echo "Changed directory to ..."
pwd

# Assign 'locate' results to a variable.
output=$(locate -b "LocalConfig.pri")

# Parse the $output variable into an array. Each line is a full path.
while read -r line; do outputArray+=("$line"); done <<<"$output"

# Loop through the array.
for idx in "${!outputArray[@]}"; do
    # Print array element index and contents.
    printf ' Output number %d is %s' "$idx" "${outputArray[idx]}"
    printf '\n'
    # Cat each file into one 'master' file.
    echo -e "\tconcatenating\n\t\t ${outputArray[idx]} \n\tinto \n\t\t./LocalConfigMaster.pri ..."
    cat "${outputArray[idx]}" >> ./LocalConfigMaster.pri
done

# 2  
Old 01-23-2017
Since you have all of this debugging information, why don't you share it with us so we can see what is going on? (Did you consider using set -xv instead of adding all of those debugging statements?)
# 3  
Old 01-23-2017
Are you seeing any errors like ./LocalConfigMaster.pri: Permission denied?

You could also avoid using arrays and use a for loop to set your file like this:

Code:
idx=0
locate -b "LocalConfig.pri" | while read file
do
    # Print element number and contents.
    printf 'Output number %d is %s\n' $((++idx)) "$file"
    # Cat each file into one 'master' file.
    echo -e "\tconcatenating\n\t\t $file \n\tinto \n\t\t./LocalConfigMaster.pri ..."
    cat "$file" >> ./LocalConfigMaster.pri
done

# 4  
Old 01-23-2017
Don't you mean to use echo instead of cat? Or is the array element a filename? with a complete path....

Another point: You may want to declare outputArray as an array:

Code:
outputArray=()
# or
declare -a outputArray

I assume this is bash 3.2 or higher. Arrays were somewhat experimental in v3.1 bash.
What output do you get/see in stdout? Please post a sample.
# 5  
Old 01-23-2017
The debug output is nothing but full path file names followed by my target file name which is
Quote:
./LocalConfigMaster.pri
. There are no errors and there is no output generated by the cat command. (from the command line) If I substitute anyone of these full path files names for the array variable, the cat command works just fine. These full path file names look something like this
Quote:
/fouo/release-develop/project1/LocalConfig.pri
. Again, works great from the command line, does nothing inside this script of mine. I am unfamiliar with
Quote:
set -xv
but not for long. :-) I am fairly new to the bash world. Thanks for your help and patience Don, it is much appreciated.

---------- Post updated at 09:51 PM ---------- Previous update was at 09:45 PM ----------

Quote:
Originally Posted by jim mcnamara
Don't you mean to use echo instead of cat? Or is the array element a filename? with a complete path....

Another point: You may want to declare outputArray as an array:

Code:
outputArray=()
# or
declare -a outputArray

I assume this is bash 3.2 or higher. Arrays were somewhat experimental in v3.1 bash.
What output do you get/see in stdout? Please post a sample.
Thanks Jim. The samples I read suggested cat. The array is working, I can verify that - I am getting the expected results, it's just this cat command that is doing nothing. I provided a small one-line sample output in my reply to Don. Thank you for the suggestions though. I'm a little green when it comes to bash. And I do not know what version it is but I'll find out.

---------- Post updated at 09:53 PM ---------- Previous update was at 09:51 PM ----------

Quote:
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
---------- Post updated at 09:55 PM ---------- Previous update was at 09:53 PM ----------

Quote:
Originally Posted by Chubler_XL
Are you seeing any errors like ./LocalConfigMaster.pri: Permission denied?

You could also avoid using arrays and use a for loop to set your file like this:

Code:
idx=0
locate -b "LocalConfig.pri" | while read file
do
    # Print element number and contents.
    printf 'Output number %d is %s\n' $((++idx)) "$file"
    # Cat each file into one 'master' file.
    echo -e "\tconcatenating\n\t\t $file \n\tinto \n\t\t./LocalConfigMaster.pri ..."
    cat "$file" >> ./LocalConfigMaster.pri
done

At first I did see the permission denied error but that went away after invoking the script with sudo. And I appreciate the tips on tightening up this code. Thanks.
# 6  
Old 01-24-2017
Quote:
Originally Posted by demmith
The debug output is nothing but full path file names followed by my target file name which is.
In general it is usually a good idea to just post the information instead of telling us about it. The reason is: being a newcomer (as per your own admission) you might lack the knowledge of what is vital information and what is not. Show the scripts code (you did that), then run it and copy its output from start to end (ideally along with the return code it produced) here, enclosed in CODEtags, like this:

Code:
# ./myscript.sh
bla
foo
error: unable to make the flurbishes grommicking

# echo $?
2

Quote:
Originally Posted by demmith
And I appreciate the tips on tightening up this code. Thanks.
Here is one more such suggestion: are you sure you want to add always to the file in question? Wouldn't you rather want to start an empty file for every run of the script and then fill the contents of all the other files to this one?

If so, supposing outputarray is an array variable containing path names AND there are no special characters (blanks, tabs, ...) in the file names you can do:

Code:
cat ${outputarray[*]} > /some/file

${outputarray[*]} will expand to a list of all the array elements and cat takes an (open-ended) list of filenames to process as arguments. There are some restrictions on this and if the array has several hundreds of entries this method might break eventually (the exact point depending on your systems configuration), but for a handful (anything less than hundred for sure) of filenames this works well.

I hope this helps.

bakunin

[code]
This User Gave Thanks to bakunin For This Post:
# 7  
Old 01-24-2017
Note that the command:
Code:
cat ${outputarray[*]} > /some/file

may fail if any of the filenames in the array contain whitespace characters. To protect against this case, the following is usually safer:
Code:
cat "${outputarray[@]}" > /some/file

This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Input redirection within bash script

Hi, when I try to redirect input and the command is described as a string within an array redirection does not work. why? #!/bin/bash dir=("tail < ./hello.txt") tail < ./hello.txt #works ${dir} #does not work (2 Replies)
Discussion started by: heinzel
2 Replies

2. Shell Programming and Scripting

Input redirection script

Hi, #!/bin/bash while ; do rm -f /tmp/pipe mkfifo /tmp/pipe ./yuv4mpeg_to_v4l2 < /tmp/pipe & mplayer tom_and_jerry.mp4 -vf scale=480:360 -vo yuv4mpeg:file=/tmp/pipe sleep 65; done When I run this - after mplayer finishes playing video it says - Exiting... (End of... (2 Replies)
Discussion started by: ashokvpp
2 Replies

3. Shell Programming and Scripting

STDOUT and STDERR redirection within a script

Hello all, I have a for loop executing in a script that I want to redirect STDOUT to screen and to file, while directing STDERR to the bit bucket. Here is the general sentax of what I'm doing: for i in thingy do some_command ${i} done 1>&1 | tee ${LOGFILE} 2> /dev/null What I am... (2 Replies)
Discussion started by: LinuxRacr
2 Replies

4. Shell Programming and Scripting

How to check for Input Redirection in my script?

All, I have a requirement to write a script where I check for Input redirection when the script was executed, based on which I handle my logic. Below is the example: my.script #! /bin/ksh # Not sure how to frame the if condition below if ; then echo "Input Redirected from a file" ... (7 Replies)
Discussion started by: bharath.gct
7 Replies

5. Shell Programming and Scripting

Difficulty using "execute immediate" in shell - Sql script

Hello members, I get an unexpected "end of file" error while trying to execute the following piece of code in the bash / ksh shell. I'm assuming the problem is with using the "execute immediate statement" #! /bin/bash tname="table" for i in * do sqlstr="create table $tname$i as select... (3 Replies)
Discussion started by: novice82
3 Replies

6. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

7. UNIX for Dummies Questions & Answers

Regarding redirection using cat.

The behavior of the following 2 operations is unexpected. K1 and K2 are files here :- 1) cat < K1 K2 The above operation should actually display contents of the both files. But it gives the contents of K2 only. How is that ? 2) cat > K1 K2 Above operation takes the contents of... (2 Replies)
Discussion started by: marconi
2 Replies

8. UNIX for Dummies Questions & Answers

Difference between cat , cat > , cat >> and touch !!!

Hi Can anybody tell the difference between Difference between cat , cat > , cat >> and touch command in UNIX? Thanks (6 Replies)
Discussion started by: skyineyes
6 Replies

9. UNIX for Dummies Questions & Answers

redirection to tty** with cat

I tried to cat a file to another user that was logged in, but I received an error message that displayed something like: %cat funny > /dev/ttyp3 zsh: permission denied: /dev/ttyp3 Thank you all for your help (1 Reply)
Discussion started by: zorro
1 Replies
Login or Register to Ask a Question