Shell Programming and Scripting

Hi,

I have been trying to execute the below command by changing directory and then copying contents of one directory to another by doing some file name manipulations in between. However this isnt working since as soon as the statement completes it goes back to the original folder. Can someone help in telling me how to change a directory in a script so I can execute subsequent steps within it using and modifying a variable through the steps.

This works fine from command line, however gives bad substitution error from the script because of the issue explained above

Code:

for file in /ABC/DEF/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* /ABC/EDG/; done

Basically check for all files with name like CAT_TURNAROUND_201811120_075221_3.done in the folder ABC/DEF/*.done, then get all the files starting with just this part CAT_TURNAROUND_201811120_075221from this directory/ABC/DEF/ copied to /ABC/EDG/

Will really appreciate a solution. I have been trying to figure this out and checked various forums and tried alias, function etc. But it isn't working.

Moderator's Comments:

Please use CODE tags as required by forum rules!

Regarding the permanent "cd":
if you run a script like a program then it cannot change anything in the calling shell.
But you can source it

Code:

source script

or

Code:

. script

This will also work around the error you have got. The error you could also avoid by running the script with

Code:

/bin/bash script

or changing its shebang (first line) to #!/bin/bash

Welcome to the forum.



Not sure I understand the problem. Would you please carefully proofread your post and correct typos and logical errors (using the "edit" button at the lower right)? Like the ABC?DEF/*.done - is that a (wildcarded) directory or a sequence of files?


You don't seem to cd to anywhere, so why should it (what is it?) go "back to the original folder"? What exactly "isnt working"? Show the error messages.


And, give some data, like real filenames, as your sample "FilePrefix_20181120_075221_3.done" doesn't go well with the "${file:0:31}" shell expansion. Is your target directory meant to be an absolute path? Did you realize that your second expansion "${file##*.}" doesn't do anything as there's no . left in the file name?

On top of what my esteemed colleagues already said: you should not need to do any cd in any script! Use absolute pathes whenever possible (that effectively means: always), this will always guarantee that the script works the same from whereever you call it.

Instead of i.e.

Code:

cd /some/where
for file in * ; do
     cp $file ../../some/where/else
done

Do it this way:

Code:

fSource="/some/where"
fTarget="/some/where/else"

for fFile in "$fSource"/* ; do
     fFile="${fFile##*/}"            # /some/where/filename -> filename
     cp "${fSource}/${fFile}" "${fTarget}/${fFile}"
done

I hope this helps.

bakunin

Thanks for the responses Madeingermany and Bakunin. I am using #!/bin/bash on the first line of the script. I am new to scripting, I will lookup and try the source script method for this. Hoping that works, will let you know if it does.

Bakunin, I had tried using absolute paths, as my statement has multiple steps in between, the value of 'file' is lost in each step.

Code:

for file in "$fSource"/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* "$fPrepped"/; done

Moderator's Comments:

Please use CODE tags as required by forum rules!

I do not think that the value "file" is "lost", i just think that "file" at each step doesn't contain what you think it does. This is your code:

Code:

for file in "$fSource"/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* "$fPrepped"/; done

The first thing is: do NOT write one-liners. It is hard to understand what is going on when. So first, let us reformat:

Code:

for file in "$fSource"/*.done ; do
     file="${file%.*}"
     file="${file##*.}"
     cp "${file:0:31}"*.* "$fPrepped"/
done

Next thing: make sure that each step really does what you think it does. Notice that i took the final manipulation out of the cp command for clarity:

Code:

for file in "$fSource"/*.done ; do
     echo " after for: $file"
     file="${file%.*}" ; echo "first manipulation: $file"
     file="${file##*.}" ; echo "second manipulation: $file"
     file="${file:0:31}" ; echo "third manipulation: $file"
     echo "cp ${file}*.* $fPrepped/"
done

Now run that and observe: does "$file" in every step really contain what you expect it to contain? I suppose it doesn't. It is not "lost" IMHO. To quote from your post #1:

So let us assume the content of "$file" is "CAT_TURNAROUND_201811120_075221_3.done" or "/ABC/DEF/CAT_TURNAROUND_201811120_075221_3.done", depending on you using absolute pathes or not. Running the manipulations above would give:

Code:

file => CAT_TURNAROUND_201811120_075221_3.done
${file%.*} => CAT_TURNAROUND_201811120_075221_3
${file##*.} => CAT_TURNAROUND_201811120_075221_3
${file:0:31} => CAT_TURNAROUND_201811120_075221

As you can see the one manipulation in the middle does absolute nothing. It can't do anything, because you cut off any "." that is in a files name with your first manipulation! And the last manipulation depends entirely on the exact length of the filename. What would it do with, say, "HORSE_TURNAROUND_201811120_075221_3.done"?

Let us try again: you want to cut off the extension (".done") from the filename. Fine! Then just do so! You know from the for-loop that all the values of "$file" will have exactly ".done" as extension, so there is no need for ${file%.*}. Do it explicitly: ${file%.done}. The net manipulation doesn't do anything (and will never do anything, see above), so get rid of it. Now for the last one: it produces the correct result in your example but as i have shown you it might break with other values. Taking a step back: we want to cut off everything up to the last underscore ("_") from the filename. So let us do that: ${file%_*}.

Notice that we could have done so immediately without all the intermediary steps! This of course is only the case as long as the extension contains no underscore. So until you change your for-loop to something like for file in *.do_ne you can just skip all the steps and use just the last one i suggested.

Would the result be any different if we use absolute pathes? No, because we only cut off from the end where the filename is.

You might want to adapt your script accordingly.

I hope this helps.

bakunin