Shell Programming and Scripting

Hi
Please dont consider this as duplicated post..

I am using below pattern to find delete files to bringdown disc size.. however how i can make sure ist going to correct folder and searching for files... while print "echo rm " LastFile correctly print files names for deletion, but when i remove echo and left with rm then its complaining no such file for directory.

Code:

{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {Needed = ($1 * PCT - $2) * $3
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime)
                    {print "rm " LastFile
                    SUM += $2 * $3
                    if (SUM >= Needed) exit
                    }
                 KnownTime = T[1]
                 LastFile = $1
                }
' PCT=0.4  |sh

any suggestions

As usual, no meaningful advice possible without meaningful data. What's your directory contents, what's the file that seems to be missing? What's the output of the two stat commands? What's the output of the entire script (before piping it into sh)?

As you have been told before: Please tell us what operating system and shell you're using whenever you start a new thread in the Shell Programming and Scripting forum!

Warning: The following paragraph is only partially correct. For full detail, see post #5 in this thread.

Assuming that you're using a standards conforming shell, using cd inside braces only affects the current working directory of other commands run inside those braces. Therefore, the awk and shell commands in your pipeline will be run in the directory in which you were located when you invoked your script; not necessarily in /home/asp. Try changing the first line in your script from:

Code:

{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '

to:

Code:

cd /home/asp && { stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '

With Don Cragun suggestion , put Path ,&& before the braces worked.

Thank you Don Cragun for help

I'm glad it is working for you, but my explanation in post #3 isn't entirely correct and I'm not sure that the "problem" would appear in all shells. When I originally wrote post #3, I was thinking of ommands in a compound list surrounded by parentheses; in that case the commands in the compound list are executed in a subshell environment and what you were doing would never have worked in that case.

But, commands in a compound list surrounded by braces are executed in the current process environment. But the order in which commands are executed in a pipeline isn't as clearly specified. If the cd in the compound list is executed before the sh at the end of the pipeline is invoked, the pipeline as written would work. If the cd in the compound list is executed after the sh at the end of the pipeline is invoked, the pipeline as written would fail with the diagnostic messages the submitter stated. Therefore, the code you showed us in post #1 has a race condition that may work sometimes and fail at other times. Your choice of shells might alter the likelihood of success or failure, but the race condition would always be there.

By putting the cd before the pipeline in an AND list, we are forcing it to be completed successfully before the pipeline is started; therefore, we know that all elements of the pipeline have to be executed in the desired working directory

Dear Don Cragun

Thanks for detailed explanation. Actually what happened was , if you remember your earlier solution on the thread (the code that i was using).. at that time, for testing I have placed those files in / root directory as there was no mount point seperately for backup on test VM.

Now coming into solution implementation, this script is executed from central server on different remote machines (where I will pass IP of that remote) via ansible script..it will go to remote place and then execute the code. What happened here is, when executed from central server it actually login (ssh) into remote server specified and should actually go to backup directiry(which is separate mount), instead that ssh going into default user directory not the backup mount point where our solution get executed. I believe that is the reason, where my testing machine i placed those files in default root directory and it was working, as default directory is same,where as in real implementation it has to go into /backup mount instead, i guess its going to user default directory on remote (ssh remotely going to default user directory and searching for files , obviously saying no files found).

And coming to the warning you mentioned, Yes, I still agree and not sure whether it will work on machines. (as of now my guess is above mentioned reason). We are using all Redhat 7.5 latest boxes(bash shell), where my test machine also redhat 7.5 one.

However as of now you && solution before braces works as expected, may be i need to verify on different boxes how it responds Thank you very much again

Maybe I wasn't clear. The change suggested at the end of post #3 should always work; the only thing that was wrong in post #3 was my explanation of why it would work. The correct explanation of why the suggestion in post #3 gets rid of the race condition in your code is presented in post #5.