Conditional delete -- New glitch


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Conditional delete -- New glitch
# 1  
Conditional delete -- New glitch

Hi
Please dont consider this as duplicated post..

I am using below pattern to find delete files to bringdown disc size.. however how i can make sure ist going to correct folder and searching for files... while print "echo rm " LastFile correctly print files names for deletion, but when i remove echo and left with rm then its complaining no such file for directory.

Code:
{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {Needed = ($1 * PCT - $2) * $3
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime)
                    {print "rm " LastFile
                    SUM += $2 * $3
                    if (SUM >= Needed) exit
                    }
                 KnownTime = T[1]
                 LastFile = $1
                }
' PCT=0.4  |sh

any suggestions
# 2  
As usual, no meaningful advice possible without meaningful data. What's your directory contents, what's the file that seems to be missing? What's the output of the two stat commands? What's the output of the entire script (before piping it into sh)?
# 3  
As you have been told before: Please tell us what operating system and shell you're using whenever you start a new thread in the Shell Programming and Scripting forum!

Warning: The following paragraph is only partially correct. For full detail, see post #5 in this thread.

Assuming that you're using a standards conforming shell, using cd inside braces only affects the current working directory of other commands run inside those braces. Therefore, the awk and shell commands in your pipeline will be run in the directory in which you were located when you invoked your script; not necessarily in /home/asp. Try changing the first line in your script from:
Code:
{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '

to:
Code:
cd /home/asp && { stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '

This User Gave Thanks to Don Cragun For This Post:
# 4  
Quote:
Originally Posted by RudiC
As usual, no meaningful advice possible without meaningful data. What's your directory contents, what's the file that seems to be missing? What's the output of the two stat commands? What's the output of the entire script (before piping it into sh)?
With Don Cragun suggestion , put Path ,&& before the braces worked. Smilie

Thank you Don Cragun for help

Last edited by onenessboy; 10-03-2018 at 09:28 AM..
# 5  
Quote:
Originally Posted by onenessboy
With Don Cragun suggestion , put Path ,&& before the braces worked. Smilie

Thank you Don Cragun for help
I'm glad it is working for you, but my explanation in post #3 isn't entirely correct and I'm not sure that the "problem" would appear in all shells. When I originally wrote post #3, I was thinking of ommands in a compound list surrounded by parentheses; in that case the commands in the compound list are executed in a subshell environment and what you were doing would never have worked in that case.

But, commands in a compound list surrounded by braces are executed in the current process environment. But the order in which commands are executed in a pipeline isn't as clearly specified. If the cd in the compound list is executed before the sh at the end of the pipeline is invoked, the pipeline as written would work. If the cd in the compound list is executed after the sh at the end of the pipeline is invoked, the pipeline as written would fail with the diagnostic messages the submitter stated. Therefore, the code you showed us in post #1 has a race condition that may work sometimes and fail at other times. Your choice of shells might alter the likelihood of success or failure, but the race condition would always be there.

By putting the cd before the pipeline in an AND list, we are forcing it to be completed successfully before the pipeline is started; therefore, we know that all elements of the pipeline have to be executed in the desired working directory
# 6  
Quote:
Originally Posted by Don Cragun
I'm glad it is working for you, but my explanation in post #3 isn't entirely correct and I'm not sure that the "problem" would appear in all shells. When I originally wrote post #3, I was thinking of ommands in a compound list surrounded by parentheses; in that case the commands in the compound list are executed in a subshell environment and what you were doing would never have worked in that case.

But, commands in a compound list surrounded by braces are executed in the current process environment. But the order in which commands are executed in a pipeline isn't as clearly specified. If the cd in the compound list is executed before the sh at the end of the pipeline is invoked, the pipeline as written would work. If the cd in the compound list is executed after the sh at the end of the pipeline is invoked, the pipeline as written would fail with the diagnostic messages the submitter stated. Therefore, the code you showed us in post #1 has a race condition that may work sometimes and fail at other times. Your choice of shells might alter the likelihood of success or failure, but the race condition would always be there.

By putting the cd before the pipeline in an AND list, we are forcing it to be completed successfully before the pipeline is started; therefore, we know that all elements of the pipeline have to be executed in the desired working directory
Dear Don Cragun

Thanks for detailed explanation. Actually what happened was , if you remember your earlier solution on the thread (the code that i was using).. at that time, for testing I have placed those files in / root directory as there was no mount point seperately for backup on test VM.

Now coming into solution implementation, this script is executed from central server on different remote machines (where I will pass IP of that remote) via ansible script..it will go to remote place and then execute the code. What happened here is, when executed from central server it actually login (ssh) into remote server specified and should actually go to backup directiry(which is separate mount), instead that ssh going into default user directory not the backup mount point where our solution get executed. I believe that is the reason, where my testing machine i placed those files in default root directory and it was working, as default directory is same,where as in real implementation it has to go into /backup mount instead, i guess its going to user default directory on remote (ssh remotely going to default user directory and searching for files , obviously saying no files found).

And coming to the warning you mentioned, Yes, I still agree and not sure whether it will work on machines. (as of now my guess is above mentioned reason). We are using all Redhat 7.5 latest boxes(bash shell), where my test machine also redhat 7.5 one.

However as of now you && solution before braces works as expected, may be i need to verify on different boxes how it responds Smilie Thank you very much again
# 7  
Maybe I wasn't clear. The change suggested at the end of post #3 should always work; the only thing that was wrong in post #3 was my explanation of why it would work. The correct explanation of why the suggestion in post #3 gets rid of the race condition in your code is presented in post #5.
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #871
Difficulty: Medium
Hello World in Lisp is coded as: [ print "Hello world" ]
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Conditional delete

Hi Friends, I have somefiles like 20180720_1812.tar.gz 20180720_1912.tar.gz 20180720_2012.tar.gz 20180720_2112.tar.gz 20180721_0012.tar.gz 20180721_0112.tar.gz 20180721_0212.tar.gz 20180721_0312.tar.gz in a directory and so on..these files gets created every 3 hours where as... (28 Replies)
Discussion started by: onenessboy
28 Replies

2. UNIX for Beginners Questions & Answers

(g)awk conditional substitution issues when attempting to delete character

A portion of my input is as follows: 1087 IKON01,49 A WA- -1 . -1 . 0 W WA- -1 . -1 . 0 . -1 . -1 -1 -1 -1 -1 -1 W 1088 IKON01,49 A J.@QU80MW. 2... (2 Replies)
Discussion started by: jvoot
2 Replies

3. Shell Programming and Scripting

Conditional search and delete using SED / Shell script

Hi, I want to perform a conditional search and remove my search string. Input string: "abcdaabcadgfaarstab" Character to search: "a" Condition: Remove all "a" in the input string except if it is "aa" Output string: "bcdaabcdgfaarstb" Can you please help me in this? (5 Replies)
Discussion started by: dominiclajs
5 Replies

4. Shell Programming and Scripting

use statements and system glitch

hi, i have a perl script that runs as a cron job... Once in a while, the perl script fails with: Can't locate <module>.pm in @INC (@INC contains: .............) because one of the perl modules specified in the "use" statements is unavailable due to an NFS glitch. Is there some... (1 Reply)
Discussion started by: Andrewkl
1 Replies

5. Shell Programming and Scripting

If conditional

Hi, I am new to unix and shell scripting.In my script,there is a line using the "if" conditional - if && ; then do something Here "x" is a variable holding string value.If it is not equal to a comma or a string,only then I want to enter the "if" loop. But I am getting error while... (12 Replies)
Discussion started by: abhinavsinha
12 Replies

6. UNIX for Dummies Questions & Answers

If conditional

Hi, I am new to unix and shell scripting.In my script,there is a line using the "if" conditional - if && ; then do something Here "x" is a variable holding string value.If it is not equal to a comma or a string,only then I want to enter the "if" loop. But I am getting error while... (1 Reply)
Discussion started by: abhinavsinha
1 Replies

7. UNIX for Dummies Questions & Answers

conditional

conditional is not wworking can any one figure out what goes wrong xx1=`$ORACLE_HOME/bin/sqlplus -s apps/ostgapps1 2>/dev/null << EOF WHENEVER SQLERROR EXIT 1 set head off feedback off ; WHENEVER SQLERROR EXIT SQL.SQLCODE; select count(*) from CMS_INVOICE_ALL... (2 Replies)
Discussion started by: u263066
2 Replies

8. Ubuntu

Ubuntu costumized like OSX glitch

I have a laptop running Ubuntu Gusty. I have recently used applications such as AWN, Emerald, etc... to create a Mac OS X Leopard like environment. It works almost perfectly except for one little glitch. When I open most of my apps/windows, they all open with the top bar(with includes close,... (5 Replies)
Discussion started by: Texasone
5 Replies

9. Shell Programming and Scripting

need a little kick with sed, got it almost but on glitch

hi friends. yo i have a textfile with urlīs in it sometimes middle in text, sometimes one url alone is a line i append a string right behind the domain name so that http://unix.com becomes http://unix.com.APPENDTHIS on all occasions. i use this sed-line to achieve this: sed -i... (3 Replies)
Discussion started by: scarfake
3 Replies

Featured Tech Videos