Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Conditional delete

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Conditional delete
# 15  
Old 08-29-2018
Quote:
Originally Posted by RudiC
PCT=0.4 defines the necessary free space... this minus the actually available space yields the files' sizes to be deleted.
Thank you

Shall test it
# 16  
Old 08-29-2018
Is your data perhaps more compressible ?
Then perhaps a simple switch in your source zip command can save a lot more.

This is just a wild guess, perhaps to avoid more coding.

Regards
Peasant.
# 17  
Old 08-30-2018
Quote:
Originally Posted by RudiC
Assuming the stat command on your system allows for the -f (--file-system) option, and shamelessly stealing from Don Cragun's earlier post, and NOT being able to ultimately test this on my system, I'd propose this (printing some meaningful numbers for debug purposes) and ask you to comment back:
Code:
{ cd /backup; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {print Needed = ($1 * PCT - $2) * $3 
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime) print "echo rm " LastFile
                 KnownTime = T[1]
                 LastFile = $1
                 print SUM += $2 * $3
                 if (SUM >= Needed) exit
                }
 ' PCT=0.4

Once happy with what is delivered, we can pipe this into an sh command to be executed.
Hi RudiC,

I have tested your code . do you see if your wonderful solution to on right track to reach my goal). Its lot of pain to duplicate files and renamed it correctly to make sure it crosses 70% of discspace (as this is test vm)Smilie By the way, if every thing looks good, where I need to place actual rm command Smilie

Code:
[asp@abcsd34 ~]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        40G   30G   11G  74% /
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G   18M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           377M     0  377M   0% /run/user/1000
[asp@abcsd34 ~]$ ls
20180701_0012.tar.gz  20180705_1412.tar.gz  20180710_2112.tar.gz  20180716_0012.tar.gz  20180721_1412.tar.gz  20180726_2112.tar.gz  20180801_0012.tar.gz  20180805_0712.tar.gz
20180701_0712.tar.gz  20180705_2112.tar.gz  20180711_0012.tar.gz  20180716_1412.tar.gz  20180721_2112.tar.gz  20180727_0012.tar.gz  20180801_0712.tar.gz  20180805_1412.tar.gz
20180701_1412.tar.gz  20180706_0012.tar.gz  20180711_1412.tar.gz  20180716_2112.tar.gz  20180722_0012.tar.gz  20180727_1412.tar.gz  20180801_1412.tar.gz  20180805_2112.tar.gz
20180701_2112.tar.gz  20180706_1412.tar.gz  20180711_2112.tar.gz  20180717_0012.tar.gz  20180722_1412.tar.gz  20180727_2112.tar.gz  20180801_2112.tar.gz  20180806_0012.tar.gz
20180702_0012.tar.gz  20180706_2112.tar.gz  20180712_0012.tar.gz  20180717_1412.tar.gz  20180722_2112.tar.gz  20180728_0012.tar.gz  20180802_0012.tar.gz  20180806_0712.tar.gz
20180702_0712.tar.gz  20180707_0012.tar.gz  20180712_1412.tar.gz  20180717_2112.tar.gz  20180723_0012.tar.gz  20180728_2112.tar.gz  20180802_0712.tar.gz  20180806_1412.tar.gz
20180702_1412.tar.gz  20180707_1412.tar.gz  20180712_2112.tar.gz  20180718_0012.tar.gz  20180723_1412.tar.gz  20180729_0012.tar.gz  20180802_1412.tar.gz  20180806_2112.tar.gz
20180702_2112.tar.gz  20180707_2112.tar.gz  20180713_0012.tar.gz  20180718_1412.tar.gz  20180723_2112.tar.gz  20180729_1412.tar.gz  20180802_2112.tar.gz  20180807_0012.tar.gz
20180703_0012.tar.gz  20180708_0012.tar.gz  20180713_1412.tar.gz  20180718_2112.tar.gz  20180724_0012.tar.gz  20180729_2112.tar.gz  20180803_0012.tar.gz  abspacetest.sh
20180703_0712.tar.gz  20180708_1412.tar.gz  20180713_2112.tar.gz  20180719_0012.tar.gz  20180724_1412.tar.gz  20180730_0012.tar.gz  20180803_0712.tar.gz  
20180703_1412.tar.gz  20180708_2112.tar.gz  20180714_0012.tar.gz  20180719_1412.tar.gz  20180724_2112.tar.gz  20180730_1412.tar.gz  20180803_1412.tar.gz
20180703_2112.tar.gz  20180709_0012.tar.gz  20180714_1412.tar.gz  20180719_2112.tar.gz  20180725_0012.tar.gz  20180730_2112.tar.gz  20180803_2112.tar.gz
20180704_0012.tar.gz  20180709_1412.tar.gz  20180714_2112.tar.gz  20180720_0012.tar.gz  20180725_1412.tar.gz  20180731_0012.tar.gz  20180804_0012.tar.gz
20180704_1412.tar.gz  20180709_2112.tar.gz  20180715_0012.tar.gz  20180720_1412.tar.gz  20180725_2112.tar.gz  20180731_0712.tar.gz  20180804_0712.tar.gz
20180704_2112.tar.gz  20180710_0012.tar.gz  20180715_1412.tar.gz  20180720_2112.tar.gz  20180726_0012.tar.gz  20180731_1412.tar.gz  20180804_1412.tar.gz
20180705_0012.tar.gz  20180710_1412.tar.gz  20180715_2112.tar.gz  20180721_0012.tar.gz  20180726_1412.tar.gz  20180731_2112.tar.gz  20180805_0012.tar.gz
[asp@abcsd34 ~]$ cat abspacetest.sh
{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {print Needed = ($1 * PCT - $2) * $3
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime) print "echo rm " LastFile
                 KnownTime = T[1]
                 LastFile = $1
                 print SUM += $2 * $3
                 if (SUM >= Needed) exit
                }
 ' PCT=0.4
[asp@abcsd34 ~]$ sh abspacetest.sh
5.63358e+09
251428864
echo rm 20180701_0012.tar.gz
502857728
echo rm 20180701_0712.tar.gz
754286592
echo rm 20180701_1412.tar.gz
1005715456
1257144320
echo rm 20180702_0012.tar.gz
1508573184
echo rm 20180702_0712.tar.gz
1760002048
echo rm 20180702_1412.tar.gz
2011430912
2262859776
echo rm 20180703_0012.tar.gz
2514288640
echo rm 20180703_0712.tar.gz
2765717504
echo rm 20180703_1412.tar.gz
3017146368
3268575232
echo rm 20180704_0012.tar.gz
3520004096
echo rm 20180704_1412.tar.gz
3771432960
4022861824
echo rm 20180705_0012.tar.gz
4274290688
echo rm 20180705_1412.tar.gz
4525719552
4777148416
echo rm 20180706_0012.tar.gz
5028577280
echo rm 20180706_1412.tar.gz
5280006144
5531435008
echo rm 20180707_0012.tar.gz
5782863872
[asp@abcsd34 ~]$

So after every echo, the printed line is number of blocks to be removed ?.. I think file wise it showing first 3 files based on names marked for deletion..thats good.. I think i am almost there what I need.. Thank you very much.. Awaiting your comments.
# 18  
Old 08-30-2018
Unfortunately, neither the file system you want to "clean" is shown in your upfront df -h nor the files' sizes. But we see that (what I think are) the sizes are summed up, starting with the oldest, and leaving out the youngest per day, exiting when the target is reached or topped. What concerns me is that, obviously, the size of the files left out is summed up nevertheless (this is what I could not test over here). Please try this modification and report back:

Code:
                if (T[1] == KnownTime)  {print "echo rm " LastFile

                                         print SUM += $2 * $3
                                         if (SUM >= Needed) exit
                                        }

# 19  
Old 08-30-2018
Quote:
Originally Posted by RudiC
Unfortunately, neither the file system you want to "clean" is shown in your upfront df -h nor the files' sizes. But we see that (what I think are) the sizes are summed up, starting with the oldest, and leaving out the youngest per day, exiting when the target is reached or topped. What concerns me is that, obviously, the size of the files left out is summed up nevertheless (this is what I could not test over here). Please try this modification and report back:

Code:
                if (T[1] == KnownTime)  {print "echo rm " LastFile

                                         print SUM += $2 * $3
                                         if (SUM >= Needed) exit
                                        }

Hi RudiC,

Here the changed script , i ran the output is below

Code:
[asp@abcsd34 ~]$ sh abc.sh
5.6336e+09
[asp@abcsd34 ~]$ cat abc.sh
{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {print Needed = ($1 * PCT - $2) * $3
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime)
                  {print "echo rm " LastFile
                   print SUM += $2 * $3
                   if (SUM >= Needed) exit
                  }
                 }
' PCT=0.4
[asp@abcsd34 ~]$ sh abc.sh
5.6336e+09
[asp@abcsd34 ~]$

# 20  
Old 08-30-2018
Sorry to have been imprecise - you should have left
Code:
 KnownTime = T[1]
 LastFile = $1

where it was, just move the two lines (SUM & exit) into the if branch...
This User Gave Thanks to RudiC For This Post:
onenessboy
# 21  
Old 08-30-2018
Quote:
Originally Posted by RudiC
Sorry to have been imprecise - you should have left
Code:
 KnownTime = T[1]
 LastFile = $1

where it was, just move the two lines (SUM & exit) into the if branch...
Hi RudiC

here the modified version and output:

Code:
[asp@abcsd34 ~]$ cat abc.sh
{ cd /home/asp; stat -fc"%b %a %S" .; stat -c"%n %b %B" 2018*.tar.gz | sort; } | awk '
NR == 1         {print Needed = ($1 * PCT - $2) * $3
                 next
                }

                {split ($1, T, "_")
                 if (T[1] == KnownTime)
                 KnownTime = T[1]
                 LastFile = $1
                 {print "echo rm " LastFile
                   print SUM += $2 * $3
                   if (SUM >= Needed) exit
                  }
                 }
' PCT=0.4
[asp@abcsd34 ~]$ sh abc.sh
5.63412e+09
echo rm 20180701_0012.tar.gz
251428864
echo rm 20180701_0712.tar.gz
502857728
echo rm 20180701_1412.tar.gz
754286592
echo rm 20180701_2112.tar.gz
1005715456
echo rm 20180702_0012.tar.gz
1257144320
echo rm 20180702_0712.tar.gz
1508573184
echo rm 20180702_1412.tar.gz
1760002048
echo rm 20180702_2112.tar.gz
2011430912
echo rm 20180703_0012.tar.gz
2262859776
echo rm 20180703_0712.tar.gz
2514288640
echo rm 20180703_1412.tar.gz
2765717504
echo rm 20180703_2112.tar.gz
3017146368
echo rm 20180704_0012.tar.gz
3268575232
echo rm 20180704_1412.tar.gz
3520004096
echo rm 20180704_2112.tar.gz
3771432960
echo rm 20180705_0012.tar.gz
4022861824
echo rm 20180705_1412.tar.gz
4274290688
echo rm 20180705_2112.tar.gz
4525719552
echo rm 20180706_0012.tar.gz
4777148416
echo rm 20180706_1412.tar.gz
5028577280
echo rm 20180706_2112.tar.gz
5280006144
echo rm 20180707_0012.tar.gz
5531435008
echo rm 20180707_1412.tar.gz
5782863872
[asp@abcsd34 ~]$

But if you see output, it showing all files for given day..its not leaving last copy of the day...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Conditional delete -- New glitch

Hi Please dont consider this as duplicated post.. I am using below pattern to find delete files to bringdown disc size.. however how i can make sure ist going to correct folder and searching for files... while print "echo rm " LastFile correctly print files names for deletion, but when i... (7 Replies)
Discussion started by: onenessboy
7 Replies

2. UNIX for Beginners Questions & Answers

(g)awk conditional substitution issues when attempting to delete character

A portion of my input is as follows: 1087 IKON01,49 A WA- -1 . -1 . 0 W WA- -1 . -1 . 0 . -1 . -1 -1 -1 -1 -1 -1 W 1088 IKON01,49 A J.@QU80MW. 2... (2 Replies)
Discussion started by: jvoot
2 Replies

3. Shell Programming and Scripting

Conditional execution

Here's an interesting (to me, anyway) little puzzle. Background: I have a process that restores a number of large(ish) files from tape backup. As an individual file is being written, it is owned by root, then the ownership is changed when that file is complete. Since this process can take... (3 Replies)
Discussion started by: edstevens
3 Replies

4. Shell Programming and Scripting

Conditional search and delete using SED / Shell script

Hi, I want to perform a conditional search and remove my search string. Input string: "abcdaabcadgfaarstab" Character to search: "a" Condition: Remove all "a" in the input string except if it is "aa" Output string: "bcdaabcdgfaarstb" Can you please help me in this? (5 Replies)
Discussion started by: dominiclajs
5 Replies

5. Shell Programming and Scripting

if conditional statement

Hi, I have a script like this: sample.sh mapping=$1 if then echo "program passed" fi I'm running the above script as ./sample.sh pass The script is not getting executed and says "integer expression expected" Could anyone kindly help me? (2 Replies)
Discussion started by: badrimohanty
2 Replies

6. Shell Programming and Scripting

conditional statement

Hi all, The following code is to find if a list of numbers from one file are within the range in another file. awk -F, '\ BEGIN { while ((getline < "file2") > 0) file2=$3 } {for (col1 in file2) if ($0>=30 && $1<=45) print $0} ' FILE1 But where I have the number 30 and 45, I... (3 Replies)
Discussion started by: dr_sabz
3 Replies

7. Shell Programming and Scripting

If conditional

Hi, I am new to unix and shell scripting.In my script,there is a line using the "if" conditional - if && ; then do something Here "x" is a variable holding string value.If it is not equal to a comma or a string,only then I want to enter the "if" loop. But I am getting error while... (12 Replies)
Discussion started by: abhinavsinha
12 Replies

8. UNIX for Dummies Questions & Answers

If conditional

Hi, I am new to unix and shell scripting.In my script,there is a line using the "if" conditional - if && ; then do something Here "x" is a variable holding string value.If it is not equal to a comma or a string,only then I want to enter the "if" loop. But I am getting error while... (1 Reply)
Discussion started by: abhinavsinha
1 Replies

9. UNIX for Dummies Questions & Answers

conditional

conditional is not wworking can any one figure out what goes wrong xx1=`$ORACLE_HOME/bin/sqlplus -s apps/ostgapps1 2>/dev/null << EOF WHENEVER SQLERROR EXIT 1 set head off feedback off ; WHENEVER SQLERROR EXIT SQL.SQLCODE; select count(*) from CMS_INVOICE_ALL... (2 Replies)
Discussion started by: u263066
2 Replies

10. Shell Programming and Scripting

Conditional Statements

How can I compare two decimal values within a function using Bash? Function fun2 isn't comparing the decimal values. Is there a way to do this using Bash or Korn? #!/bin/bash set -x x=1 z=110 function fun1() { i=`bc << EOF 2>> /dev/null scale=3 ... (1 Reply)
Discussion started by: cstovall
1 Replies
Login or Register to Ask a Question