Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help on a fairly complicated shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help on a fairly complicated shell script
# 1  
Old 05-08-2017
Help on a fairly complicated shell script
Hello,

also with the help of some great users of this forum, I have created following shell script.

Code:
MM=120
GG=5000

# get size of directory
szm=$(du -s --block-size M  ./192.168.1.xxx | awk '{print int($0)}')
data=$(date --rfc-3339=seconds)

if [ $szm -ge $GG ] ; then  # too big delete older files

        find ./192.168.1.199 -type f -mmin +$MM -delete
        find ./192.168.1.199 -mindepth 1 -depth -type d -empty -delete

fi

Its purpose is to keep the size of the directory 192.168.1.xxx (where my network video recorders uploads the backup snapshot of the recordings) lower than a given value, i.e. GG oder 5000 mb.

This shell script is executed every hour by a cron.

What I would like to do is to change the script and make it delete xx amount of files (in terms of minutes or if not possible of mb) starting from the oldest ones.

Can anyone please help me?

Thank you so much.
Daniele
# 2  
Old 05-08-2017
You could try this:

Code:
MM=120
GG=5000
XX=20

# get size of directory
szm=$(du -s --block-size M  ./192.168.1.xxx | awk '{print int($0)}')
data=$(date --rfc-3339=seconds)

if [ $szm -ge $GG ] ; then  # too big delete older files

    find ./192.168.1.199 -type f -mmin +$MM -printf "%T@ %h/%f\\n" |
        sort -rn | head -$XX | sed 's/^[^ ]* //' |
        xargs --no-run-if-empty -d\\n rm
    find ./192.168.1.199 -mindepth 1 -depth -type d -empty -delete

fi


Here we have find print out the epoch modification time and file path like this:

Code:
1494273484.0000000000 ./192.168.1.199/Images/A42301
1494273404.0000000000 ./192.168.1.199/Images/A42310
1494273430.0000000000 ./192.168.1.199/Images/A42317
1494273416.0000000000 ./192.168.1.199/Images/A42322
1494273397.0000000000 ./192.168.1.199/Images/A42336
1494273261.0000000000 ./192.168.1.199/Images/A42344

Then sort on first column with numeric and descending to get oldest files first.
We then pass this to use head to select oldest $XX files, sed to cut off the epoch time and xargs to pass them to the rm command.
These 2 Users Gave Thanks to Chubler_XL For This Post:
dcaccount RavinderSingh13
# 3  
Old 05-08-2017
Quote:
Originally Posted by dcaccount
. . . make it delete xx amount of files (in terms of minutes or if not possible of mb) starting from the oldest ones. . . .
Do I read that correctly that the oldest files' sizes should sum up to $GG, and, when below, be deleted? Try (shamelessky stolen from Chubler_XL):

Code:
find . -type f -mmin +$MM -printf "%T@ %s %h/%f\\n" | sort -n | awk -vMXSZ=$GG '
(TOTSZ+=$2) < MXSZ      {print "rm -i", $3
                         next
                        }
                        {exit
                        }

If happy with the result, pipe it through sh!
Last edited by RudiC; 05-10-2017 at 05:53 PM.. Reason: typo
This User Gave Thanks to RudiC For This Post:
dcaccount
# 4  
Old 05-09-2017
Thanks for the kind replies.

It is a bit different, maybe I have been not 100% clear. Sorry.

I need to identify the oldest xx mb of files (or the oldest xx files if it is not possible to identity the oldest xx mb of files) inside the directory 192.168.xxx and then delete them.

Can you please recheck your suggestions?
Thanks
# 5  
Old 05-09-2017
So - what in your new specification is not satisfied by the proposal in post#3?
# 6  
Old 05-09-2017
Thanks for replying.
I apologize, being a newbie I understand slowly.
You wrote "the oldest files' sizes should sum up to $GG, and, when below, be deleted?".

Actually every time I launch the script, it has to remove the oldest xx mb of files.

You wrote "when below, be deleted".

This is what I do not understand.
Thanks
# 7  
Old 05-09-2017
The script takes a list of files by modification date, oldest first, sums up their sizes, and prints them together with the rm command to stdout. If the $GG value is exceeded, the printed list stops.

Did you run the script? Does the output satisfy your request? If yes, pipe it through sh to actually perform the deletions.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Assistance on complicated perl script

As a followup to my previous thread, I'm trying to make a complicated perl script that involves storing information from a text file into a hash, and giving the user the ability to change the information present/write the information currently inside the hash to a new file. This is the code I've... (8 Replies)
Discussion started by: Eric1
8 Replies

2. Solaris

Solaris? Is it really this complicated?

Hello Everybody, I am a mid-level Linux administrator trying to get a hold of Solaris. So far, my experience has been painful. I think it's confusing and it's just a big mess... To be more specific, I have noticed that whenever I install a package... it never followed a specific pattern.... (4 Replies)
Discussion started by: EssenceNY
4 Replies

3. Shell Programming and Scripting

Script Search replace - complicated

I have a text file for which i need a script which does some fancy search and replace. Basically i want to loop through each line, if i find an occurance of certain string format then i want to carry on search on replace another line, once i replaced this line i will contine to search for the... (7 Replies)
Discussion started by: kelseyh
7 Replies

4. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies

5. Programming

Does realloc free fairly?

Hello, my program works properly but valgrind tells me I am not freeing allocated memory. I think the problem is in realloc. I am pretty sure I do something wrong with realloc, because I changed it a bit and valgrind noticed less errors (that the program wasn't working properly with less errors... (3 Replies)
Discussion started by: samciz
3 Replies

6. Shell Programming and Scripting

need help with script - output to file, but complicated

ok, so what i want to do is make a script that will do the following: take out from a command in the terminal put that output into a text file already on my computer. the only thing is that i need to put the output in the file kinda weird: i need to take it and put each character of output... (13 Replies)
Discussion started by: twoodcc
13 Replies

7. UNIX for Dummies Questions & Answers

complicated(?) grep

Hi all, I need help to figure out this grep. I would like to search all files in a directory for the occurrence of abc AND xyz. I started with this: grep -i abc * | grep xyz But this only returns lines which contain both values on the same line. So I am thinking that I need to do... (5 Replies)
Discussion started by: khearnen
5 Replies

8. UNIX for Dummies Questions & Answers

Fairly new to scripting

Hi, my team has existing ftp scripts that basically ftp files from one server to the other. Can anyone show me how to add some lines where if the ftp fails, an email could be sent out to myself and others on my team so that we are alerted? Thanks in advance. (11 Replies)
Discussion started by: ocanyc
11 Replies

9. Shell Programming and Scripting

Very complicated script..

I have a script that I need to create tha involves moving files and renaming them(see previous post) Are there any websites with user made shell scripts? (5 Replies)
Discussion started by: rocinante
5 Replies

10. Shell Programming and Scripting

Need help with complicated script (reading directories, extracting data)

Hi, people. I was searching script tutorials/examples and found this forum, and thought to ask for help, because there seems to be many wise people here. (Try to bare with me, I'm a Windows user, so this stuff is somewhat strange to me, OK? :rolleyes: ) Anyways, I need to create a script that... (3 Replies)
Discussion started by: Gill Bates
3 Replies
Login or Register to Ask a Question