Shell script to monitor tmp folder for uploads


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to monitor tmp folder for uploads
# 1  
Old 08-14-2008
Shell script to monitor tmp folder for uploads

Hello,

We have been having some issues with our users overwriting files, and then not having a backup. What I would love to do, is create a shell script to monitor /tmp, for uploads, and make a copy of the file they are trying to upload before the upload finishes.

Is this possible at all?
# 2  
Old 08-15-2008
Sounds possible, but I'm not sure I fully understand the requirement. If the upload is not finished the file would be incomplete. Do you still want to make copies of the partial files?

Also, is there a way you can recognise these files when compared to all the other junk that appears in /tmp?
# 3  
Old 08-15-2008
I'm trying to make a copy of the file that is being overwritten. So, if a user was trying to upload index.html to /www/index.html, I would want to make a backup of index.html in the same directory, before it's overwritten, (/www/index2008-08-09.html)
# 4  
Old 08-15-2008
Having a program check /proc/pid/fd and compare to your /tmp directory would be very slow. As such, you would have to rewrite the open() function (or sys_open() in the kernel). Either you create a program and use LD_PRELOAD to "stop" the open() function and only allow it to continue after a backup is made or you create a kernel module // modify the kernel to change the system call (sys_open()) or the actual internal function do_open() and also backup before allowing them to continue.

Or, if you just wanna keep it simple, read the documentation on your uploading program and it should allow backups of uploaded files.
# 5  
Old 08-15-2008
Tools Can you control the upload request?

If so, change the process for uploading a file.
Instead of a
Code:
cp junkfile /tmp/junkfile

Do a
Code:
sp_cp junkfile /tmp/junkfile

where you define a script called sp_cp that handles the making of a backup copy.


I have a script I call to make sequential backups. In other words, at first execution it will copy myscript to mysript.001; next execution, it will determine that the copy should create myscript.002

Take a look at the following, as it might give you some ideas.

Code:
#! /bin/bash
#
# script to backup a file in current directory
# will read directory analyzing scriptname.???
# take the last ??? & increment by one
# finally cp scriptname to scriptname.###
#
# 07/29/2008,jmg  Program Create

clear
# set variables

# verify correct number of parameters
if [ "$#" -ne 1 ]
   then
      echo "This program makes a copy of a program or script"
      echo "copying from scriptname to scriptname.001 (or next number)"
      echo "[located at /opt/dputils] "
      echo " "
      echo "Incorrect number of arguments"
      echo "command scriptname -- examples are"
      echo "script_bk pcsmain"
      echo "script_bk runedit"
      echo " "
      exit
fi

# read the parameters
scriptname="$1"

# determine if any archive copies currently exist
num_arc=$(ls -l "$scriptname".* 2>/dev/null | wc -l)
if [ $num_arc -ge 999 ]
   then
   echo "Warning - already 999 files; please review versions"
   echo "Program terminating"
   exit
fi

if [ $num_arc -eq 0 ]
   then
      arc_ver="001"
   else
      arc_v=$((num_arc+1))
      arc_ver=$(printf "%.3d" "$arc_v")
fi

# make the actual archive file
echo "Copying "$scriptname" to "$scriptname"."$arc_ver
cp $scriptname $scriptname"."$arc_ver

exit 0

# 6  
Old 08-17-2008
What you could do is have a process sitting checking for uploaded files on a regular basis, and as soon as they appear mv them to another unique name or location (subdirectory?) on the destination filesystem.

Once the ftpd has that output file open it should happily continue writing to it even if you rename it while the upload is still running.

The only thing is that the user uploading the file(s) may be confused by the fact that the file they just uploaded is no longer there, but if you give them sufficient warning that shouldn't be a big problem.
# 7  
Old 08-19-2008
Thanks for all the ideas. I'm not sure what solution I'm going to employ yet, this is a more complicated situation then I had originally thought.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request for Shell script to move files from Subfolder to Parent folder and delete sub folder

Hi Team, I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder. Eg: parent folder --> /Interface/data/test/IN Sub folder -->/Interface/data/test/IN/Invoice20180607233338 Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies

2. Homework & Coursework Questions

Need shell script to monitor files

Hi Experts, I am not good in writing script. Just stared. I am looking for shell script to check following parameters. 1) Number of files on remote Linux SUSE server.- Any directory and sub directory. 2) I should define number of files in script. Files should be variable. 3) Age of the... (2 Replies)
Discussion started by: ApmPerfMonitor
2 Replies

3. Shell Programming and Scripting

Shell script to find the GB files in /tmp directory in remote server

Hi, i need help on shell scripting. Main intention of the script is step 1: ssh to remote server Step 2: cd /tmp in remote server Step 3: in tmp i want to grep only files and directories which are in GB sizes All the servers list file is - tmpsrv.txt vi tmpsrv.txt ... (17 Replies)
Discussion started by: kumar85shiv
17 Replies

4. Shell Programming and Scripting

Sqlplus inside shell script writing to tmp file

Hi, facing an issue while calling sqlplus inside shell script. It for some reason goes to tmp file to write something and i get error as permission denied as i dont have access there. ANy idea why sqlplus writes in /tmp and how to change or stop this ? (2 Replies)
Discussion started by: rushikeshs
2 Replies

5. UNIX for Advanced & Expert Users

Large crout files created in /tmp folder

Hello All, We are having a archiving script which runs every monday. When ever the script runs it creates crout* files in /tmp folder. This file appends till /tmp become 100% full. We are facing this problem now very frequently. We caanot delete this files,we have to kill the script. when we... (5 Replies)
Discussion started by: Upendra Bhushan
5 Replies

6. Shell Programming and Scripting

How to monitor a command inside shell script

Hi All, Is there any way to monitor a command inside shell script ? I have a script inside which I have a tar command which zips around 200GB data. tar zcvf $Bckp_Dir/$Box-BaseBackup-$Day.tar.gz * --exclude 'dbserver_logs/*' --exclude postmaster.pid --exclude 'pg_xlog/*' I want to... (3 Replies)
Discussion started by: sussus2326
3 Replies

7. Shell Programming and Scripting

Shell Script to monitor folder and upload found files via FTP

Hi everyone! I'm in a need of a shell script that search for all files in a folder, move all those files to a temp folder, and upload those files via FTP. When the file transfer via FTP completes successfully, the file is moved to a completed folder. In case any of those files fails, the file... (4 Replies)
Discussion started by: pulsorock
4 Replies

8. Shell Programming and Scripting

how to monitor a folder using script

hi, i want to monitor a particuler folder and have to report is there any new file added to the dir or not? can anyone tell me how to monitor the folder using script. the script has to run in the background continiously. Is there any way to do it? (2 Replies)
Discussion started by: vij_krr
2 Replies

9. Shell Programming and Scripting

Shell script partitions space monitor

Hello friends, I'm newbie in shell scripting... Until now i have this script: #!/bin/sh emailok = email1@domain.com emailissue = email2@domain.com limit="50" df -h | grep -vE '^Filesystem' | awk '{print $1 " " $4 " " $5}'|while read val do partition=$(echo $val | awk '{print... (9 Replies)
Discussion started by: john_doe
9 Replies

10. Shell Programming and Scripting

monitor daily file uploads

hey all, i am a shell scripting n00b so bear with me. i got a server that every night uploads one file to a remote server. the file is prodserver_date_time. i would like to make a script, run by root on a daily cron job. i want it to determine if the file was received or not. no md5... (2 Replies)
Discussion started by: jweinraub
2 Replies
Login or Register to Ask a Question