Copy and modify a file if the original has changed


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Copy and modify a file if the original has changed
# 1  
Old 06-27-2013
Copy and modify a file if the original has changed

Hi there,

I have built up my own little "cloud" for my family as the amount of computers grows all day. By now we use 3 smartphones, 2 notebooks und 4 PCs, so this "home cloud" was made to store all personal data (photos, documents,...) and do a backup once in a while. It is running on a Ubuntu Server 12.10 (not sure right now) as a headless configuration with encrypted ssh to work on it. A second HDD is turned on just once a day automatically to do a full backup of all the important data.

All devices have an UPnP / DLNA connection to this cloud which is running minidlna and samba. As there are 2 SmartTVs connected to the network as well by WLAN, we like to see and show our taken photos "on the big screens". For that, I would like to scale down all photos to max 1920x1080 and store them as JPG with 80% compression. Therefore I use the imagemagick package:
Code:
mogrify -format jpg -resize 1920x1080 -quality 80

The thing is, I want to store the photos originals as well. This is my now running setup / configuration:

- There are public shares in samba ("Photos", "Music", "Videos") as well as private ones. Photos has subfolders for each family member capable of using a camera. Thats where the originals are stored.

- There is a cronjob which checks every 30 Minutes for new files, converts them to jpg and resizes them if necessary. It looks like this:
Code:
*/30 8-23 * * * rsync -rltDvu --ignore-existing /files/samba/Photos/ /files/dlna/Pictures; 
     find /files/dlna/Pictures -type f -not -name "*.jpg" 
     -exec mogrify -format jpg -quality 100 {} \; 
     -exec rm {} \;; 
     find /files/dlna/Pictures -iname "*.jpg" 
     -exec mogrify -resize 1920x1080 -quality 80 {} \;

(Everything in one line, just modified for better reading.)

- There is another cronjob which deletes the downscaled versions if the original is removed. Runs once a week:
Code:
0 2 * * 1 rsync -rltDvu --delete --ignore-existing /files/samba/Photos /files/dlna/Pictures


This setup has 2 problems:

1st: When you modify a picture (like photoshop stuff), and overwrite the original, it wont be updated, as the sync process's --ignore-existing is set.
This tag is set so it wont sync all the originals because of downsized versions.

2nd: Running the job every 30 minutes is bad. If you load pictures into the cloud, you'll have to wait (worst case) 30 minutes to have them appear in DLNA clients. But running it more frequently is bad as well, as the job does some heavy load (cpu, ram) while running because of the find- and mogrify-stuff.


Now I thought about making a small program (I can code c++/c#), which simply creates a dummy file (like //cloud/dlna/.resize) in the cloud. The cronjob would look like:
Code:
* * * * * if [ -f /files/dlna/.resize ]; then 
     <<do all the magic here>>; 
     rm -f /files/dlna/.resize;
     fi

(Again, everyhing in one line.)

Would solve the 2nd problem, but not the first.


Do you have some suggestions how to solve problem #1? Something better than "find every picture and do a mogrify-thing" on it? I thought about something like creating dummy files somewhere to sync (compared to those) just newer or modified files. But I dont know how to do it.
# 2  
Old 06-27-2013
If you only need to run the job on Linux, perhaps you can use inotifywait to listen for file creation and modification, solving both problems.

Regards,
Alister
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Make a copy of a file when someone tries to modify the file

I wish to take a backup of the file when someone tries to modify a file say /tmp/test.txt using vi, vim, cat, copy, mv commands. I'm aware of inotify which will notify me of any changes to the file however, I wish to make a backup before and not after the changes are made to the file. I... (1 Reply)
Discussion started by: mohtashims
1 Replies

2. Shell Programming and Scripting

Script to echo "File permissions or ownership changed from required " when accidentally changed.

Hi All, I have to work in the late nights some times for server maintenance and in a hurry to complete I am accidentally changing ownership or permission of directories :( which have similar names ( /var in root and var of some other directory ).:confused: Can some one suggest me with the... (1 Reply)
Discussion started by: shiek.kaleem
1 Replies

3. UNIX for Advanced & Expert Users

copy original files from links

I have folder ABC and files in ABC are links. I want to create the same ABC folder in different path and copy the actual files from source ABC dir. Can anyone provide HP-UX command for this? note: cp -L is not working in HP-UX Thanks in advance. (1 Reply)
Discussion started by: venkatababu
1 Replies

4. Shell Programming and Scripting

command to copy original files from links in HP-UX

I have folder ABC and files in ABC are links. I want to create the same ABC folder in different path and copy the actual files from source ABC dir. Can anyone provide command for this? Thanks in advance. (2 Replies)
Discussion started by: venkatababu
2 Replies

5. UNIX for Dummies Questions & Answers

how to copy files and record original file location?

:EDIT: I think my post name should have been labeled: how to copy files and record original file location. not "retain". Hello, this is my first post! I searched the forums a lot before posting, but was unable to answer my question. Here's my problem: There are several hundred text files... (4 Replies)
Discussion started by: willie8605
4 Replies

6. Shell Programming and Scripting

Help with a shell script to modify one line and copy the next 9 to same file

Hi everyone, the problem is quite simple, yet I can't find an easy solution using awk. I need to search for a string in $3, then if I find this string, copy the line,modify $3, and copy the next 9 lines to the same file. My problem is in the copying of the lines... Finding and modifying... (5 Replies)
Discussion started by: Teroc
5 Replies

7. UNIX for Dummies Questions & Answers

Where can I get original copy of Unices

BSD, AFAIK, Linux, etc are developed/based upon UNIX... but they are not UNIX... I'm wandering where can I obtain a original copy of UNIX(without flavor) (1 Reply)
Discussion started by: laduch
1 Replies

8. UNIX for Advanced & Expert Users

command to copy files with original ownership

Hi, I need a command that to copy files from others and to keep files' ownership. Example: I copy file.txt from users "abc" to my local, and file.txt is own by user "abc" in local. Thanks in advance! (3 Replies)
Discussion started by: need_help
3 Replies

9. Shell Programming and Scripting

Trying to Copy Files Changed Recently

I have been toying around with a script that will copy all files altered in a development directory over to a testing directory and have been trying to construct the command to meet my needs. Basically I am using find in a directory to see what files have changed over the past 24 hours. Then if... (4 Replies)
Discussion started by: scotbuff
4 Replies

10. AIX

way to copy only changed files

Hello, we are going through a restructoring of file systems and will need to copy the contents of one file system to another box, then a few days later copy it again. Is there a way on the second copy to only copy files that have changed? Thanks in advance (3 Replies)
Discussion started by: zuessh
3 Replies
Login or Register to Ask a Question