Script to filter by date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to filter by date
# 8  
Old 10-26-2012
The problem with find -mtime is that it checks just the last 86400 seconds, so it it starts at a different second some days, say recovering from an outage, you may get 2 or zero files. It is great for casual file browsing, but for tighter control, you are better off with a tight time bracket from two marker files or timestamps in file names or in the file (header, trailer, etc.).

If you avoid backing up file still open for write using 'file', and no file is reopened for addend, then you can make a list of backed up files and back up those not on the list. If you put the date, length or checksum into the list, too, you can detect those that changed and back them up again.
This User Gave Thanks to DGPickett For This Post:
# 9  
Old 10-26-2012
I'll try that too. But you know how do I include the command output into another file? Should I use a variable? How would this be?
Using the example command, what I should add more to it?
# 10  
Old 10-26-2012
I like pipes. One command produces a list into a pipe, the next modified that list, and so on down the pipe. A shell subscript can be on the pipe, like " | while read x y z ;do . . . done | ". Variables are nice to catch one field of one line, or the output of one command, for testing and output. You can process lists using the comm command, which takes sorted files or pipelines and tells you which lines are in file a, only, which in file b only, and which in both files. In bash and some systems' ksh, this drops a name of a pipe from a sort " <( sort file1 ) ". So, this produces a list of lines new in file b and passes it to a subshell loop for processing, one line $l at a time:
Code:
comm -13 <( sort filea ) <( sort fileb ) | while read l
do
 ...
done

This User Gave Thanks to DGPickett For This Post:
# 11  
Old 10-26-2012
---------- Post updated at 07:03 PM ---------- Previous update was at 06:51 PM ----------

Quote:
Originally Posted by hdegenaro
Sorry, my knowledge with scripts is small. I have a doubt of how to relate to my command upload script. Please help me to implement it. Follow my script that works only for files without modification.

Code:
#!/bin/sh

# Variables
FILE="$1"
CREATED="$2"
DATE=$(date)
FROM="Automatic Backup"
SUBJECT="Upload Ready"
EMAIL="admin@domain.com.br"
USER="root"
HOST="hostname.no-ip.info"
FILE_DIR_DB="/home/applications/backup/database"
FILE_DB="database.sql"
FILE_DIR_SITE="/home/applications/backup/site"
FILE_SITE="site.tar.gz"
PATH_DEST="/mnt/backup"

cat > /tmp/$$.sftp_tmp <<EOF
lcd $FILE_DIR_DB
put $FILE_DB $PATH_DEST
lcd $FILE_DIR_SITE
put $FILE_SITE $PATH_DEST
exit
EOF

sftp -P 2222 -b /tmp/$$.sftp_tmp $USER@$HOST

I could use the find command in this script? How would I use the command's output on it?
# 12  
Old 10-29-2012
DGPickett, can you help me?
# 13  
Old 10-30-2012
If you have sftp, you may have ssh, so you can run scripts there on the ssh command line or by creating or installing them there. You can ssh host find ... | whatever to see what is up there, compare it to what is down here, and make a list of files needing to be moved, read that all on the same pipeline and scp or sftp the files. I prefer scp, as it has a simple syntax.

Learning a little scripting is a good thing, enjoyable to most. It's a learn by doing, copying, one bi at a time, always growing thing. It allows you to modify scripts or make special scripts or command sequences when you need. Most of us use the same 20% of what shell tricks we know every day. I am a big fan of '... | while read var ; do . . . done | ....' and long pipelines: parallel processing, no temp files, good data and flow control. I learned 'while : do ... done' for unconditional loops, here, just this year, and I started UNIX in 1990 (with 23 years of other stuff not too different).

A summary email, even when nothing is backed up, periodically, is good form,, best practices, as it tells you it is running.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

Filter log file contents between date

Hi, Could you please provide me command to filter contents between date in a log file? Say for example, in a log file I want to capture contents between date May 01 from 5am to 9 am. OS -- Linux Regards, Maddy (1 Reply)
Discussion started by: Maddy123
1 Replies

2. Shell Programming and Scripting

How to filter date input in awk?

Hi, I've no problems filtering text in awk but whenever I try to filter date format such as 03022013, I couldn't get awk to filter the item out. Any tips? (8 Replies)
Discussion started by: commonwealth
8 Replies

3. Shell Programming and Scripting

Date capture & filter

Dear All, I am capturing system date and creating the file by using that time stamp, file is getting appended with checks of application & database check logs. But when the date is in between 1 to 9 both inclusive, it appends a single space to file name but after 9th it works fine. ... (5 Replies)
Discussion started by: pradeep84in
5 Replies

4. Shell Programming and Scripting

how to filter file for specific date

Hi My OS is solaris 64 bit 10, I have many files in a logs directory where we recive 40-50 logs every day. i have last 20 days file present in this directory. I want to move each day file to a particulaar directory whose name is appended with the date of file. eg Code: 1.txt file... (1 Reply)
Discussion started by: guddu_12
1 Replies

5. Shell Programming and Scripting

find command to filter specific type of files older than certain date.

Hi I need to find the list of files in a directory and to do some specific operations based on the type of files. suppose in a directory am having .dat , .log, .err, .rej file types. i need to filter out .dat and .log only which are older than six months. i used the below query but the... (2 Replies)
Discussion started by: msathees
2 Replies

6. Shell Programming and Scripting

Filter date and time form a file using sed command

I want to filter out the date and time from this line in a file. How to do this using sed command. on Tue Apr 19 00:48:29 2011 (12 Replies)
Discussion started by: vineet.dhingra
12 Replies

7. Shell Programming and Scripting

Grep script to filter

Hi, Wondering if anyone could help me with a simple script to filter out multiple things from a file. Right now I just have long lines of grep -v remove file | greg -v etc etc What I would like to do is have grep -v <run everything in a file> tofilter If that makes sense. Basically a... (2 Replies)
Discussion started by: kevin9
2 Replies

8. Shell Programming and Scripting

Filter by modify date.

I want to filter an "ls -al" command so it only shows me files with modify dates older than two weeks. What is the best way of doing this? (2 Replies)
Discussion started by: millerdc
2 Replies

9. UNIX for Dummies Questions & Answers

ftp by date filter

i would be grateful if someone could supply me with a shell script which performed a ftp but only retrieved files which had a created date greater than a specific date - is this possible with ftp? many thanks mc (1 Reply)
Discussion started by: campbem
1 Replies

10. Shell Programming and Scripting

Shell Script for searching files with date as filter

Hi , Assume today's date is 10-May-2002. I want to get a list of files which were last modified since 01-May-2002. If I run the script after 5 days, it should still list me the files modified from 01-May-2002 till today. I also plan to pass the date 01-May-2002 as an argument to the shell script... (3 Replies)
Discussion started by: kanakaraj_s
3 Replies
Login or Register to Ask a Question