Using, modifying and reporting on timestamps to track changes in a filesystem
So, I have a filesystem with an ever growing number of files in it.
Multiple users are able to drop new files into this area and I need to know what's changed (a granularity of month by month is ok, more accurately would be better).
There are however a few issues, I can't just use the timestamps of the files as some of the timestamps are way in the past (and with some whacky users, way in the future).
I don't mind changing the timestamps and I'm also at the point where I know that all historical information is lost, I just want to know where I am going forward.
So one method I'm trying currently to implement is first of all a blanket timestamp change of everything to 11:11:11 on the 1st of the month.
This is easy enough, and will give me a baseline.
But then I need to run something (probably via cron) that once a day will go through the filesystem and find any files that don't have a timestamp of 11:11:11 on the 1st of ANY month and set them to 11:11:11 on the 1st of the CURRENT month.
Once I have a known set of timestamps for all files I can just use find to list out the changed files easily.
So the first question has to be... is there a better way to do this?
...and the second one is if so, let me know, if not... any suggestions?
check the find command and see if yours has the -newer option. if so, touch an empty marker file before you reset all the timestamps and then use something similar to following.
but then if then timestamps are marked prior to the last reset at first of the month, you might not get the desired results. although this is something you should be able to test now to determine.
HTH
Yeah, the problem with that is that in the future users could drop files onto the filesystem which have timestamps that bear no relation to reality (old files, new files, files with timestamps in the future or any time at all).
So searching for "newer than" or "older than" (which would be quite easy) breaks down pretty quickly.
Hence wanting to set the timestamps to something artificial (11:11:11 on 1st of a given month) to allow me to filter between what I already know and everything else.
or create a file with all known filenames as of the last scan, and next month create a new file and compare for differences. this would depend if users are using same or different filenames from month to month.
this month
next month, using 04 and 05 to indicate previous and current month numbers.
although, depending on the size of the filesystem, this could be done more frequently, adjusting the date variable accordingly.
I'd really been trying to do this without having to keep loads of historical text files laying about if I could (hence my focus on timestamps)...
This is a 20TB filesystem, about to scale up to 40TB in the next few months, so as I'm sure you can imagine, keeping the full text files of the listings each time could itself consume quite a bit of space. Especially if I squeeze the granularity down to daily.
But your suggestion seems to be the easiest way to get me started at least.
Thanks again... any other suggestions are of course welcome!
---------- Post updated at 03:51 PM ---------- Previous update was at 12:02 PM ----------
So, I've continued poking at this and I'm nearly at a solution which still involves the magical timestamps.
It's a bit of a hack, but it doesn't involve creating any extra files.
However I'm stuck at the final hurdle, it explodes horribly if the file names have spaces in them.... Any suggestions to get me to the final solution?
This works fine as long as the file names don't have spaces in, but as soon as that happens, it all goes wrong
hello
i'm using SOX to generate a spectrogram from a wave file with the command :
#sox file.wav -n spectrogram
is there a way to create a spectrogram using the same command but reading file timestamps instead of the namefile.wav , since name is changing every 4 hours? (it's saved with... (2 Replies)
So given filenames of varying lengths, I was wondering how I would remove or modify appended timestamps of the current date DD-MM-YY.
So say:
test_DD-MM-YY.txt
coolbeans_DD-MM-YY.pdf
And what I expect the output to be:
test.txt
coolbeans.pdf
Thanks :) (2 Replies)
Hello fellow Unix geeks,
I have been given a very urgent assignment in my office on writing a particular Shell script but I'm very much new to it.I would appreciate any help from you on solving this problem--which might seem very trivial to you.
The Unix flavour is a Sun Solaris one..(not... (6 Replies)
Dear all,
We are facing prolem when we are going to mount AIX filesystem, the system returned the following error
0506-307The AFopen call failed
: A file or directory in the path name does not exist.
But when we ls filesystems in the /etc/ directory it show
-rw-r--r-- 0 root ... (2 Replies)
Hi guys!
Could you tell me what's the difference of filesystem of Solaris to filesystem of Windows? I need to compare both.
I have read some over the net but it's so much technical. Could you explain it in a more simpler term? I am new to Solaris. Hope you help me guys.
Thanks! (4 Replies)
Hi,
I wanted to find out that in my database server which filesystems are shared storage and which filesystems are local. Like when I use df -k, it shows "filesystem" and "mounted on" but I want to know which one is shared and which one is local.
Please tell me the commands which I can run... (2 Replies)
Hi all,
currently , my root filesystem already reach 90 ++%
I already add more cylinder in the root partition as below
Part Tag Flag Cylinders Size Blocks
0 root wm 67 - 5086 38.46GB (5020/0/0) 80646300
1 swap wu 1 - ... (11 Replies)
Hello!
I have the following problem.
I read a file using perl, each line of this file has the fllowing format.
14/4/2008 8:42:03 πμ|10800|306973223399|4917622951117|1||1259|1|126|492|433||19774859454$
Th first field is the timestamp and the second field is the offset in seconds.
How can... (1 Reply)
Like the topic says, does anyone know if it is possible to check to see when an FTP only user has logged in? Because the shell is /bin/false and they are only using FTP to access the system doing a "finger" or "last" it says they have never logged in.
Is there a way to see when ftp users log in... (1 Reply)
I'm writting a script to find the difference between two timestamp. One field i get on delivery time of the file like 07:17 AM and other is my SLA time 06:30 AM
I need to find the difference between these two time (time exceeded to meet SLA). Need some suggestions. (8 Replies)