Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep from file modified one minute ago

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep from file modified one minute ago
# 1  
Old 07-06-2016
Grep from file modified one minute ago
Hello,

I have a list of files, an example below:

Code:
-rw-r--r--   1 smf_oper esg       910773 Jul  6 12:52 am1slc02_ACS_201607061242571_20346.cdr
-rw-r--r--   1 smf_oper esg       995838 Jul  6 12:52 am1slc01_ACS_201607061243125_19895.cdr
-rw-r--r--   1 smf_oper esg       557235 Jul  6 12:52 am1slc02_ACS_201607061243110_20309.cdr
-rw-r--r--   1 smf_oper esg       934318 Jul  6 12:52 am1slc02_ACS_201607061243164_20340.cdr
-rw-r--r--   1 smf_oper esg      1018777 Jul  6 12:52 am1slc02_ACS_201607061243160_20350.cdr
-rw-r--r--   1 smf_oper esg      1049209 Jul  6 12:52 am1slc02_ACS_201607061247501_20318.cdr
-rw-r--r--   1 smf_oper esg        15611 Jul  6 12:53 am1slc01_ACS_201607061243130_19939.cdr
-rw-r--r--   1 smf_oper esg      1048796 Jul  6 12:53 am1slc01_ACS_201607061247517_19953.cdr
-rw-r--r--   1 smf_oper esg      1048826 Jul  6 12:53 am1slc01_ACS_201607061247528_19963.cdr
-rw-r--r--   1 smf_oper esg      1049420 Jul  6 12:53 am1slc01_ACS_201607061246136_19909.cdr
-rw-r--r--   1 smf_oper esg         6911 Jul  6 12:53 am1slc01_ACS_201607061242428_19949.cdr
-rw-r--r--   1 smf_oper esg      1049219 Jul  6 12:53 am1slc01_ACS_201607061247364_19957.cdr

My aim is to try to grep from the files which were modified one minute ago, i.e. in the above example grep from the files which were modified at 12:52, separately since they are different file (example am1slc01* and am1slc02*)

I have tried to build a script below in which it managed to take the last files modified (not one minute ago yet) however the grep won't work, it seems the grep works only on the first file

Code:
#! /usr/bin/bash
 cd /IN/service_packages/SMS/cdr/received
 mydate=`date +"%H:%M"`
 myfiles=`ls -ltr am1slc01_* | grep $mydate | awk '{print$9}'`
 echo $myfiles
 echo Data:
r=`grep -s CALL_TYPE=VOICE-MO $myfiles | wc -l`
t=`echo "scale=10; ($r/60)*1000" | bc`
echo CAPmS-slc01 $t

When echoing the files they are not listed as when making a simple 'ls' command but near each other so I'm suspecting that's the reason why it won't work.

Any ideas?
# 2  
Old 07-06-2016
You could try a simple while loop
Code:
ls -ltr am1slc01_* | grep $mydate | awk '{print$9}' |
while read fname
do
  r=`grep -s CALL_TYPE=VOICE-MO $fname | wc -l`
  t=`echo "scale=10; ($r/60)*1000" | bc`
  echo CAPmS-slc01 $t
done

FWIW:
Rather than messing around with all of those subprocesses you could do what you want with a single awk call.
This User Gave Thanks to jim mcnamara For This Post:
nms
# 3  
Old 07-06-2016
I support it also needs to be clarified as to what you mean by "one minute ago"

Is it:-
  • Within the last minute (ie. less than 60 seconds old)
  • Current minute (probably very few if the time is 12:53:01 but more if it's 12:52:59)
  • Exactly one minute (up to seconds/milliseconds)
  • At least one minute but less than two minutes
  • Files timestamped as the previous minute to now (ie. if I'm at 12:53, I want all 12:52 files, even if they are 90 seconds old
  • Something else?
There are probably several ways to get these, but we need to know exactly your definition of "one minute ago" first so we don't go off on a wild chase.


Kind regards,
Robin
# 4  
Old 07-07-2016
Hi

The while loop seems to do the job, however I am now left with two or more outputs depending upon files found. For example when tested there were two files so grep was performed for both and output two values. Can these be summed up in the code?

Code:
 CAPmS-slc01 266.6666666000
CAPmS-slc01 16.6666666000

Also this loop still takes the last files but not one minute ago. To clarify i am trying to achieve what Robin listed below:
  • Files timestamped as the previous minute to now (ie. if I'm at 12:53, I want all 12:52 files, even if they are 90 seconds old

---------- Post updated 07-07-16 at 12:01 PM ---------- Previous update was 07-06-16 at 06:34 PM ----------

Hi,

just found a dirty solution for this. copy the files based on date:

Code:
mydate=`date +"%H:%M"`
 for i in `ls -lrt am1slc01_* | grep $mydate | awk '{print$9}'`; do cp $i /home/nms/scripts/CDRs/slc01; done
for i in `ls -lrt am1slc02_* | grep $mydate | awk '{print$9}'`; do cp $i /home/nms/scripts/CDRs/slc02; done

then do the required grep and delete files

Now the next step is to copy the files from one minute ago
Last edited by rbatte1; 07-07-2016 at 04:55 PM.. Reason: Added CODE tags.
# 5  
Old 07-07-2016
You haven't said what you need that for, but i suppose you want to process "incoming" files somehow and want to prevent processing them twice.

In this case, wouldn't it be easier to move the files once they are processed? You wouldn't need to diddle with time stamps and complicated calculations and could adapt something like the following (sketch)

Two directories: /some/where/incoming and /some/where/processed, files go into "incoming"

Code:
while : ; do
     ls /some/where/incoming | while read FILE ; do
          process_file $FILE
          mv $FILE /some/where/processed
     done
     sleep 5
done

I hope this helps.

bakunin
# 6  
Old 07-07-2016
Depending on what the process_file requirements are, you might gobble up the input to your while loop, especially if you have a while read loop. I'd suggest something more like:-
Code:
while :
do
   for FILE in /some/where/incoming/*
   do
      process_file "$FILE"
      mv "$FILE" /some/where/processed
   done
   sleep 5
done

If your files might have spaces in them, change the for loop to be:-
Code:
   for FILE in $(ls /some/where/incoming/*)



Does that help at all?


Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to find a file that's modified more than 2 days ago but less than 5 days ago?

How to find a file that's modified more than 2 days ago but was modified less than 5 days ago by use of any Linux utility ? (4 Replies)
Discussion started by: abdulbadii
4 Replies

2. Shell Programming and Scripting

Want to add those that have the same minute in a file

Hi All, Need your help on how i can sum up the values. I have a file that contains the count and the time. I wanted to add up all the first column having the same datestamp. Please see below. INPUT 1721 2015-12-26 00:01 1440 2015-12-26 00:02 1477 2015-12-26 00:02 411 ... (4 Replies)
Discussion started by: ernesto
4 Replies

3. UNIX for Advanced & Expert Users

Find files modified in previous minute only

Hi, How can I get files which are modified only in last minute ? it should not display 2 minutes back filels -la -rw-rw-r-- 1 stuser st 51 Dec 3 09:22 a.csv -rw-rw-r-- 1 stiser st 50 Dec 3 09:25 b.csv -rw-rw-r-- 1 stuser st 53 Dec 3 09:33 c.csv When I run command at 9:34am then I... (7 Replies)
Discussion started by: sbjv
7 Replies

4. Shell Programming and Scripting

Take minute per minute from a log awk

Hi, I've been trying to develop a script that performs the parsing of a log every 1 minute and then generating some statistics. I'm fairly new to programming and this is why I come to ask if I can lend a hand. this is my log: xxxx 16/04/2012 17:00:52 - xxxx714 - E234 - Time= 119 ms.... (8 Replies)
Discussion started by: jockx
8 Replies

5. Shell Programming and Scripting

Process file every minute

Hey guy's, How can I schedule this script using AWK, what I require is it runs every minute as new data will be added to the file? And I need to capture these data and save it to the output file. I tried using sleep 1 to allow the file to be processed but is not working. awk ' ... (35 Replies)
Discussion started by: James_Owen
35 Replies

6. Homework & Coursework Questions

list of files modified 10 hours ago using grep

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Here is the question: Make a list of files in your home directory that were changed less that 10 hours ago,... (3 Replies)
Discussion started by: fight4love
3 Replies

7. Homework & Coursework Questions

Grep for modified time

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: is it possible to come up with a list of files that are modified before a certain number of hours only using the... (3 Replies)
Discussion started by: momo.reina
3 Replies

8. Shell Programming and Scripting

Grep for modified time

is it possible to come up with a list of files that are modified before a certain number of hours only using the grep command? ex. list files that were modified less than 10 hours ago i've only managed to list files that were created on the same day, i can't seem to figure out how to work... (3 Replies)
Discussion started by: momo.reina
3 Replies

9. Shell Programming and Scripting

Finding files which are modified few mins ago

Hi All, I have a requirement to find out the files which are modified in the last 10 minutes. I tried the find command with -amin and -mmin options, but its not working on my AIX server. Can anyone of you could help me. Thanks in advance for your help. Raju (3 Replies)
Discussion started by: rajus19
3 Replies

10. UNIX for Dummies Questions & Answers

how to retrieve original contents of a modified file (modified using vi)

Made changes to a file using vi editor and saved those changes now realised that the changes are not required How can I get the previous version of the file.i.e the one which was there on which I had made changes (3 Replies)
Discussion started by: novice100
3 Replies
Login or Register to Ask a Question