Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to calculate date and show data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to calculate date and show data
# 15  
Old 11-21-2014
sorry. my mistake. it is actually working. just that, the file i'm giving it is almost 200MB. so it takes a while to process it.

however, it appears its not properly subtracting the entry_time from the current time. as i still see the same data in the results returned from the file.
# 16  
Old 11-21-2014
Quote:
Originally Posted by SkySmart
same thing here. it seems to just hang. looks like we're getting closer Smilie
My script cannot hang unless you didn't supply a filename. Please double check.
This User Gave Thanks to jlliagre For This Post:
SkySmart
# 17  
Old 11-21-2014
Quote:
Originally Posted by jlliagre
My script cannot hang unless you didn't supply a filename. Please double check.
yes, you are right. it's not hanging. that was my mistake. the file was just so big so it took a while to get a response.

however, it appears i'm still getting the same data.

the goal of this is to trim down the 200MB file by finding the chunks that have a date older than 60 days and eliminating them so they are no longer in the file.

I apologize if i wasn't clear before.
# 18  
Old 11-21-2014
Why don't you try it with a few "chunks" of which some are older and some newer than those 60 days?
This User Gave Thanks to RudiC For This Post:
SkySmart
# 19  
Old 11-21-2014
Quote:
Originally Posted by RudiC
Why don't you try it with a few "chunks" of which some are older and some newer than those 60 days?

These are the chunks in my test file:
Code:
servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1408251613
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1416593731
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1408251613
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1416593664
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entiiiry_time=1408251613
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}
root@mojomo-VirtualBox:~# 
root@mojomo-VirtualBox:~# 
root@mojomo-VirtualBox:~# cat data.u 
servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1408251613
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1416593731
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1408251613
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entry_time=1416593664
expires=0
expire_time=0
author=hpsm
comment_data=IM02015654
}

servicecomment {
host_name=myhost01
service_description=Load Average
entry_type=4
comment_id=70711
source=0
persistent=1
entiiiry_time=1408251613
expires=0
expire_time=0
author=hyat
comment_data=IM02015654
}

i updated the entry_time for a couple of the chunks to have a date of today.

so, the awk code should only be outputting those chunks that have a date newer than 60 days (which would be the two chunks whose date i updated to be recent).

also, the code should output the chunk i bolded because this chunk does not have both "comment_data" AND "entry_time".

any ideas?
# 20  
Old 11-21-2014
Quote:
Originally Posted by jlliagre
Although it is not a well known fact, all awk releases are able to retrieve the current epoch time by standard :
Code:
$ awk 'BEGIN {srand();time=srand();print time}'
1416532245

Thank you so much for this tip Smilie

Frankly I didn't know this fact and after your post I looked up the srand function in the awk man page...which says that it sets the seed value for rand and returns the previous seed value which would be the time of day if no argument was supplied in the first place which explains why we need to call it twice...
# 21  
Old 11-21-2014
Quote:
Originally Posted by SkySmart
so, the awk code should only be outputting those chunks that have a date newer than 60 days (which would be the two chunks whose date i updated to be recent).

also, the code should output the chunk i bolded because this chunk does not have both "comment_data" AND "entry_time".
That last rule was unclear, I only retained records where both comment_data and entry_time were missing. Fixed that.
Here is an updated script with also the 60 days delay fixed (negleting the fact not all days are 24 hours where DST is followed).

Code:
awk -v pattern1="entry_time" -v pattern2="comment_data" '
    BEGIN {
        srand(); now=srand();
        least=now-(60*24*3600);
        RS=""
        FS="="
        ORS="\n\n"
    }
    {
      p=t=0
      for(i=1;i<=NF;i++)  {
        if($i==pattern1) t=$(i+1)
        if($i==pattern2) p=1
      }
      if((t==0 || p==0) || (t>least && p==1)) print
    }
' $1

Last edited by jlliagre; 11-21-2014 at 08:00 PM..
This User Gave Thanks to jlliagre For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

How to calculate the quarter end date according to the current date in shell script?

Hi, My question is how to calculate the quarter end date according to the current date in shell script? (2 Replies)
Discussion started by: Divya_1234
2 Replies

2. Shell Programming and Scripting

Calculate Julian date of a given date

How to get Julian date (Three digit) of a given date (Not current date)? I do not have root privilege - so can not use date -d. Assume that we have three variables year, month and date. Thx (5 Replies)
Discussion started by: Soham
5 Replies

3. Shell Programming and Scripting

Calculate date

Hello, i need calculate the day before in this format aammdd, for example if i have 110701 i need a script to return the value 110630. I only know what with this comand "date %y%m%d" return the date system. Sorry for my english, it is very bad, i know... Thank you very much Grettings! (1 Reply)
Discussion started by: Xedrox
1 Replies

4. Shell Programming and Scripting

Using awk to rearrange data according to date

Hi, I have a large data frame as shown below, where data is separated into years. 10 May 2011 Created: 10 May 11 15:05 GMT Scale: SIO-2005 and others GC-MD, Cape Grim, Tasmania, Lat.: 40.68S, Lon.: 144.69E, Alt: 94m above sea level You can use the following format in Fortran to read data... (4 Replies)
Discussion started by: gd9629
4 Replies

5. Shell Programming and Scripting

How to show last month date?

Hi Guys, Please somebody give me a hand to show the Month & Year in assigning to a variable and with format "MMMYY" (i.e. Jul11). See below preferred output. I tried using this but it is giving me the current month... date = "`date +%b%y`" Aug11 DESIRED OUTPUT: Jul11 ... (3 Replies)
Discussion started by: pinpe
3 Replies

6. Shell Programming and Scripting

Calculate data and make it into new column using awk

Hi everyone, just some simple question... i've been using a awk script to calculate my data... i have 3 files: file a1.txt: 2 3 4 5 3 4 file a2.txt: 4 5 6 7 8 (1 Reply)
Discussion started by: yat
1 Replies

7. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

8. UNIX for Dummies Questions & Answers

ls how to not show date

Hello, I'm using ls -laR to print out a list of file and folders. I want to print only the permission, file size and file name. Also, excluding the '.' and '..'. result from ls -laR: total 6 drwxr-xr-x 8 user staff 512 Nov 28 16:17 . drwxr-x--- 16 user staff 1024... (3 Replies)
Discussion started by: minifish
3 Replies

9. Shell Programming and Scripting

calculate the date of next satureday of current date.

I want to calculate the date of next satureday of current date using shell script. Suppose, today is 27-feb-08 I want to get the date of next satureday, which means 01-mar-08, in the formate '' YYMMDD ". I do this in ksh.. Please tell me any type of command which help me out. Thanks in... (3 Replies)
Discussion started by: rinku
3 Replies

10. Shell Programming and Scripting

How to show yesterday date

HI all, i am a junior learner, can u teach me how to show yesterday time with unix command? thanks! Cloud (1 Reply)
Discussion started by: wind_n_cloud
1 Replies
Login or Register to Ask a Question