Sponsored Content
Top Forums Shell Programming and Scripting Calculating expiry date using date,sed,grep Post 302567488 by r4v3n on Monday 24th of October 2011 10:19:13 AM
Old 10-24-2011
Calculating expiry date using date,sed,grep

Hi,

I would greatly appreciate it if someone can help me with my problem.

I have a crawler which collects spam URLs everyday & this data needs to be published in a blacklist.

Here's the catch:

The "Time To Live" (TTL) for each URL is 3 months (or whatever for that matter). If i see the same URL again within the expiry of its TTL, I need to update that URLs TTL, so it stays in the blacklist for another 3 months (TTL).

The URLs which were never seen after the TTL need to be removed from the list after the TTL expires, so I don't have old data & can manage the size of my blacklist.

Here's an example current URL list which my crawler would have got today:
[URL followed by TTL of 3 days or whatever for that matter]
Code:
b.com 23
e.com 23
f.com 23

Here's an example of the current master URL file used for comparison:
Code:
a.com 19
b.com 20
c.com 21
d.com 21

Here's an example of the updated master URL file after comparison:
Code:
b.com 23
c.com 21
d.com 21
e.com 23
f.com 23

Here's what the final blacklist should look like:
Code:
b.com
c.com
d.com
e.com
f.com

How do can do this using using sed/grep/date (if it is indeed possible)? Unfortunately, I can't install any SQL db on this machine, which I realize would make things easy.

Again, any help would be much appreciated.

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

domain expiry date

Hi all, I want to write a shell script that read line by line of a file that have all domain names written, after reading complete file it will give us following output. Domain Name Expiry Days-Left abcd.com 20-Feb-200 6 10 How can i acheive that script... (6 Replies)
Discussion started by: learnbash
6 Replies

2. UNIX for Advanced & Expert Users

Digital Certificate Expiry Date

HI, I want to know the certificate expiry dates and who is owner of the certificate installed in UNIX. example certiface name 1)SSL 2)Veri Sign Trust Network (1 Reply)
Discussion started by: RG18173
1 Replies

3. Shell Programming and Scripting

Extract Expiry date and server_name from the following file

Hi Guys, I am trying to write one ksh script where I have to go to lots of unix boxes and read one text file which will have data like this though this is not complete data. And I just have to extract expire date ( which is 2nd date value 08/31/2009 in the first one ) and the server name ( astro... (0 Replies)
Discussion started by: pareshan
0 Replies

4. UNIX for Dummies Questions & Answers

Find user password expiry date

Does anyone know a simple way to find out the expiry date of my UNIX user password? I do not have root access to the box. Surely there is a way to so this... (4 Replies)
Discussion started by: oztoma
4 Replies

5. SuSE

Disable the password expiry date

Hi, We are going to create the new user, using that user we are automate the work, but every 90 days password get expired how can i create or avoid the expiry the password. (1 Reply)
Discussion started by: kingganesh04
1 Replies

6. UNIX for Advanced & Expert Users

expiry date for files?

Hi, Is there any way i can give an expiry date for files that i create? For example, i would like to get a file deleted automatically after 60 days from the creation date. Is there any possibility for this in Unix? Thanks, RRVARMA (3 Replies)
Discussion started by: RRVARMA
3 Replies

7. Shell Programming and Scripting

Expiry date of shell script

Hello folks, I have question about my expiry of shell script, today date is 2009-11-11, i will have to deploy a script today, how i should setup that my script will not work after 2010-05-11, Between if someone will change the date of server then script will again not work, please give... (3 Replies)
Discussion started by: learnbash
3 Replies

8. Shell Programming and Scripting

Password expiry date check for hp servers

Hi All, Can anybody help me, to find the command to check for the password expiry date for the hp servers. Thanks, Deepak (3 Replies)
Discussion started by: dswain
3 Replies

9. Shell Programming and Scripting

Calculating using date

I need to help to calculating using date in a script. One application is licensed by date, some month at a time. I can read the date from system and get an output like this: echo $status 6A34 System4 01.01.11-31.01.11 My goal is to use license date 31.01.11 and subtract todays date... (7 Replies)
Discussion started by: Jotne
7 Replies

10. Solaris

expiry date of an account

can an user see the expiry date of its own account,also can the user know whether it is having peer access or not. Thanks, Megh (2 Replies)
Discussion started by: megh
2 Replies
DNSSEC-COVERAGE(8)						       BIND9							DNSSEC-COVERAGE(8)

NAME
dnssec-coverage - checks future DNSKEY coverage for a zone SYNOPSIS
dnssec-coverage [-K directory] [-f file] [-d DNSKEY TTL] [-m max TTL] [-r interval] [-c compilezone path] [zone] DESCRIPTION
dnssec-coverage verifies that the DNSSEC keys for a given zone or a set of zones have timing metadata set properly to ensure no future lapses in DNSSEC coverage. If zone is specified, then keys found in the key repository matching that zone are scanned, and an ordered list is generated of the events scheduled for that key (i.e., publication, activation, inactivation, deletion). The list of events is walked in order of occurrence. Warnings are generated if any event is scheduled which could cause the zone to enter a state in which validation failures might occur: for example, if the number of published or active keys for a given algorithm drops to zero, or if a key is deleted from the zone too soon after a new key is rolled, and cached data signed by the prior key has not had time to expire from resolver caches. If zone is not specified, then all keys in the key repository will be scanned, and all zones for which there are keys will be analyzed. (Note: This method of reporting is only accurate if all the zones that have keys in a given repository share the same TTL parameters.) OPTIONS
-f file If a file is specified, then the zone is read from that file; the largest TTL and the DNSKEY TTL are determined directly from the zone data, and the -m and -d options do not need to be specified on the command line. -K directory Sets the directory in which keys can be found. Defaults to the current working directory. -m maximum TTL Sets the value to be used as the maximum TTL for the zone or zones being analyzed when determining whether there is a possibility of validation failure. When a zone-signing key is deactivated, there must be enough time for the record in the zone with the longest TTL to have expired from resolver caches before that key can be purged from the DNSKEY RRset. If that condition does not apply, a warning will be generated. The length of the TTL can be set in seconds, or in larger units of time by adding a suffix: 'mi' for minutes, 'h' for hours, 'd' for days, 'w' for weeks, 'mo' for months, 'y' for years. This option is mandatory unless the -f has been used to specify a zone file. (If -f has been specified, this option may still be used; it will overrde the value found in the file.) -d DNSKEY TTL Sets the value to be used as the DNSKEY TTL for the zone or zones being analyzed when determining whether there is a possibility of validation failure. When a key is rolled (that is, replaced with a new key), there must be enough time for the old DNSKEY RRset to have expired from resolver caches before the new key is activated and begins generating signatures. If that condition does not apply, a warning will be generated. The length of the TTL can be set in seconds, or in larger units of time by adding a suffix: 'mi' for minutes, 'h' for hours, 'd' for days, 'w' for weeks, 'mo' for months, 'y' for years. This option is mandatory unless the -f has been used to specify a zone file, or a default key TTL was set with the -L to dnssec-keygen. (If either of those is true, this option may still be used; it will overrde the value found in the zone or key file.) -r resign interval Sets the value to be used as the resign interval for the zone or zones being analyzed when determining whether there is a possibility of validation failure. This value defaults to 22.5 days, which is also the default in named. However, if it has been changed by the sig-validity-interval option in named.conf, then it should also be changed here. The length of the interval can be set in seconds, or in larger units of time by adding a suffix: 'mi' for minutes, 'h' for hours, 'd' for days, 'w' for weeks, 'mo' for months, 'y' for years. -c compilezone path Specifies a path to a named-compilezone binary. Used for testing. SEE ALSO
dnssec-checkds(8), dnssec-dsfromkey(8), dnssec-keygen(8), dnssec-signzone(8) AUTHOR
Internet Systems Consortium COPYRIGHT
Copyright (C) 2013 Internet Systems Consortium, Inc. ("ISC") BIND9 April 16, 2012 DNSSEC-COVERAGE(8)
All times are GMT -4. The time now is 12:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy