Sponsored Content
Top Forums Shell Programming and Scripting Calculating expiry date using date,sed,grep Post 302567488 by r4v3n on Monday 24th of October 2011 10:19:13 AM
Old 10-24-2011
Calculating expiry date using date,sed,grep

Hi,

I would greatly appreciate it if someone can help me with my problem.

I have a crawler which collects spam URLs everyday & this data needs to be published in a blacklist.

Here's the catch:

The "Time To Live" (TTL) for each URL is 3 months (or whatever for that matter). If i see the same URL again within the expiry of its TTL, I need to update that URLs TTL, so it stays in the blacklist for another 3 months (TTL).

The URLs which were never seen after the TTL need to be removed from the list after the TTL expires, so I don't have old data & can manage the size of my blacklist.

Here's an example current URL list which my crawler would have got today:
[URL followed by TTL of 3 days or whatever for that matter]
Code:
b.com 23
e.com 23
f.com 23

Here's an example of the current master URL file used for comparison:
Code:
a.com 19
b.com 20
c.com 21
d.com 21

Here's an example of the updated master URL file after comparison:
Code:
b.com 23
c.com 21
d.com 21
e.com 23
f.com 23

Here's what the final blacklist should look like:
Code:
b.com
c.com
d.com
e.com
f.com

How do can do this using using sed/grep/date (if it is indeed possible)? Unfortunately, I can't install any SQL db on this machine, which I realize would make things easy.

Again, any help would be much appreciated.

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

domain expiry date

Hi all, I want to write a shell script that read line by line of a file that have all domain names written, after reading complete file it will give us following output. Domain Name Expiry Days-Left abcd.com 20-Feb-200 6 10 How can i acheive that script... (6 Replies)
Discussion started by: learnbash
6 Replies

2. UNIX for Advanced & Expert Users

Digital Certificate Expiry Date

HI, I want to know the certificate expiry dates and who is owner of the certificate installed in UNIX. example certiface name 1)SSL 2)Veri Sign Trust Network (1 Reply)
Discussion started by: RG18173
1 Replies

3. Shell Programming and Scripting

Extract Expiry date and server_name from the following file

Hi Guys, I am trying to write one ksh script where I have to go to lots of unix boxes and read one text file which will have data like this though this is not complete data. And I just have to extract expire date ( which is 2nd date value 08/31/2009 in the first one ) and the server name ( astro... (0 Replies)
Discussion started by: pareshan
0 Replies

4. UNIX for Dummies Questions & Answers

Find user password expiry date

Does anyone know a simple way to find out the expiry date of my UNIX user password? I do not have root access to the box. Surely there is a way to so this... (4 Replies)
Discussion started by: oztoma
4 Replies

5. SuSE

Disable the password expiry date

Hi, We are going to create the new user, using that user we are automate the work, but every 90 days password get expired how can i create or avoid the expiry the password. (1 Reply)
Discussion started by: kingganesh04
1 Replies

6. UNIX for Advanced & Expert Users

expiry date for files?

Hi, Is there any way i can give an expiry date for files that i create? For example, i would like to get a file deleted automatically after 60 days from the creation date. Is there any possibility for this in Unix? Thanks, RRVARMA (3 Replies)
Discussion started by: RRVARMA
3 Replies

7. Shell Programming and Scripting

Expiry date of shell script

Hello folks, I have question about my expiry of shell script, today date is 2009-11-11, i will have to deploy a script today, how i should setup that my script will not work after 2010-05-11, Between if someone will change the date of server then script will again not work, please give... (3 Replies)
Discussion started by: learnbash
3 Replies

8. Shell Programming and Scripting

Password expiry date check for hp servers

Hi All, Can anybody help me, to find the command to check for the password expiry date for the hp servers. Thanks, Deepak (3 Replies)
Discussion started by: dswain
3 Replies

9. Shell Programming and Scripting

Calculating using date

I need to help to calculating using date in a script. One application is licensed by date, some month at a time. I can read the date from system and get an output like this: echo $status 6A34 System4 01.01.11-31.01.11 My goal is to use license date 31.01.11 and subtract todays date... (7 Replies)
Discussion started by: Jotne
7 Replies

10. Solaris

expiry date of an account

can an user see the expiry date of its own account,also can the user know whether it is having peer access or not. Thanks, Megh (2 Replies)
Discussion started by: megh
2 Replies
DATETIME.SETDATE(3)							 1						       DATETIME.SETDATE(3)

DateTime::setDate - Sets the date

       Object oriented style

SYNOPSIS
public DateTime DateTime::setDate (int $year, int $month, int $day) DESCRIPTION
Procedural style DateTime date_date_set (DateTime $object, int $year, int $month, int $day) Resets the current date of the DateTime object to a different date. PARAMETERS
o $object -Procedural style only: A DateTime object returned by date_create(3). The function modifies this object. o $year - Year of the date. o $month - Month of the date. o $day - Day of the date. RETURN VALUES
Returns the DateTime object for method chaining or FALSE on failure. CHANGELOG
+--------+---------------------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------------------+ | 5.3.0 | | | | | | | Changed the return value on success from NULL to | | | DateTime. | | | | +--------+---------------------------------------------------+ EXAMPLES
Example #1 DateTime.setDate(3) example Object oriented style <?php $date = new DateTime(); $date->setDate(2001, 2, 3); echo $date->format('Y-m-d'); ?> Procedural style <?php $date = date_create(); date_date_set($date, 2001, 2, 3); echo date_format($date, 'Y-m-d'); ?> The above examples will output: 2001-02-03 Example #2 Values exceeding ranges are added to their parent values <?php $date = new DateTime(); $date->setDate(2001, 2, 28); echo $date->format('Y-m-d') . " "; $date->setDate(2001, 2, 29); echo $date->format('Y-m-d') . " "; $date->setDate(2001, 14, 3); echo $date->format('Y-m-d') . " "; ?> The above example will output: 2001-02-28 2001-03-01 2002-02-03 SEE ALSO
DateTime.setISODate(3), DateTime.setTime(3). PHP Documentation Group DATETIME.SETDATE(3)
All times are GMT -4. The time now is 04:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy