Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Calculating expiry date using date,sed,grep
Top Forums Shell Programming and Scripting Calculating expiry date using date,sed,grep Post 302567488 by r4v3n on Monday 24th of October 2011 10:19:13 AM
Old 10-24-2011
Calculating expiry date using date,sed,grep
Hi,

I would greatly appreciate it if someone can help me with my problem.

I have a crawler which collects spam URLs everyday & this data needs to be published in a blacklist.

Here's the catch:

The "Time To Live" (TTL) for each URL is 3 months (or whatever for that matter). If i see the same URL again within the expiry of its TTL, I need to update that URLs TTL, so it stays in the blacklist for another 3 months (TTL).

The URLs which were never seen after the TTL need to be removed from the list after the TTL expires, so I don't have old data & can manage the size of my blacklist.

Here's an example current URL list which my crawler would have got today:
[URL followed by TTL of 3 days or whatever for that matter]
Code:
b.com 23
e.com 23
f.com 23

Here's an example of the current master URL file used for comparison:
Code:
a.com 19
b.com 20
c.com 21
d.com 21

Here's an example of the updated master URL file after comparison:
Code:
b.com 23
c.com 21
d.com 21
e.com 23
f.com 23

Here's what the final blacklist should look like:
Code:
b.com
c.com
d.com
e.com
f.com

How do can do this using using sed/grep/date (if it is indeed possible)? Unfortunately, I can't install any SQL db on this machine, which I realize would make things easy.

Again, any help would be much appreciated.

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

domain expiry date

Hi all, I want to write a shell script that read line by line of a file that have all domain names written, after reading complete file it will give us following output. Domain Name Expiry Days-Left abcd.com 20-Feb-200 6 10 How can i acheive that script... (6 Replies)
Discussion started by: learnbash
6 Replies

2. UNIX for Advanced & Expert Users

Digital Certificate Expiry Date

HI, I want to know the certificate expiry dates and who is owner of the certificate installed in UNIX. example certiface name 1)SSL 2)Veri Sign Trust Network (1 Reply)
Discussion started by: RG18173
1 Replies

3. Shell Programming and Scripting

Extract Expiry date and server_name from the following file

Hi Guys, I am trying to write one ksh script where I have to go to lots of unix boxes and read one text file which will have data like this though this is not complete data. And I just have to extract expire date ( which is 2nd date value 08/31/2009 in the first one ) and the server name ( astro... (0 Replies)
Discussion started by: pareshan
0 Replies

4. UNIX for Dummies Questions & Answers

Find user password expiry date

Does anyone know a simple way to find out the expiry date of my UNIX user password? I do not have root access to the box. Surely there is a way to so this... (4 Replies)
Discussion started by: oztoma
4 Replies

5. SuSE

Disable the password expiry date

Hi, We are going to create the new user, using that user we are automate the work, but every 90 days password get expired how can i create or avoid the expiry the password. (1 Reply)
Discussion started by: kingganesh04
1 Replies

6. UNIX for Advanced & Expert Users

expiry date for files?

Hi, Is there any way i can give an expiry date for files that i create? For example, i would like to get a file deleted automatically after 60 days from the creation date. Is there any possibility for this in Unix? Thanks, RRVARMA (3 Replies)
Discussion started by: RRVARMA
3 Replies

7. Shell Programming and Scripting

Expiry date of shell script

Hello folks, I have question about my expiry of shell script, today date is 2009-11-11, i will have to deploy a script today, how i should setup that my script will not work after 2010-05-11, Between if someone will change the date of server then script will again not work, please give... (3 Replies)
Discussion started by: learnbash
3 Replies

8. Shell Programming and Scripting

Password expiry date check for hp servers

Hi All, Can anybody help me, to find the command to check for the password expiry date for the hp servers. Thanks, Deepak (3 Replies)
Discussion started by: dswain
3 Replies

9. Shell Programming and Scripting

Calculating using date

I need to help to calculating using date in a script. One application is licensed by date, some month at a time. I can read the date from system and get an output like this: echo $status 6A34 System4 01.01.11-31.01.11 My goal is to use license date 31.01.11 and subtract todays date... (7 Replies)
Discussion started by: Jotne
7 Replies

10. Solaris

expiry date of an account

can an user see the expiry date of its own account,also can the user know whether it is having peer access or not. Thanks, Megh (2 Replies)
Discussion started by: megh
2 Replies

LEARN ABOUT SUSE

gappletviewer

GAPPLETVIEWER(1)							GNU							  GAPPLETVIEWER(1)

NAME

       gappletviewer - Load and runs an applet

SYNOPSIS

       appletviewer [OPTION]... URL...

       appletviewer [OPTION]... -code CODE

       appletviewer [OPTION]... -plugin INPUT,OUTPUT

DESCRIPTION

       The appletviewer tool loads and runs an applet.

       Use the first form to test applets specified by tag.  The URL should resolve to an HTML document from which the appletviewer will extract
       applet tags.  The APPLET, EMBED and OBJECT tags are supported.  If a given document contains multiple applet tags, all the applets will be
       loaded, with each applet appearing in its own window.  Likewise, when multiple URLs are specified, each applet tag instance is given its
       own window.  If a given document contains no recognized tags the appletviewer does nothing.

	       appletviewer http://www.gnu.org/software/classpath/

       Use the second form to test an applet in development.  This form allows applet tag attributes to be supplied on the command line.  Only one
       applet may be specified using the -code option.	The -code option overrides the URL form -- any URLs specified will be ignored.

	       appletviewer -code Test.class -param datafile,data.txt

       gcjwebplugin uses the third form to communicate with the appletviewer through named pipes.

OPTIONS

       URL OPTIONS

       -debug
	   This option is not yet implemented but is provided for compatibility.

       -encoding CHARSET
	   Use this option to specify an alternate character encoding for the specified HTML page.

       APPLET TAG OPTIONS

       -code CODE
	   Use the -code option to specify the value of the applet tag CODE attribute.

       -codebase CODEBASE
	   Use the -codebase option to specify the value of the applet tag CODEBASE attribute.

       -archive ARCHIVE
	   Use the -archive option to specify the value of the applet tag ARCHIVE attribute.

       -width WIDTH
	   Use the -width option to specify the value of the applet tag WIDTH attribute.

       -height HEIGHT
	   Use the -height option to specify the value of the applet tag HEIGHT attribute.

       -param NAME,VALUE
	   Use the -param option to specify values for the NAME and VALUE attributes of an applet PARAM tag.

       PLUGIN OPTION

       -plugin INPUT,OUTPUT
	   gcjwebplugin uses the -plugin option to specify the named pipe the appletviewer should use for receiving commands (INPUT) and the one
	   it should use for sending commands to gcjwebplugin (OUTPUT).

       DEBUGGING OPTION

       -verbose
	   Use the -verbose option to have the appletviewer print debugging messages.

       STANDARD OPTIONS

       -help
	   Use the -help option to have the appletviewer print a usage message, then exit.

       -version
	   Use the -version option to have the appletviewer print its version, then exit.

       -JOPTION
	   Use the -J option to pass OPTION to the virtual machine that will run the appletviewer.  Unlike other options, there must not be a
	   space between the -J and OPTION.

BUGS

SEE ALSO

AUTHOR

0.98								    2010-07-05							  GAPPLETVIEWER(1)
All times are GMT -4. The time now is 06:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy