Creating searches?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Creating searches?
# 1  
Old 05-07-2008
Creating searches?

Hello. Could do with some help on where to get started really. If anyone could help me it would be greatly appreciated.

I have been working on this for a while now and I don't really know where to start but I am looking into creating a script that will process website hit files and output statistical information to the screen.

I have all of the hit files already and i have them populated like this:

137.44.2.8 Mon Feb 4 22:02:35 GMT 2008
149.192.2.81 Mon Feb 4 23:22:12 GMT 2008
132.53.17.171 Tue Feb 5 01:56:16 GMT 2008

What i want to do is create script(s) that will:

1. Request from the user a particular time period of interest for them to search all of this information

2. find

(i) the number of hits occurring during the given time period, and

(ii) the number of hits from unique IP addresses (counting two as one)

3. Present this information in table with the headings “page”, “hits” and “unique hits”


Also all the hit files are stored in a hits directory and I want the script to be executed from the parent directory of hits.

Again if anyone can point me in a good direction this would be greatly appreciated.

All i can think of so far is using "if" functions. but this is the first time i have used unix. (im using SUSE10.3 if that helps?) i just really wanted some pointers/solutions.

thanks for your time.
# 2  
Old 05-12-2008
I'd suggest working with perl for this as it's rather good at parsing files and creating nicely formatted reports.

Here's some pseudo-code to help get you going:
Code:
#!/usr/local/bin/do-what-i-want-not-what-i-write
print "Start date to search from: "
read from STDIN to $datestart
print "End date to search up until: "
read from STDIN to $dateend
print "File to search on: "
read from STDIN to $file

$hitcount=0
new array($iplist)

while (read $line from $file) {
  split $line into ($ip,$date)
  if ($date >= $datestart && $date <= $dateend) {
    $iplist[$ip]++
    $hitcount++
  }
}
$hitcount_unique=arraysize($iplist)
print_pretty_report($hitcount,$hitcount_unique)

# 3  
Old 05-13-2008
**************Thread Closed *****************************
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Customized text searches by using grep

I tried to ease text searches so made a customized grep: g () { if then i= for s in $2 do i="$i --include=*.$s" done else i='--include=*.txt --include=*.ini --include=*.*sh --include=*.c* --include=*.h --include=*.js --include=*.reg' fi grep -P -e \'$1\' -r "$i" } but I... (3 Replies)
Discussion started by: abdulbadii
3 Replies

2. UNIX for Dummies Questions & Answers

Output based on multiple searches

I have a file that looks like this: >Sample 539 GCCCAGCGCGCGILTGCCGCCGTCTCCGCCTGTCJOHNCCGCCATTGCCCCCGGTTAC I am using the following code to search specific patterns: awk '/^>/ { print $0 } NR==2 {if (/GIL/) { print "\t" "1" } else { print "\t" "0" }} NR==2 {if (/JOHN/) { print "\t""\t"... (7 Replies)
Discussion started by: Xterra
7 Replies

3. Shell Programming and Scripting

Need help in awk for multiple searches

I have a below file RCS File name : abc.txt something something .... symbolic names: implemented : 1.1 ssssssumthing Revision 1.2 date : 12/12/12 author : abc Revision 1.1 date : 11/11/11 author xyz So now , in this file i have to first look for the implemented... (1 Reply)
Discussion started by: ashishagg2005
1 Replies

4. Shell Programming and Scripting

Can someone please help me optimize my code (script searches subdirectories)?

Here is my code. What it does is it reads an input file (input.txt which contains roughly 2,000 search phrases) and searches a directory for files that contains the search phrase. The directory contains roughly 1900 files and 84 subdirectories. The output is a file (output.txt) that shows only the... (23 Replies)
Discussion started by: jl487
23 Replies

5. UNIX and Linux Applications

Alpine: LDAP searches hang

I just configured my ldap server in Alpine, but every search hangs indefinitely (or so it seems) and I have to end up killing Alpine and starting back up. The LDAP server runs over SSL on port 636, so I have specified port 636, but there doesn't seem to be an SSL option available so I turned on... (0 Replies)
Discussion started by: retrovertigo
0 Replies

6. Shell Programming and Scripting

Perl syntax for sed searches

I am aware that Perl has a lot of features that originally came from sed and awk. I have a pattern that I am using like this: sed -n '/|Y|/p' I want to do the same thing in Perl and be able to either save that value in some kind of variable or array or potentially write it out to a file. ... (11 Replies)
Discussion started by: masinick
11 Replies

7. News, Links, Events and Announcements

New Tool Searches and Replaces SCO Code

See this article: http://story.news.yahoo.com/news?tmpl=story&cid=74&ncid=738&e=9&u=/cmp/20030809/tc_cmp/13000487 (3 Replies)
Discussion started by: Neo
3 Replies

8. UNIX for Dummies Questions & Answers

grep: do multiple searches?

I want to search the file /etc/passwd for all lines containing 'csh' but exlude all those lines that have '/usr' in them and dump the results into the file result. IMPORTANT: I need to do this in one command line. The following does not work: grep -v \(\/usr\) \(csh\) /etc/passwd >... (4 Replies)
Discussion started by: sdemba
4 Replies
Login or Register to Ask a Question