Sponsored Content
Top Forums Shell Programming and Scripting help on most efficient search Post 302182144 by era on Saturday 5th of April 2008 02:42:36 PM
Old 04-05-2008
If these are web uploads, perhaps it would be simpler to process the web server's log file?
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Efficient Dispatching

Does anyone know what's new with Efficient dispatching in the Solaris 2.8 release (vs Solaris 2.6) release? Specifically, does anyone know of a good website to get detailed information on thread dispatching using efficient dispatching in solaris 2.8? Thank you. (1 Reply)
Discussion started by: uchachra
1 Replies

2. Shell Programming and Scripting

Is there a more efficient way?

I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method. If you know of one, please suggest a shorter, more efficient method. ############### FUNCTIONS ... (6 Replies)
Discussion started by: SelectSplat
6 Replies

3. Shell Programming and Scripting

efficient search

Hi, i have 2 files each with 200K lines. Each line contains a number. Now, i need to get the list of numbers existing in one fine and NOT in other file. I'm doing this by reading each number from 1 file and grepping on other file. But this taking LOT of time. Is there any efficient way of doing... (14 Replies)
Discussion started by: prvnrk
14 Replies

4. UNIX for Advanced & Expert Users

efficient repace

some of the data i receive has been typed in manually due to which there are often places where i find 8 instead of ( and the incorrect use of case what according to you is the best way to correct such data. The data has around 20,000 records. The value i want to change is in the 4th field.... (2 Replies)
Discussion started by: VGR
2 Replies

5. UNIX for Dummies Questions & Answers

Is this regex efficient?

I want to match the red portion: 9784323456787-Unknown Phrase with punctuation "Some other PhrASE." Is this the best regex to match this? '978\{10\}-*' (4 Replies)
Discussion started by: glev2005
4 Replies

6. Shell Programming and Scripting

Better and efficient way to reverse search a file for first matched line number.

How to reverse search for a matched string in a file. Get line# of the first matched line. I am getting '2' into 'lineNum' variable. But it feels like I am using too many commands. Is there a better more efficiant way to do this on Unix? abc.log aaaaaaaaaaaaa bbbbbbbbbbbbb... (11 Replies)
Discussion started by: kchinnam
11 Replies

7. Shell Programming and Scripting

Help with Efficient Looping

Hello guys My requirement is to read a file with parent-child relationship we need to iterate through each row to find its latest child. for eg. parent child ABC PQR PQR DEF DEF XYZ Expected Output ABC XYZ PQR XYZ DEF XYZ Script Logic : read parent from file seach child... (4 Replies)
Discussion started by: joshiamit
4 Replies

8. Shell Programming and Scripting

Need an efficient way to search for a tag in an xml file having millions of rows

Hi, I have an XML file with around 1 billion rows in it and i am trying to find the number of times a particular tag occurs in it. The solution i am using works but takes a lot of time (~1 hr) .Please help me with an efficient way to do this. Lets say the input file is <Root> ... (13 Replies)
Discussion started by: Sheel
13 Replies

9. Shell Programming and Scripting

Search and replace multiple patterns in a particular column only - efficient script

Hi Bigshots, I have a pattern file with two columns. I have another data file. If column 1 in the pattern file appears as the 4th column in the data file, I need to replace it (4th column of data file) with column 2 of the pattern file. If the pattern is found in any other column, it should not... (6 Replies)
Discussion started by: ss112233
6 Replies

10. Shell Programming and Scripting

Efficient way to search array in text file by awk

I have one array SPLNO with approx 10k numbers.Now i want to search the subscriber number from MDN.TXT file (containing approx 1.5 lac record)from the array.if subscriber number found in array it will perform below operation.my issue is that it's taking more time because for one number it's search... (6 Replies)
Discussion started by: siramitsharma
6 Replies
PENLOGD(1)						      General Commands Manual							PENLOGD(1)

NAME
penlogd - consolidate web server logs SYNOPSIS
penlogd [-fd] [-j dir] [-l logfile] [-n N] [-p pidfile] [-u user] port EXAMPLE
penlogd -l /var/log/access_log -p /var/run/penlogd.pid 10000 DESCRIPTION
Penlogd receives log entries from Pen and from each of the web servers. It consolidates the entries by replacing the source addresses in each entry with the "real" client address and writes the result to stdout or to the file given on the command line. This completely removes the need for postprocessing with mergelogs, since the logs are already merged. Pen must be instructed to send its log to penlogd. See HOWTO and pen man page for details. Sending penlogd a HUP signal will make it close and reopen the logfile, unless it is logging to stdout. Rotate the log like this: mv access_log access_log.1 kill -HUP `cat <pidfile>` where <pidfile> is the file containing pen's process id. Sending penlogd a TERM signal will make it close the log file and exit cleanly. OPTIONS
-d Turn on debugging. The output goes to stderr if we are running in the foreground (see -f) and to syslog (facility user, priority debug) otherwise. -f Stay in foreground. -j dir Run in a chroot environment. -l logfile Write output into logfile. -n N Number of pen log entries to cache (default 1000). -p pidfile Write process id into pidfile. -u user Run as a different user. port The UDP port where penlogd receives log entries. SEE ALSO
pen(1), penlog(1), webresolve(1) AUTHOR
Copyright (C) 2002-2003 Ulric Eriksson, <ulric@siag.nu>. LOCAL PENLOGD(1)
All times are GMT -4. The time now is 06:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy