Sponsored Content
Top Forums Shell Programming and Scripting what is the most effective way to process a large logfile? Post 302096187 by fedora on Tuesday 14th of November 2006 09:46:59 AM
Old 11-14-2006
Thank you! aigles

I am just wondering is this the most efficient way to do the job?

I will compare the time difference.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

most effective search ?

what's the most efficient and effective search for a file in a dir ? I see many guys use this # find - print or something as such ? and sometimes pipe it to something else ? Is there a better way of using "grep" in all of this ? thanks simon2000 (3 Replies)
Discussion started by: simon2000
3 Replies

2. Shell Programming and Scripting

get last section from large logfile

Hi all, I need to pull the (last section) from the header to the end of file. This logfile will have many sections appended, but I only need to capture the last one and email it to someone. Any ideas? Thanks Kathy Example Logfile: ============= = BOX : SAPCFI = JOB : SAPCFI =... (9 Replies)
Discussion started by: kburrows
9 Replies

3. Shell Programming and Scripting

Scripting the process to edit a large file

Hi, I need to make a script to edit a file. File is a large file in below format Version: 2008120101 ;$INCLUDE ./abc/xyz/Delhi ;$INCLUDE ./abc/xyz/London $INCLUDE ./abc/xyz/New York First line in the file is version number which is in year,month,date and serial number format. Each... (5 Replies)
Discussion started by: makkar4u
5 Replies

4. Red Hat

Fork wait in background process - large delay

hi all, We are trying to run a process in the background and in the process we call fork ;and wait for the child process to finish .We find that the died = wait(&status); happens after 10 seconds randomly and sometimes completes in time (within 1 sec) This behavior is seen only when the... (0 Replies)
Discussion started by: vishnu.priya
0 Replies

5. Red Hat

Fork wait in background process - large delay

hi all, We are trying to run a process in the background and in the process we call fork ;and wait for the child process to finish .We find that the died = wait(&status); happens after 10 seconds randomly and sometimes completes in time (within 1 sec) This behavior is seen only when the... (1 Reply)
Discussion started by: vishnu.priya
1 Replies

6. Shell Programming and Scripting

how to track a log file of one process and mail that logfile to a group?

Hi Friendz, I have 14 DB load scripts say 1,2,3....14. I want a script to call each script automatically, and after completion of every script, it needs to track the logfile and mail that log file to a group.and again it should run the next script in sequence 1,2,3...14 Please help me,need... (1 Reply)
Discussion started by: shirdi
1 Replies

7. UNIX for Dummies Questions & Answers

Real and Effective IDs

Can anyone explain me in details of Real and Effective IDs (6 Replies)
Discussion started by: kkalyan
6 Replies

8. Cybersecurity

Password rules not effective

I was looking for a good list of words to exclude people from using as passwords, i.e. those that could be guessed easily. I'm working through a whole bunch of suggestions from skullsecurity.org, but I managed to find this page that seems to suggest I have more options than I thought. :b: I... (1 Reply)
Discussion started by: rbatte1
1 Replies

9. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

10. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a... (4 Replies)
Discussion started by: camor
4 Replies
QSTAT(1)						User Contributed Perl Documentation						  QSTAT(1)

NAME
qstat - display job/partition information in a familiar pbs format SYNOPSIS
qstat [-f] [-a|-i|-r] [-n [-1]] [-G|-M] [-u user_list] [-? | --help] [--man] [job_id...] qstat -Q [-f] qstat -q DESCRIPTION
The qstat command displays information about jobs. OPTIONS
-a Displays all jobs in a single-line format. See the STANDARD OUTPUT section for format details. -i Displays information about idle jobs. This includes jobs which are queued or held. -f Displays the full information for each selected job in a multi-line format. See the STANDARD OUTPUT section for format details. -G Display size information in gigabytes. -M Show size information, disk or memory in mega-words. A word is considered to be 8 bytes. -n Displays nodes allocated to a job in addition to the basic information. -1 In combination with -n, the -1 option puts all of the nodes on the same line as the job id. -r Displays information about running jobs. This includes jobs which are running or suspended. -u user_list Display job information for all jobs owned by the specified user(s). The format of user_list is: user_name[,user_name...]. -? | --help brief help message --man full documentation STANDARD OUTPUT
Displaying Job Status If the -a, -i, -f, -r, -u, -n, -G, and -M options are not specified, the brief single-line display format is used. The following items are displayed on a single line, in the specified order, separated by white space: the job id the job name the job owner the cpu time used the job state C - Job is completed after having run E - Job is exiting after having run. H - Job is held. Q - job is queued, eligible to run or routed. R - job is running. T - job is being moved to new location. W - job is waiting for its execution time (-a option) to be reached. S - job is suspended. the queue that the job is in If the -f option is specified, the multi-line display format is used. The output for each job consists of the header line: Job Id: job identifier followed by one line per job attribute of the form: attribute_name = value If any of the options -a, -i, -r, -u, -n, -G or -M are specified, the normal single-line display format is used. The following items are displayed on a single line, in the specified order, separated by white space: the job id the job owner the queue the job is in the job name the session id (if the job is running) the number of nodes requested by the job the number of cpus or tasks requested by the job the amount of memory requested by the job either the cpu time, if specified, or wall time requested by the job, (in hh:mm) the job state The amount of cpu time or wall time used by the job (in hh:mm) EXIT STATUS
On success, qstat will exit with a value of zero. On failure, qstat will exit with a value greater than zero. perl v5.14.2 2012-04-10 QSTAT(1)
All times are GMT -4. The time now is 08:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy