Sponsored Content
Top Forums Shell Programming and Scripting Extract information from Log file formatted Post 302503479 by dummie55 on Thursday 10th of March 2011 06:33:12 PM
Old 03-10-2011
Extract information from Log file formatted

Good evening! Trying to make a shell script to parse log file and show only required information.
log file has 44 fields and alot of lines, each columns separated by ":".
log file is like:
Code:
first_1:3:4:5:6:1:3:4:5:something:notinterested
second_2:3:4:3:4:2
first_1:3:4:6:6:7:8

I am interested only in 5 fields which are : $3,$6,$14,$37,$38
So what i need is to parse that log file and display on screen something like this:
Code:
echo "Group	Jobs	WallClock	CPU	TotalMem	Memory"
        first_1     2              4                6              7                  8
        second_2    1              3                4              8                  3

The problem is that log file has group names repeated and what i have to do is just to read every line , pick unique group name and sum number of times its repeats on other lines so i get number of Jobs. Any help would be appreciated.

---------- Post updated at 06:33 PM ---------- Previous update was at 04:55 PM ----------

Trying something like :


Code:
#! /bin/sh
echo "Group	Jobs	WallClock	CPU	TotalMem	Memory"
awk -F":" ' { printf("%s	%d	%d	%d	%d	%d\n",$3,$6,$14,$37,$38) } ' $1

PS: runing script like ./myscript.sh log

i know its nothing but any suggestions how to continue?

Last edited by Scott; 03-10-2011 at 06:50 PM.. Reason: Please use code tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc... (2 Replies)
Discussion started by: Marcor
2 Replies

2. Shell Programming and Scripting

extract and format information from a file

Hi, Following is sample portion of the file; <JDBCConnectionPool DriverName="oracle.jdbc.OracleDriver" MaxCapacity="10" Name="MyApp_DevPool" PasswordEncrypted="{3DES}7tXFH69Xg1c=" Properties="user=MYAPP_ADMIN" ShrinkingEnabled="false" ... (12 Replies)
Discussion started by: sujoy101
12 Replies

3. Shell Programming and Scripting

Create shell script to extract unique information from one file to a new file.

Hi to all, I got this content/pattern from file http.log.20110808.gz mail1 httpd: Account Notice: close igchung@abc.com 2011/8/7 7:37:36 0:00:03 0 0 1 mail1 httpd: Account Information: login sastria9@abc.com proxy sid=gFp4DLm5HnU mail1 httpd: Account Notice: close sastria9@abc.com... (16 Replies)
Discussion started by: Mr_47
16 Replies

4. Shell Programming and Scripting

extract information from a log file (last days)

I'm still new to bash script , I have a log file and I want to extract the items within the last 5 days . and also within the last 10 hours the log file is like this : it has 14000 items started from march 2002 to january 2003 awk '{print $4}' < *.log |uniq -c|sort -g|tail -10 but... (14 Replies)
Discussion started by: matarsak
14 Replies

5. Shell Programming and Scripting

Extract various information from a log file

Hye ShamRock If you can help me with this difficult task for me then it will save my day Logs : ================================================================================================================== ... (4 Replies)
Discussion started by: SilvesterJ
4 Replies

6. Shell Programming and Scripting

How to extract information from a file?

Hi, i have a file like this: <Iteration> <Iteration_iter-num>3</Iteration_iter-num> <Iteration_query-ID>lcl|3_0</Iteration_query-ID> <Iteration_query-def>G383C4U01EQA0A length=197</Iteration_query-def> <Iteration_query-len>197</Iteration_query-len> ... (9 Replies)
Discussion started by: the_simpsons
9 Replies

7. Shell Programming and Scripting

Extract information from txt file

Hello! I need help :) I have a file like this: AA BC FG RF TT GH DD FF HH (a few number of rows and three columns) and I want to put the letters of each column in a variable step by step in order to give them as input in another script. So I would like to obtain: for the 1° loop:... (11 Replies)
Discussion started by: edekP
11 Replies

8. Shell Programming and Scripting

Extract information from file

Gents, If is possible please help. I have a big file (example attached) which contends exactly same value in column, but from column 2 to 6 these values are diff. I will like to compile for all records all columns like the example attached in .csv format (output.rar ).. The last column in the... (11 Replies)
Discussion started by: jiam912
11 Replies

9. Shell Programming and Scripting

Extract information from file

In a particular directory, there can be 1000 files like below. filename is job901.ksh #!/bin/ksh cront -x << EOJ submit file=$PRODPATH/scripts/genReport.sh maxdelay=30 &node=xnode01 tname=job901 &pfile1=/prod/mldata/data/test1.dat ... (17 Replies)
Discussion started by: vedanta
17 Replies

10. Shell Programming and Scripting

sed / awk / grep to extract information from log

Hi all, I have a query that runs that outputs data in the following format - 01/09/12 11:43:40,ADMIN,4,77,Application Group Load: Name(TESTED) LoadId(5137-1-0-1XX-15343-15343) File(/dir/dir/File.T03.CI2.RYR.2012009.11433350806.ARD) InputSize(5344) OutputSize(1359) Rows(2) Time(1.9960)... (8 Replies)
Discussion started by: jeffs42885
8 Replies
LOGGER(1)						    BSD General Commands Manual 						 LOGGER(1)

NAME
logger -- make entries in the system log SYNOPSIS
logger [-is] [-f file] [-p pri] [-t tag] [message ...] DESCRIPTION
Logger provides a shell command interface to the syslog(3) system log module. Options: -i Log the process id of the logger process with each line. -s Log the message to standard error, as well as the system log. -f file Log the specified file. -p pri Enter the message with the specified priority. The priority may be specified numerically or as a ``facility.level'' pair. For example, ``-p local3.info'' logs the message(s) as informational level in the local3 facility. The default is ``user.notice.'' -t tag Mark every line in the log with the specified tag. message Write the message to log; if not specified, and the -f flag is not provided, standard input is logged. The logger utility exits 0 on success, and >0 if an error occurs. EXAMPLES
logger System rebooted logger -p local0.notice -t HOSTIDM -f /dev/idmc SEE ALSO
syslog(3), syslogd(8) STANDARDS
The logger utility conforms to IEEE Std 1003.2-1992 (``POSIX.2''). 4.3 Berkeley Distribution June 6, 1993 4.3 Berkeley Distribution
All times are GMT -4. The time now is 09:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy