Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extract information from Log file formatted
Top Forums Shell Programming and Scripting Extract information from Log file formatted Post 302503479 by dummie55 on Thursday 10th of March 2011 06:33:12 PM
Old 03-10-2011
Extract information from Log file formatted
Good evening! Trying to make a shell script to parse log file and show only required information.
log file has 44 fields and alot of lines, each columns separated by ":".
log file is like:
Code:
first_1:3:4:5:6:1:3:4:5:something:notinterested
second_2:3:4:3:4:2
first_1:3:4:6:6:7:8

I am interested only in 5 fields which are : $3,$6,$14,$37,$38
So what i need is to parse that log file and display on screen something like this:
Code:
echo "Group	Jobs	WallClock	CPU	TotalMem	Memory"
        first_1     2              4                6              7                  8
        second_2    1              3                4              8                  3

The problem is that log file has group names repeated and what i have to do is just to read every line , pick unique group name and sum number of times its repeats on other lines so i get number of Jobs. Any help would be appreciated.

---------- Post updated at 06:33 PM ---------- Previous update was at 04:55 PM ----------

Trying something like :


Code:
#! /bin/sh
echo "Group	Jobs	WallClock	CPU	TotalMem	Memory"
awk -F":" ' { printf("%s	%d	%d	%d	%d	%d\n",$3,$6,$14,$37,$38) } ' $1

PS: runing script like ./myscript.sh log

i know its nothing but any suggestions how to continue?
Last edited by Scott; 03-10-2011 at 06:50 PM.. Reason: Please use code tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc... (2 Replies)
Discussion started by: Marcor
2 Replies

2. Shell Programming and Scripting

extract and format information from a file

Hi, Following is sample portion of the file; <JDBCConnectionPool DriverName="oracle.jdbc.OracleDriver" MaxCapacity="10" Name="MyApp_DevPool" PasswordEncrypted="{3DES}7tXFH69Xg1c=" Properties="user=MYAPP_ADMIN" ShrinkingEnabled="false" ... (12 Replies)
Discussion started by: sujoy101
12 Replies

3. Shell Programming and Scripting

Create shell script to extract unique information from one file to a new file.

Hi to all, I got this content/pattern from file http.log.20110808.gz mail1 httpd: Account Notice: close igchung@abc.com 2011/8/7 7:37:36 0:00:03 0 0 1 mail1 httpd: Account Information: login sastria9@abc.com proxy sid=gFp4DLm5HnU mail1 httpd: Account Notice: close sastria9@abc.com... (16 Replies)
Discussion started by: Mr_47
16 Replies

4. Shell Programming and Scripting

extract information from a log file (last days)

I'm still new to bash script , I have a log file and I want to extract the items within the last 5 days . and also within the last 10 hours the log file is like this : it has 14000 items started from march 2002 to january 2003 awk '{print $4}' < *.log |uniq -c|sort -g|tail -10 but... (14 Replies)
Discussion started by: matarsak
14 Replies

5. Shell Programming and Scripting

Extract various information from a log file

Hye ShamRock If you can help me with this difficult task for me then it will save my day Logs : ================================================================================================================== ... (4 Replies)
Discussion started by: SilvesterJ
4 Replies

6. Shell Programming and Scripting

How to extract information from a file?

Hi, i have a file like this: <Iteration> <Iteration_iter-num>3</Iteration_iter-num> <Iteration_query-ID>lcl|3_0</Iteration_query-ID> <Iteration_query-def>G383C4U01EQA0A length=197</Iteration_query-def> <Iteration_query-len>197</Iteration_query-len> ... (9 Replies)
Discussion started by: the_simpsons
9 Replies

7. Shell Programming and Scripting

Extract information from txt file

Hello! I need help :) I have a file like this: AA BC FG RF TT GH DD FF HH (a few number of rows and three columns) and I want to put the letters of each column in a variable step by step in order to give them as input in another script. So I would like to obtain: for the 1° loop:... (11 Replies)
Discussion started by: edekP
11 Replies

8. Shell Programming and Scripting

Extract information from file

Gents, If is possible please help. I have a big file (example attached) which contends exactly same value in column, but from column 2 to 6 these values are diff. I will like to compile for all records all columns like the example attached in .csv format (output.rar ).. The last column in the... (11 Replies)
Discussion started by: jiam912
11 Replies

9. Shell Programming and Scripting

Extract information from file

In a particular directory, there can be 1000 files like below. filename is job901.ksh #!/bin/ksh cront -x << EOJ submit file=$PRODPATH/scripts/genReport.sh maxdelay=30 &node=xnode01 tname=job901 &pfile1=/prod/mldata/data/test1.dat ... (17 Replies)
Discussion started by: vedanta
17 Replies

10. Shell Programming and Scripting

sed / awk / grep to extract information from log

Hi all, I have a query that runs that outputs data in the following format - 01/09/12 11:43:40,ADMIN,4,77,Application Group Load: Name(TESTED) LoadId(5137-1-0-1XX-15343-15343) File(/dir/dir/File.T03.CI2.RYR.2012009.11433350806.ARD) InputSize(5344) OutputSize(1359) Rows(2) Time(1.9960)... (8 Replies)
Discussion started by: jeffs42885
8 Replies

LEARN ABOUT OSX

uulog

uulog(1)						      General Commands Manual							  uulog(1)

NAME

       uulog - display entries in the UUCP log file.

SYNOPSIS

       uulog  [-#]  [-n  lines] [-sf system] [-u user] [-DSF] [--lines lines] [--system system] [--user user] [--debuglog] [--statslog] [--follow]
       [--follow=system]

DESCRIPTION

       The uulog program can be used to display entries in the UUCP log file.  It can select the entries for a particular system or  a	particular
       user.  You can use it to see what has happened to your queued jobs in the past.	Different options may be used to select which parts of the
       file to display.

OPTIONS

       -#, -n lines, --lines lines
	    Here '#' is a number; e.g., `-10'.	The specified number of lines is displayed from the end of the log file.  The default is  to  dis-
	    play the entire log file, unless the -f, -F, or --follow options are used, in which case the default is to display 10 lines.

       -s system, --system system
	    Display only log entries pertaining to the specified system.

       -u, --user user
	    Display only log entries pertaining to the specified user.

       -D --debuglog
	    Display the debugging log file.

       -S, --statslog
	    Display the statistics log file.

       -F, --follow
	    Keep displaying the log file forever, printing new lines as they are appended to the log file.

       -f system, --follow=system
	    Keep displaying the log file forever, displaying only log entries pertaining to the specified system.

       Standard UUCP options:

       -X type, --debug type
	    Turn  on  particular debugging types.  The following types are recognized: abnormal, chat, handshake, uucp-proto, proto, port, config,
	    spooldir, execute, incoming, outgoing.  --debug option may appear multiple times.  A number may also be given, which will turn on that
	    many types from the foregoing list; for example, --debug 2 is equivalent to --debug abnormal,chat.

       -I file, --config
	    Set configuration file to use.

       -v, --version
	    Report version information and exit.

       --help
	    Print a help message and exit.

SEE ALSO

       uucp(1)

AUTHOR

       Ian Lance Taylor <ian@airs.com>.  Text for this Manpage comes from Taylor UUCP, version 1.07 Info documentation.

								 Taylor UUCP 1.07							  uulog(1)
All times are GMT -4. The time now is 06:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy