Extract data from file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract data from file
# 15  
Old 01-18-2006
thanks a lot for ur reply and help ..
but i am getting one problem .. i am getting records but not count of all records ..


the fails records are filterd but not sum ..

one more thing i dont want to sum number of fails ...

i want to add number of failuers one record is one fail doesnt matter how many fails ..

pls do needful
# 16  
Old 01-18-2006
$ more logfile
2006-01-17 09:32:39 #### Testsuite started (plugin: sicat)
2006-01-17 09:32:40 (devel: 1/526) Running daily/DB/CmdLn/CmdLn1.exp ...
2006-01-17 09:34:40 (devel: 2/526) Running daily/DB/CmdLn/CmdLn2.exp ...
2006-01-17 09:34:41 (devel: 3/526) Running daily/DB/CmdLn/CmdLn3.exp ...
2006-01-17 09:34:42 (devel: 4/526) Running daily/DB/CmdLn/cliCS.exp ...
2006-01-17 09:34:42 (devel: 5/526) Running daily/DB/CmdLn/comma.exp ...
2006-01-17 09:34:43 (devel: 6/526) Error :daily/DB/CmdLn/no_licnse.exp :2 fails
2006-01-17 09:34:43 (devel: 7/526) Running daily/DB/CmdLn/parfile.exp ...
2006-01-17 09:34:44 (devel: 8/526) Running daily/DB/CmdLn/range.exp ...
2006-01-17 09:34:45 (devel: 9/526) Error : daily/DB/Filter/Filter1.exp :1 fails
2006-01-17 09:34:46 (devel: 10/526) Error : daily/DB/SFM/SFMDoublePointHole.exp : 5 fails

2006-01-17 09:34:47 (devel: 11/526) Running daily/DB/SFM/SFMDoublePointHole2.exp ...

2006-01-17 09:34:48 (devel: 12/526) Running daily/DB/SFM/SFMDoublePointHull.exp ...
2006-01-17 09:34:49 (devel: 13/526) Running daily/DB/SFM/SFMDoublePointHull2.exp ...
2006-01-17 09:34:50 (devel: 14/526) Running daily/DB/SFM/SFMSelfIntersection.exp ...
2006-01-17 09:34:51 (devel: 15/526) Running daily/DB/SFM/SFMSpeedAllAngleFlat.exp ...
2006-01-17 09:34:45 (prod: 9/526) Error : daily/DB/Filter/Filter1.exp :1 fails
2006-01-17 09:34:46 (prod: 10/526) Error : daily/DB/SFM/SFMDoublePointHole.exp : 5 fails

$ grep fails logfile | cut -d":" -f3 | sed 's/.*(\(.*\)/\1/g' | uniq -c
3 devel
2 prod
$
# 17  
Old 01-18-2006
or this,

awk -F":" ' /fails/ {print $3}' logfile | sed -e 's/^.*(//' | uniq -c

Last edited by matrixmadhan; 01-18-2006 at 10:11 AM.. Reason: typo
# 18  
Old 01-18-2006
hello

sorry i am getting errors ,, Pls mahendraji reply me .. i need ur help
the script which u wrote for me it is giving records of fails row by row
i need only total count ..


regards
# 19  
Old 01-18-2006
please post the error when you are getting error, also post me the output what you are getting, i could not guess what is going on at your end..

i guess the file format is different from what you have posted above..

i have provided the command based on the file format you have given, if there is any change to that, you need to modify the command accordingly...

initially you gaved one example, i have provided the command based on that..
later you have provided different log file format...

please note that this command would not work for any format, we need to make changes according to the format chat.

the command i have provided above works perfectly for the file format you have provided.

i would suggest you to take out 100 lines and excute the command i have provided above.

if it is not giving the output you expected, post me the 100 lines here and also the output from my command above and also your expected output...
# 20  
Old 01-19-2006
replace urfile by the logfile name u want

Add the following lines in a script, give it execute permissions and execute it.


## grep the lines with word fails, take the last field delimited by :, replace leading
## spaces, sort the output and store it in a tmp file
grep fails urfile|awk -F":" '{printf("%s\n",$NF)}'|sed 's/^ *//g'|sort > /tmp/tmp

## incr a counter till the first field on each line is unchanged, once changed print out
## old valued of the field and counter

awk 'BEGIN {
count=1
fld=$1;
}
{
if ( $1 == fld ) {
count++;
}
else {
printf ("version : %d\nTotal Fail : %d\n\n", fld, count);
fld=$1;
count=1;
}
}
END {
printf ("version : %d\nTotal Fail : %d\n\n", fld, count);
}' /tmp/tmp;
# 21  
Old 08-03-2006
Hi getdpg

If ur format is exactly same then run below code. For diffrent file u just change ur file name which is currently logfile

Code:
grep fails logfile | cut -d":" -f3 | cut -d"(" -f2 | uniq -c

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data from a file

Hi , I am having a file which is PIPE delimited like this : file.txt aus|start|10:00:00 nz|start|11:00:00 aus|end|10:10:00 us|start|10:00:00 nz|end|11:10:00 us|end|11:00:00 . . . I want to extract an output file like this based on start time and end time for each countries: (9 Replies)
Discussion started by: rohit_shinez
9 Replies

2. Shell Programming and Scripting

Extract data from a file

I have a text file that contains the following data. For example, aa.txt has some numbers. I need to extract the continuous numbers(minimum 3 numbers) from it.How can I do this with awk? >aa.txt 31 35 36 37 38 39 44 169 170 173 174 175 177 206 >1a.txt 39 (5 Replies)
Discussion started by: rahmanabdulla
5 Replies

3. Shell Programming and Scripting

Extract data from a file

Hello All, I have a small xml file which looks like below: <Check:defaultval Val="crash" value="crash_report_0013&#xA;generate_check_0020 generate_check_0022&#xA;&#xA;This is where the fault is."/> <Check:defaultval Val="crash" value="crash_report_1001&#xA;generate_check_1001... (9 Replies)
Discussion started by: suvendu4urs
9 Replies

4. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

5. Shell Programming and Scripting

Help with File Data Extract

Hello, Hope you are doing fine. I have been struggling with it for some time now and I would really appreciate your help. Following is file format: Currency,Name,Date, Term USD, ABC, 2011/11/11, T0, S1, S2, S3, S4 , , ,T1, 5.6, 2.3, 6.5, 4.5 , ... (5 Replies)
Discussion started by: srattani
5 Replies

6. Shell Programming and Scripting

Extract data based on match against one column data from a long list data

My input file: data_5 Ali 422 2.00E-45 102/253 140/253 24 data_3 Abu 202 60.00E-45 12/23 140/23 28 data_1 Ahmad 256 7.00E-45 120/235 140/235 22 data_4 Aman 365 8.00E-45 15/65 140/65 20 data_10 Jones 869 9.00E-45 65/253 140/253 18... (12 Replies)
Discussion started by: patrick87
12 Replies

7. Shell Programming and Scripting

Extract Data from a file

I need to create a script to extract some specific data from a file. I locate the file using the find command: find . -name "rpbol*" -print | xargs grep -li Once I locate the file I need using the above command, I would like to extract some data from that file. The data is always located... (2 Replies)
Discussion started by: jevaba
2 Replies

8. Shell Programming and Scripting

extract data from file

I m new to shell scripting & i need a help.... i have file like.... Name := sachin address:=something phone:=111 ... Note: There might be or not space between Name & := and between := & sachin. I need to extract the data from each line of file as var1=Name value1=sachin same for... (13 Replies)
Discussion started by: ps_sach
13 Replies

9. Shell Programming and Scripting

extract data from file

Hello again, how do you extract data from a file? I have created a file with PID #s in it, I need to be able to take the PID from each line and kill it. How is this done? (4 Replies)
Discussion started by: raidzero
4 Replies

10. Shell Programming and Scripting

extract data from file

My file in ksh consists of message data of varying lengths (lines), separated with headers. I would like to find a string from this file, and print out the whole message data including the headers. my plan of attack is to search the strings, print the top header, and print the whole message... (2 Replies)
Discussion started by: apalex
2 Replies
Login or Register to Ask a Question