Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parse and count lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse and count lines
# 1  
Old 12-30-2008
Parse and count lines
I have a data file in the following format (refer to input file) with multiple lines containing some information. I need an output file to loop thorough the input file with summarized information as seen below (refer to output file) ‘Date Time' and ‘Beta Id' input file values should be concatenated and count of detail records should be presented in the same line. The file might contain ‘n' number of blocks like this. I am new to shell scripting and appreciate your help.

Input file
_________

Date Time 20081209
Beta Id X123
<item type="CUST" preferredCon="1234"</item>
<item type="CUST" preferredCon="5678"</item>
<item type="CUST" preferredCon="9012"</item>
</request>
Date Time 20081211
Beta Id X123
<item type="CUST" preferredCon="4567"</item>
<item type="CUST" preferredCon="1230"</item>
<item type="CUST" preferredCon="3246"</item>
<item type="CUST" preferredCon="5465"</item>
</request>

Output File:
__________

Date Time 20081209 Beta Id X123 Count 3
Date Time 20081211 Beta Id X123 Count 4
# 2  
Old 12-30-2008
does your <item type> always ends with </request> ?

- nilesh
# 3  
Old 12-30-2008
Yes, it ends with </request> always for each set in a block.
# 4  
Old 12-30-2008
Quote:
Originally Posted by shekharaj
Yes, it ends with </request> always for each set in a block.
Doesn't that mean its a bad-ly formatted data?
# 5  
Old 12-30-2008
Here is a sample in perl

Code:
#! /opt/third-party/bin/perl

use strict;
use warnings;

open(FI, "<", $ARGV[0]) or die "Unable to open file $ARGV[0] - < $! > \n";

my $cnt = 0;
my $set = 0;

my $store;
my $sec_store;
while ( <FI> ) {
  chomp;

  if ( $set == 1 ) {
    $cnt++ if ( /item type/ );
    $sec_store = $_ if ( $cnt == 0 );
  }

  if( /Date Time/ ) {
    $store = $_;
    $set = 1;
  }

  if ( /request/ ) {
    print $store, " ", $sec_store, " Count ", $cnt, "\n";
    $set = 0;
    $cnt = 0;
  }
}

close(FI);

# 6  
Old 12-30-2008
Bug Use awk!
I send you an attached file named panos1.txt. Run:

Code:
awk -f panos1.txt input_file

panos1.txt (307 Bytes) 
# 7  
Old 12-30-2008
Hi,

as awk one-liner:

Code:
awk '/Date/{d=$0;getline;d = d FS $0};\
  /item/{i++};\
  /request/{print d " Count " i;d="";i=""}' inputfile

HTH Chris
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to parse multiple lines

What is the correct syntax to have the awk parse the next line as well? The next in bold is where I think it should go, but I wanted to ask the experts since I am a beginner. The file to be parsed is attached as well. Thank you :). awk 'NR==2 {split($2,a,"");b=substr(a,1,length(a-1));print... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. UNIX for Dummies Questions & Answers

Parse User Logon Count

Hey, My access.log for one application is not rolled over. I would like to get users count. Right now I am doing cat logon.list | grep 2013-09-11| |grep Succeeded | wc -l Since the file doesnt rotate out I have to grep for new date/ I tried to specify DATE=date "+DATE:... (2 Replies)
Discussion started by: brabored
2 Replies

3. Shell Programming and Scripting

parse lines

I have file which is having 500 lines. I want to get the first 100 lines then sleep, then again next 100 lines sleep so now till the end of the file. Can someone tell me in perl and bash. also i want to do it in threads. Thanks.. (6 Replies)
Discussion started by: Anjan1
6 Replies

4. Shell Programming and Scripting

Parse large file on line count (random lines)

I have a file that needs to be parsed into multiple files every time there line contains a number 1. the problem i face is the lines are random and the file size is random. an example is that on line 4, 65, 187, 202 & 209 are number 1's so there has to be file breaks between all those to create 4... (6 Replies)
Discussion started by: darbs121
6 Replies

5. UNIX for Dummies Questions & Answers

How to parse 2 particular lines from Command output

Hi All, I need help on the following req. I am getting output of a command as follows: 16377612 total memory 3802460 used memory 2827076 active memory 681948 inactive memory 12575152 free memory 477452 buffer memory I want to compute used... (1 Reply)
Discussion started by: mailsara
1 Replies

6. Shell Programming and Scripting

Parse out specific lines

Hello, For the life of me, I can't figure out how to extract only certain lines of a file. For example, the file contains: project.max-sem-ids privileged 1.02K - deny - system 16.8M max deny ... (2 Replies)
Discussion started by: PointyWombat
2 Replies

7. Shell Programming and Scripting

Parse and delete lines efficiently

Hi I have a set of options in the form of key value in a file. Need to find a particular value of 'a' and delete all lines till the next 'a' keyword . Ex : a bbb c ddd e fff g hhh a sss c ggg e xxx f sss a ddd d sss r sss g hhh (5 Replies)
Discussion started by: TDUser
5 Replies

8. Shell Programming and Scripting

Extra/parse lines from a file between unque lines through the file

I need help to parse a file where there are many records, all of which are consistently separated by lines containing “^=============” and "^ End of Report". Example: ============= 1 2 3 4 End of record ============= 1 3 4 End of record Etc.... I only need specific lines... (5 Replies)
Discussion started by: jouuu
5 Replies

9. Shell Programming and Scripting

parse of lines with different delimiters

Hi, I am having huge file with the following lines. 2007:10:01:00:00:49:GMT: subject=BMRA.BM.T_ABTH7.FPN, message={SD=2007:10:01:00:00:00:GMT,SP=5,NP=2,TS=2007:10:01:01:00:00:GMT,VP=0.0,TS=2007:10:01:01:30:00:GMT,VP=0.0} 2007:10:01:00:00:49:GMT: subject=BMRA.BM.T_ABTH7G.FPN,... (9 Replies)
Discussion started by: nathasha
9 Replies

10. UNIX for Dummies Questions & Answers

How to count lines - ignoring blank lines and commented lines

What is the command to count lines in a files, but ignore blank lines and commented lines? I have a file with 4 sections in it, and I want each section to be counted, not including the blank lines and comments... and then totalled at the end. Here is an example of what I would like my... (6 Replies)
Discussion started by: kthatch
6 Replies
Login or Register to Ask a Question