count characters in specific records


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting count characters in specific records
# 1  
Old 02-03-2009
count characters in specific records

I have a text file which represents a http flow like this:

Code:
HTTP/1.1 200 OK 
Date: Fri, 23 Jan 2009 17:16:24 GMT 
Server: Apache 
Last-Modified: Fri, 23 Jan 2009 17:08:03 GMT 
Accept-Ranges: bytes 
Cache-Control: max-age=540 
Expires: Fri, 23 Jan 2009 17:21:31 GMT 
Vary: Accept-Encoding 
Content-Encoding: gzip 
Age: 233 
Content-Length: 1276 
Keep-Alive: timeout=1, max=198 
Connection: Keep-Alive 
Content-Type: application/x-javascript 

.4..5...5...>...>.m.....>.Y..?...i...W....X.....&..3}..C}0...3kt.{.k.l....@..........=`...=...e
.g.w.{i.a.,~...Gy..[..s...Bo=.An.$.........m..6j..j....
Dy..Z..:a...m.....qQ...Y&F.Q..*.`<3.uO<....2.3hA.PG4.}:*o..2.z..T..PL<QUX....j........b.%.2.....].v..Nq..iK..S..d..q....9w<.\...............p.9w.v."~......#.'...M.Ju.........n...z.H%Ae..a...[R...rX....k.:.EM..j.W.Z\5..K.v....!n.)..K..JQVT0..X?{v..Bm.......rN?.>>....0..s....*.+....Q.^.#..g.^G.I........;.a..fq......^....S..?g.k[^JE...9+.....HTTP/1.1 200 OK 
Date: Fri, 23 Jan 2009 17:16:24 GMT 
Server: Apache 
Last-Modified: Thu, 18 Sep 2008 09:38:18 GMT 
Accept-Ranges: bytes 
Content-Length: 1903 
Cache-Control: max-age=604800 
Expires: Fri, 30 Jan 2009 17:08:23 GMT 
Age: 481 
Keep-Alive: timeout=1, max=197 
Connection: Keep-Alive 
Content-Type: image/jpeg 
 
......JFIF.....x.x.....C..............

I need to count characters (bytes) in packet payloads. I noticed that if a packet has a payload this is after a blank line following the headers. It is possible that a new header is attached to a preceding packet as shown in the example. I need an output like this:

pkt1 payload size
pkt2 payload size
....... ........
pktn payload size

Thanks in advance
# 2  
Old 02-05-2009
Each payload ends with something like "HTTP/1.1 200 OK" and the next one starts with Date:

So you could do something like this:
Code:
awk /^Content-Type: /,/^Date: / && /GMT$/ { packet++; count+=length($0); } /^Server: / { print "Packet " packet, "payload size",count; count=0; } END { print count; }

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Find records with specific characters in 2 nd field

Hi , I have a requirement to read a file ( 5 fields , ~ delimited) and find the records which contain anything other than Alphabets, Numbers , comma ,space and dot . ie a-z and A-Z and 0-9 and . and " " and , in 2nd field. Once I do that i would want the result to have field1|<flag> flag can... (2 Replies)
Discussion started by: ashwin3086
2 Replies

2. Shell Programming and Scripting

Count records in a block

Hi, We have a file that has a structure like this: H10 1 2 3 D10 1 D20 1 2 3 D20 3 4 5 D20 4 5 6 D10 2 D20 1 2 3 D20 3 4 5 D20 4 5 6 S10 10 H10 1 2 3 D10 1 D20 1 2 3 D20 3 4 5 D20 4 5 6 D10 2 (2 Replies)
Discussion started by: jerome_rajan
2 Replies

3. Shell Programming and Scripting

Count specific characters at specific column positions

Hi all, I need help. I have an input text file (input.txt) like this: 21 GTGCAACACCGTCTTGAGAGG 50 21 GACCGAGACAGAATGAAAATC 73 21 CGGGTCTGTAGTAGCAAACGC 108 21 CGAAAAATGAACCCCTTTATC 220 21 CGTGATCCTGTTGAAGGGTCG 259 Now I need to count A/T/G/C numbers at each character location in column... (2 Replies)
Discussion started by: thienxho
2 Replies

4. Shell Programming and Scripting

Can't figure out how to find specific characters in specific columns

I am trying to find a specific set of characters in a long file. I only want to find the characters in column 265 for 4 bytes. Is there a search for that? I tried cut but couldn't get it to work. Ex. I want to find '9999' in column 265 for 4 bytes. If it is in there, I want it to print... (12 Replies)
Discussion started by: Drenhead
12 Replies

5. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

6. Shell Programming and Scripting

Script to count particular type of records

Hi, I have a huge file containing thousands of records which are of following pattern: TYPE1 { originNodeType : "IVR" originHostName : "AAIVR" originTransactionID : "01310559" originTimeStamp : "20110620192440+0530" hostName : "hhhh" voucher : '0'D rProfileID : "ZZZZ" Before {... (5 Replies)
Discussion started by: madhukar1anand
5 Replies

7. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

8. UNIX for Dummies Questions & Answers

Unix command to count the number of files with specific characters in name

Hey all, I'm looking for a command that will search a directory (and all subdirectories) and give me a file count for the number of files that contain specific characters within its filename. e.g. I want to find the number of files that contain "-a.jpg" in their name. All the searching I've... (6 Replies)
Discussion started by: murphysm
6 Replies

9. Shell Programming and Scripting

Awk to Count Records with not null

Hi, I have a pipe seperated file I want to write a code to display count of lines that have 20th field not null. nawk -F"|" '{if ($20!="") print NR,$20}' xyz..txt This displays records with 20th field also null. I would like output as: (4 Replies)
Discussion started by: pinnacle
4 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question