Text Manipulation.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Text Manipulation.
# 1  
Old 02-22-2008
Error Text Manipulation.

Hi

I have only ever used awk and sed for basic requirements up until now.
I have had to break a log down for multiple purposes.
Using awk, sed and a date script. I am left with this:

(message id, time of msg attempt, message id, domain name[from senders address], time of msg completion)

1JRkPs-0008m8-Fd 7230901
1JRkPs-0008m8-Fd domain.com 7230902
1JRkPs-0008m8-Fd abc.com 7230961
1JRkaH-0009E0-VZ 7231546
1JRkaH-0009E0-VZ domain.co.uk 7231547
1JRl5D-000AMD-22 7229863
1JRl5D-000AMD-22 123.com 7229864
1JRl66-000AOR-AZ 7229918
1JRl66-000AOR-AZ xyz.co.za 7229919


What this represents is email logs for sending/receiving.
The first entry is MSG id and time of attempt in seconds.
Second entry is MSG id , recipient and time of msg completion in seconds.

I am attempting to subtract the 2nd entry's' time with the first entry, and if there is a 3rd entry, do the same and if there is a 4th and so on and display the output like:

1JRkPs-0008m8-Fd
1JRkPs-0008m8-Fd domain.com (1s) abc.com (60s)
1JRkaH-0009E0-VZ
1JRkaH-0009E0-VZ domain.co.uk (1s)

My attempt is to work out delays from which domains are longer than others.
So in my example above, domain.com message was queued for 1 second, where as abc.com was queued for 60 seconds.

I wrote a script below which can perform the re-arranging, however not the calculations. Could someone please be of some assistance?

Many thanks in advance.

Code:
#!/usr/bin/awk -f
BEGIN {
       KEY=""
       DATA=""
}
{
       if($1 != KEY){
               if(KEY!=""){
                       printf(" %s %s\n",
 KEY, DATA)
               }
               DATA=$2
       } else {
               DATA=DATA" "$2
       }
       KEY=$1
}
END {
       printf(" %s %s\n", KEY, DATA)
}

# 2  
Old 02-22-2008
[Update]

So far I have managed to get the output looking like this:

1JRk2I-0007wT-Dy 7229440 domain.com 7229440 abc.com 7230019
1JRkPs-0008m8-Fd 7230901 xyz.com 7230902 domain.co.uk 7230961
1JRkaH-0009E0-VZ 7231546 test.com 7231547
1JRl5D-000AMD-22 7229863 test.co.uk 7229864

By updating the script to look like this:

Code:
#!/usr/bin/awk -f
BEGIN {
       KEY=""
       DATA=""
}
{
       if($1 != KEY){
               if(KEY!=""){
                       printf(" %s %s\n",
 KEY, DATA)
               }
               DATA=$2" "$3
       } else {
               DATA=DATA" "$2" "$3
       }
       KEY=$1
}
END {
       printf(" %s %s\n", KEY, DATA)
}

Im still stuck on the calculations.

Any advice would be awesome.

Thanks.
# 3  
Old 02-22-2008
Try the following:
Code:
#!/usr/bin/awk -f

BEGIN {
   msg_id=""
}

{
   if ($1 != msg_id) {
      if (msg_id != "")
          print outstr
      msg_id=$1; stime=$2
      outstr=" "$1" "$2" "$3
   } else
      outstr=outstr" "$2" ("$3-stime"s) "
}

END {
   print outstr
}

which gives the following output from the sample data you supplied
Code:
 1JRkPs-0008m8-Fd 7230901  domain.com (1s)  abc.com (60s)
 1JRkaH-0009E0-VZ 7231546  domain.co.uk (1s)
 1JRl5D-000AMD-22 7229863  123.com (1s)
 1JRl66-000AOR-AZ 7229918  xyz.co.za (1s)

# 4  
Old 02-22-2008
Another awk solution:
Code:
awk '{
keys[$1]++
time[$1,keys[$1]]=$NF
if ( NF > 2 )
   dat[$1,keys[$1]]=$2
}
END {
for ( it in keys )
   {
   for (i=1;i<=keys[it];i++)
       if ( i == 1 )
             printf("%s\n%s",it,it)
       else
          printf(" %s (%ds) ",dat[it,i],time[it,i]-time[it,1])
      printf("\n")
   } 
}' log_file
1JRkPs-0008m8-Fd
1JRkPs-0008m8-Fd domain.com (1s) abc.com (60s) 
1JRkaH-0009E0-VZ
1JRkaH-0009E0-VZ domain.co.uk (1s) 
1JRl66-000AOR-AZ
1JRl66-000AOR-AZ xyz.co.za (1s) 
1JRl5D-000AMD-22
1JRl5D-000AMD-22 123.com (1s)


Last edited by Klashxx; 02-22-2008 at 01:36 PM.. Reason: optimization
# 5  
Old 02-25-2008
Thanks to the both of ya. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help text manipulation

Hello Forum , I need a help about text manupulation. I have a text file and I have to manipulate this file. Let's say source.txt source.txt UNB+UNOC:3+O0013000005MAN MN RVS:91+0098006688:92+190304:2313+F004169241' UNH+8146848+DELJIT:D:96A:UN' BGM+307:::JIS_SYNCRO_FIRM+2019030423234101+9'... (8 Replies)
Discussion started by: cemokam65
8 Replies

2. Shell Programming and Scripting

Text manipulation help

Hello again, I have a problem manipulating a large text document and there is no way I could edit this document by hand. Format is: Address : XXXX N 37 Ave, Hollywood, FL, 33021 Phone: XXX3190XXX Player: XXXXXX Character: Jaramillo DOB: June-14-1995 ----- Name: Alexandra Ticket... (3 Replies)
Discussion started by: galford
3 Replies

3. UNIX for Dummies Questions & Answers

Text manipulation

i want to generate a list line-by-line of normal characters using letters . for example : dnds gnos mgod pets jnfp etc... i want to use all letters with all the posibilities is there a script that can do this ? (3 Replies)
Discussion started by: suppliernr1
3 Replies

4. UNIX for Dummies Questions & Answers

Text manipulation help

Hello unix.com users, I have a ip file (line-by-line). How can I delete the ips that keep repeating by mark XXX.XXX.XXX.* ... I want to erase only the lines that keep repeating more than 2 times. Example: 1.2.3.1 1.2.3.2 1.2.3.3 I want to erase all ips blocks that are repeating by C... (1 Reply)
Discussion started by: galford
1 Replies

5. UNIX for Dummies Questions & Answers

Text Manipulation Help

Hello Unix.com, I have a text in format: john sara lee How can I make it: john:john john:john1 john:john12 john:john123 sara:sara sara:sara12 sara:sara123 and so on (2 Replies)
Discussion started by: galford
2 Replies

6. UNIX for Dummies Questions & Answers

text manipulation help

Hello again unix.com How can I extract from a large file in format: steve@aol.com steve hawkins Location of this member is bla bla bla sun@hotmail.com Sun Ying This member is using browser bla bla bla to another text in format: steve@aol.com steve hawkins sun@hotmail.com sun ying ... (5 Replies)
Discussion started by: galford
5 Replies

7. Shell Programming and Scripting

[HELP] Text manipulation... [HELP]

I need to know how can I remove all word after comma on each line. Like: jjkj,iiuiui,ijlkjkij,ookoo kijljlj,jhhkj,ijijkijkj,oijkijj kjkljlkj,kjkjlkjlkj,opok,okop to jjkj, kijljlj, ... (5 Replies)
Discussion started by: slutb3
5 Replies

8. UNIX for Dummies Questions & Answers

Text Manipulation

Greetings. I“m a biologist and I don“t have mucho knowledge on Unix/Linux, but I need to use Cygwin to change some documents from a GenBank format to a FASTA format. GenBank format goes somthing like this: LOCUS NM_013964 2568 bp mRNA linear PRI 26-APR-2009... (2 Replies)
Discussion started by: vanesa1230
2 Replies

9. UNIX for Dummies Questions & Answers

Help with text manipulation

Hi there, I have some text files in unix format that processed by a program in windows, and when I open them with less or vi in linux, a warn for opening binary file is prompted, and as shown in vi, between every two characters there was inserted a "^@". How can I fix this. Plus, there are over... (2 Replies)
Discussion started by: dustinwang2003
2 Replies

10. UNIX for Dummies Questions & Answers

text manipulation

I am tryin to figure out how to extract interested text from file example.txt blah blah blah a: child1 blah a: child2 blah b: parent1 blah blah blah .... blah a: child21 blah a: child22 blah a: child23 blah b: parent2 this kinda text repeats .. number of children is... (6 Replies)
Discussion started by: rajkishore
6 Replies
Login or Register to Ask a Question