Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

CDR manupulation

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting CDR manupulation
# 1  
Old 07-27-2012
CDR manupulation
Hello Friends,

I need to examine a huge CDR file according to a complex (at least for me) condition like below and i couldnt write anything Smilie

In CDR file there are more than hundreds of fields, I need to print the rows which matches the below condition:

while $13 field of subsequent CDRs are equal to eachother and when $NF == 202 ||500 then check if $3 of the lines and decide which of those $3 fields are smaller or bigger and sort it from smaller to bigger, data:

Code:
F1 | F2 | 2011111902634918288 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5437776699 | F14 | ...... | 202

F1 | F2 | 2011111902634918289 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5447776644 | F14 | ...... | 501

F1 | F2 | 2011111902634918255 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 202

F1 | F2 | 2011111902634918200 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 500

Id like to have the output

Code:
F1 | F2 | 2011111902634918200 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 500
F1 | F2 | 2011111902634918255 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 202


because above two rows has same $3 (gsm no) and $NF of them are 202 and 500 in turn (it could be opposite too ; 500 and 202) so the smaller of 3rd fields comes first in output..

Thanks in advance
Best Regards
# 2  
Old 07-28-2012
these two lines will be consecutive in the file? or anywhere in the file
# 3  
Old 07-28-2012
I made a couple of assumptions:

1) only records that should be printed are the ones with matching field 13. If a record has no match it isn't printed.

2) matching records would be consecutive in the input.

Code:
awk -F \| '
    function print_set(     i )
    {
        if( iidx < 2)           # print only if there were more than 1
            return;

        asort( idx3 );
        for( i = 1; i <= length( idx3 ); i++ )
            print buf[idx3[i]];
    }

    !( $NF == 202 || $NF == 500 ) { next; }

    p13 != $13 {                # not the same; print previous set if there
        print_set( );

        iidx = 0;               # reset cache
        delete idx;
        delete buf;
    }

    {                           # cache information untl we have a $13 that does not match
        p13 = $13;
        idx3[iidx++] = $3;
        buf[$3] = $0;
    }

    END {
        print_set( );
    }
' input-file >output-file

If all 202/500 records are to be printed, regardless of whether or not there was a matching record based on field 13, then comment out the first two lines (if and return) in the print_set function.
# 4  
Old 07-28-2012
Not sure if this is easier/faster:
Code:
sort -t"|" -k13,3 infile|uniq -f24 -w11 -D|grep -E "500$|202$"

after sorting the infile on fields 13 and 3, uniq -D will kill lines with field 13 unique, resulting in what you cited as desired output:
Code:
F1 | F2 | 2011111902634918200 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 500
F1 | F2 | 2011111902634918255 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | F12 | 5427776655 | F14 | ...... | 202

Last edited by RudiC; 07-28-2012 at 02:28 PM.. Reason: grepped 202|500 @ EOL
# 5  
Old 07-28-2012
Another option maybe:
Code:
awk -F\| '/ 500$| 202$/{if($13 in A)print A[$13] RS $0; else A[$13]=$0}' infile | sort -t\| -k3,3n



--
@agama:
Code:
awk: calling undefined function asort
 input record number 7, file infile
 source line number 7

@RudiC
Code:
uniq: illegal option -- w
usage: uniq [-c | -d | -u] [-i] [-f fields] [-s chars] [input [output]]

Last edited by Scrutinizer; 07-28-2012 at 05:41 PM..
This User Gave Thanks to Scrutinizer For This Post:
agama
# 6  
Old 07-29-2012
Sorry, should have stated uniq's version:
Code:
uniq (GNU coreutils) 8.13

-w indicates the # of chars to compare, i.e. sth like "field length"
# 7  
Old 07-30-2012
Quote:
Originally Posted by Scrutinizer
Another option maybe:
Code:
awk -F\| '/ 500$| 202$/{if($13 in A)print A[$13] RS $0; else A[$13]=$0}' infile | sort -t\| -k3,3n


--
@agama:
Code:
awk: calling undefined function asort
 input record number 7, file infile
 source line number 7

@RudiC
Code:
uniq: illegal option -- w
usage: uniq [-c | -d | -u] [-i] [-f fields] [-s chars] [input [output]]

Thank you all for your efforts,

Scrutinizer if the fields "500" and "202" wouldnt be consequtive in CDR files then would it be so diffucult to reach the same output? I guess it would be a long process to check 500 or 202 fields of same gsm_no in same file as there would be tousands of CDR rows in a file and the desired rows could come several hundred rows after the other one) corrcect?
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with appending random sequence to huge CDR file

Hi, I am in a terrible emergency. I have multiple cdr files with line count >6000. I need to append |0| | | | | | | |random| to end of each line. The random number should never repeat. Please help with a shell script to process all cdr's in a directory with above requirement. (23 Replies)
Discussion started by: shiburnair
23 Replies

2. Shell Programming and Scripting

awk for string manupulation

Hi All, I have one file with two columns separated by tab. I need to search for second column value of this file in the 5 column of another file. If the match is found replace the 5th column of second file with entire row of the first file. e.g. file1 123 D.abc 234 D.rde 4563 ... (2 Replies)
Discussion started by: alok2082
2 Replies

3. Shell Programming and Scripting

How to do Date Manupulation in Korn Shell?

Hi All, I need to find the date 19days back from the current date: eg: if today is 17 March 2013 then the output should be : 26 Feb 2013 Can i do this using date command in Korn Shell? And also if i need future 15 days date from current date, how to that? Any help appreciated :) ... (3 Replies)
Discussion started by: Arun Mishra
3 Replies

4. AIX

Rows manupulation using AWK or sed

Hi Everyon, I am stuck in a script.I have a file named file1.txt as given below: It contains 2 columns-count and filename. cat file1.txt count filename 100 A_new.txt 1000 A_full.txt 1100 B_new.txt 2000 B_full.txt 1100 C_new.txt 2000 C_full.txt ................... ..................... (10 Replies)
Discussion started by: rajsharma
10 Replies

5. Shell Programming and Scripting

parsing large CDR XML file

Dear Freind in the file attached how parse the data to be like a normal table :D (3 Replies)
Discussion started by: saifsafaa
3 Replies

6. UNIX for Dummies Questions & Answers

records manupulation

I am sending the data in userfile and colfile from ksh script to pl/sql script linto an array with this command grep '' $userfile |awk '{print "my_user_id("FNR") := '$SQL_QUOTE'"$1"'$SQL_QUOTE';"}' >> $SQL_TEMP_FILE grep '^\{1,10\}$' $colfile | awk '{print "my_col_id("NR") := "$1";"}' >>... (0 Replies)
Discussion started by: pinky
0 Replies

7. UNIX for Dummies Questions & Answers

Generating a CDR file using PHP scripts.

Hi, I need to do croning in PHP to generate a CDR (Call Details Record) file daily from the mysql database. The CDR file has to be named in a certain sequence such as xx00000xxx200604080850.cdr. A new file is written every day. The generated CDR file is then ftp over to a server. I am... (0 Replies)
Discussion started by: kurl
0 Replies

8. Shell Programming and Scripting

Date Manupulation

HI all, I am relatively new to Unix Shell Scripts ... I want to know how u can calculate the differnece between the 2 dates. As if in Oracle by using SYSDATE u get current date and time .. How one can achieve it in Unix ? Thanks.. (1 Reply)
Discussion started by: dhananjaysk
1 Replies

9. Filesystems, Disks and Memory

writing to a cdr drive.

ive looked around for software to do this in linux, but am still not sure how to. i want to be able to mount a cdr and write to it and be able to take the disk out and finish writing whatever i want to it at a later time. can we do this in linux? (4 Replies)
Discussion started by: norsk hedensk
4 Replies
Login or Register to Ask a Question