AWK to classify a file into several ones ..


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK to classify a file into several ones ..
# 1  
Old 09-04-2012
AWK to classify a file into several ones ..

Good Day All,

I need to make a script that will do the following :
1- read a .csv file line by line, check the 3 field of each file print the whole line if
this field matches the condition (note : FS = ",")

2-from the basic file, the script should genrate 3 new files based on the step #1 findings.

3- the script needs to keep the 1 line on the orignial .csv on each new genertaed beuase this is the title.

here is what I tried so far :

1st idea :
=========================
Code:
#!/usr/bin/bash
FILE_NAME="Statechange-huawei-bt1sss6o-20120830000506-alarm-log-auto-1.csv"
d= date '+ %Y%m%d%H%M%S'


## get LTE alarmes

awk  -F, '{$3 = "LTE" ; print $0 }'  $FILE_NAME >> LTE_alarms_.csv

=========================

2nd idea :

Code:
#!/usr/bin/bash
FILE_NAME="Statechange-huawei-bt1sss6o-20120830000506-alarm-log-auto-1.csv"
d= date '+ %Y%m%d%H%M%S'


## get GSMalarmes
cat $FILE_NAME | grep "GSM" | awk '{print $0}' >> GSM_alarms_$d.csv


Many Thanks in advance.
Kemo

Last edited by Scrutinizer; 09-05-2012 at 12:41 AM..
# 2  
Old 09-04-2012
This is untested, so there might be a bug I didn't catch. You didn't give a third type in your post, but I think you can figure out how to fill it in.

Awk can be used to create all three files at the same time:

Code:
d=$( date "+ %Y%m%d%H%M%S" )
awk -v date="$date" '
    NR == 1 {
        f1 = sprintf( "LTE_alarms_%s.csv", date );
        f2 = sprintf( "GSM_alarms_%s.csv", date );
         f3 = sprintf( "UTMS_alarms_%s.csv", date );

        print >f1;      # capture header line into each output file
        print >f2;
        print >f3;
        next;
    }

    $3 == "LTE" { print >f1; next; }   #capture each type into their own file
    $3 == "GSM" { print >f2; next; }
     $3 == "UTMS" { print >f3; next; }
' input-file


Last edited by agama; 09-05-2012 at 11:29 AM.. Reason: Minor changes to add UTMS based on later post with more clarification
# 3  
Old 09-05-2012
please provide the sample input and required output
# 4  
Old 09-05-2012
Thanks, input file is attached

Thanks agama, I'll try your script and keep you informed.

input file is attached and output files should be somthing like this :
LTE_alarms_%s.csv
GSM_alarms_%s.csv
UTMS_alarms_%s.csv
(output files should have the same headr 'title' as the input file.

Thanks,
Ibrahim.
# 5  
Old 09-05-2012
Quote:
Originally Posted by engkemo2002
awk -F, '{$3 = "LTE" ; print $0 }' $FILE_NAME >> LTE_alarms_.csv[/CODE]
It's a pretty good idea but you have to learn more about AWK:
Code:
echo 'a,b,c' | awk  -F, '{$3 = "LTE" ; print $0 }'
=> a b LTE

So if you wan to print only line with LTE in $3 there are a lot of solutions :
Code:
awk  -F, '{ if ($3 == "LTE") { print $0} }'
awk  -F, '$3 == "LTE" { print $0}'
awk  -F, '$3 == "LTE"'

please note that the awk basis syntax is :
Code:
awk 'expression {commands}'

the awk check the expression for every line and execute commands if the expression is true. If you don't put the expression (first awk) the commands will be executed on every lines. If you don't put command (so only the expression like the last awk) the "defaut" command is "print $0".

If I look your file you want *LTE* in $3. So you have to use regex with ~ :
Code:
awk  -F, '$3 ~ /LTE/'

# 6  
Old 09-06-2012
Try this as per your input file
Code:
dat1=$( date "+ %Y%m%d%H%M%S" )
awk -F"," -v dat="$dat1" '
NR==1{x=$0;next}
 ($3 ~ /GSM|LTE|UTMS/){
			split($3,a," ");
			if(!b[a[2]])
				{
					b[a[2]]=sprintf( "%s_alarms_%s.csv",a[2],dat );
					print x > b[a[2]]
				};
			print > b[a[2]]
			}' filename

# 7  
Old 09-11-2012
Thanks a Million to all of you.
problem has been solved.

Rgds,
Ibrahim
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Classify lines in file using perl

The below perl executes and does classify each of the 3 lines in file.txt. Lines 2 and 3 are correct as they fit the criteria for Rule 2. The problem is that line one should be classified VUS as it does not meet the criteria for Rule 1, so Rule 3 is used. However, currently Rule 2 is changing the... (27 Replies)
Discussion started by: cmccabe
27 Replies

2. Shell Programming and Scripting

Awk: Print count for column in a file using awk

Hi, I have the following input in a file & need output as mentioned below(need counter of every occurance of field which is to be increased by 1). Input: 919143110065 919143110065 919143110052 918648846132 919143110012 918648873782 919143110152 919143110152 919143110152... (2 Replies)
Discussion started by: siramitsharma
2 Replies

3. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

4. Shell Programming and Scripting

Split File by Pattern with File Names in Source File... Awk?

Hi all, I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example: ... (2 Replies)
Discussion started by: cul8er
2 Replies

5. UNIX for Dummies Questions & Answers

Classify value to a range

Dear All, I need to classify my data into sets or ranges based on values in the second column of a file as - low medium and high. INPUT: file1.dat 1.tmp 1.03 2.tmp 0.38 3.tmp 3.23 4.tmp 1.34 I would like to classify all the numerical values into a range based on the followng... (3 Replies)
Discussion started by: chen.xiao.po
3 Replies

6. Shell Programming and Scripting

write awk command into file using awk

hi, i want to write my nawk command into a file. i want to write: awk 'NR==14 && $NF!="Set."{print "l ./gcsw "r" '"'"'lt all;"p"'"'"'"} NR==5{r=$2} NR==3{p=$2 FS $3 FS $4 FS $5}' $logfile > /home/gc_sw/again.mos'"'"' into gc.mos file. this is my code: awk '{print "awk 'NR==14 &&... (6 Replies)
Discussion started by: gc_sw
6 Replies

7. Shell Programming and Scripting

How to use awk to classify file extension from input ls -l

i try to do this for a long time input is command ls -l and output is: Number of files : xx Number of file type – awk : 5 total size: 2345 bytes // file ex type .awk Number of file type – dat : 10 total size: 233 bytes // file ex type .dat ... Number of unknown file type... (1 Reply)
Discussion started by: retsuseiba
1 Replies

8. Shell Programming and Scripting

Parse file using awk and work in awk output

hi guys, i want to parse a file using public function, the file contain raw data in the below format i want to get the output like this to load it to Oracle DB MARWA1,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 MARWA2,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 this the file raw format: Number of... (6 Replies)
Discussion started by: dagigg
6 Replies
Login or Register to Ask a Question