Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with awk command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with awk command
# 1  
Old 03-22-2012
Help with awk command
Hi all,

I have to parse something like this file
Code:
! may appear after the conversion factors (e.g. (0:855)).

! The U.S. unit conversions in this file have been
! optimized for the HVAC industry.

set program_id 00:00:00:00:00:00:00:00;
set selector STANDARD;

SCPTactFbDly: 	text("%d %d:%d:%d:%d", day, hour, minute, second, millisecond);
SCPTactFbDly#LO: 	text(("%d ", day), time(hour, minute, second, millisecond));
SCPTactuatorCharacteristic: 	text("%m");
SNVT_dev_c_mode: 	text("%m");
SNVT_dev_fault: 	text(("DS=%m ",device_select),((device_select == 0) ?("SF: [VL=%d VH=%d PH=%d NF=%d PL=%d PH=%d {%d %d}] DF: [MT=%d MF=%d PB=%d ET=%d EF=%d EX=%d SF=%d {%d %d}]", 

dev_type.pump_ctrl.sf_voltage_low,dev_type.pump_ctrl.sf_voltage_high,dev_type.pump_ctrl.sf_phase,dev_type.pump_ctrl.sf_no_fluid,dev_type.pump_ctrl.sf_press_low,dev_type.pump_ctrl.sf_press_high,dev_type.pump_ctrl.sf_reserved1_6,dev_type.pump_ctrl.sf_reserved1_7,dev_type.pump_ctrl.df_motor_temp,dev_type.pump_ctrl.df_motor_failure,dev_type.pump_ctrl.df_pump_blocked,dev_type.pump_ctrl.df_elect_temp,dev_type.pump_ctrl.df_elect_failure_nf,dev_type.pump_ctrl.df_elect_failure,dev_type.pump_ctrl.df_sensor_failure,dev_type.pump_ctrl.df_reserved2_7,dev_type.pump_ctrl.reserved3_0_7) :( (device_select == 1) ?("DF: [VB=%d DO=%d DC=%d PE=%d SOR=%d I=%d VC=%d ED=%d {%d}] EE: [O=%d VL=%d VS=%d {%d %d}] SF: [VOR=%d ET=$d FR=%d {$d}] GF=%d", 

dev_type.valve_pos.df_valve_blocked,dev_type.valve_pos.df_blocked_direction_open,dev_type.valve_pos.df_blocked_direction_close,dev_type.valve_pos.df_position_error,dev_type.valve_pos.df_stroke_out_of_range,dev_type.valve_pos.df_initialization,dev_type.valve_pos.df_vibration_cavitation,dev_type.valve_pos.df_ed_too_high,dev_type.valve_pos.reserved1_0_2,dev_type.valve_pos.ee_oscillating,dev_type.valve_pos.ee_valve_too_large,dev_type.valve_pos.ee_valve_too_small,dev_type.valve_pos.reserved2_6_7,dev_type.valve_pos.reserved3_0_7,dev_type.valve_pos.sf_voltage_out_of_range,dev_type.valve_pos.sf_electronic_high_temp,dev_type.valve_pos.sf_frictional_resistance,dev_type.valve_pos.reserved4_4_6,dev_type.valve_pos.general_fault) :("UNKNOWN"))));
SNVT_dev_maint: 	text(("DS=%m ",device_select),((device_select == 0) ?("RQ=%d BC=%d BL=%d SC=%d {%d %d %d}",dev_type.pump_ctrl.service_required,dev_type.pump_ctrl.bearings_change,dev_type.pump_ctrl.bearings_lubricate,dev_type.pump_ctrl.shaftseal_change,dev_type.pump_ctrl.reserved1_4_7,dev_type.pump_ctrl.reserved2_0_7,dev_type.pump_ctrl.reserved3_0_7) :((device_select == 1) ?("MM=%d PC=%d EC=%d PC=%d LC=%d RC=%d BC=%d {%d %d %d} GM=%d",dev_type.valve_pos.motor_maint,dev_type.valve_pos.packing_change,dev_type.valve_pos.electronics_check,dev_type.valve_pos.positioning_check,dev_type.valve_pos.lubrication_check,dev_type.valve_pos.return_check,dev_type.valve_pos.battery_check,dev_type.valve_pos.reserved1_7,dev_type.valve_pos.reserved2_0_7,dev_type.valve_pos.reserved3_0_6,dev_type.valve_pos.general_maint) : ("UNKNOWN") )));
SCPTalrmClrT1#LO: 	text(("%d ", day), time(hour, minute, second, millisecond));

I need to have 2 field.

The first one should have this word only
Code:
SCPTactFbDly
SCPTactFbDly#LO
SCPTactuatorCharacteristic
SNVT_dev_maint
SCPTalrmClrT1#LO

and a second one with
Code:
text("%d %d:%d:%d:%d", day, hour, minute, second, millisecond);
text(("%d ", day), time(hour, minute, second, millisecond));
etc...

The first one cannot have space in it's name.
He can't start by !
should finish by :

The second one can by on multiple line
Must finish by ;
can be anything in there (visible char)

Well I thing the too to use is awk.

I try to figure the FS and the RS parameter.

I try to gigure out the regex for the FS, but I could'n find the right thing.

Here is the starting point that I have now.
Code:
#!/bin/awk -f


BEGIN {
# change the record separator from newline to nothing	
	RS=";"
# change the field separator from whitespace to newline
	FS=": "
	
	OFS="-----"
	
	ORS="======\r\n"
}
{
# print the second and third line of the file
	print $1, $2;
}

can someone can help me on this one ? also Is it the right tool

regards

Jonathan
Last edited by dumarjo; 03-22-2012 at 09:59 AM..
# 2  
Old 03-22-2012
Based on your description this is what I came up with:

Code:
awk '/[^ ]+:.*text\(.*\)/ { printf( "%s;", $0 ); }'  RS=';'  input-fiiiile >output-file

It doesn't attempt to join lines that have newlines embedded, and it assumes that the function you are trying to parse out always starts 'text('. It allows any character, except a space, in the first field before the column.

Hope this helps, and yes I think awk is a fine tool for this.
# 3  
Old 03-22-2012
Hi,

thank for the fast answer. But I think this doesn't give me the reseult expected.

This give me the complete line. I need 2 field at the end

I need it splited. like

$1 = SCPTactFbDly
$2 = text("%d %d:%d:%d:%d", day, hour, minute, second, millisecond);
.

Regards

Jonathan
# 4  
Old 03-22-2012
Ok, misunderstood.... this get you closer?

Code:
awk   '
    /[^ ]+:.*text\(.*\)/ {
        gsub( "\n\n", "\n");    # prevent blank lines from inserting lead spaces
        sub( "^\n", "");        # nix leading newline
        gsub( "\n", " ");       # join remaining lines
        a = $0;                 # first "segment" will end up in a
        b = $0;                 # second "segment" will end up in b
        sub( ":.*", "", a );
        sub( "[^ ]+:.*text", "text", b );
        printf( "%s\n", a );    # print each segment on its own line
        printf( "%s;\n", b );
    }
'  RS=';'  input-file >output-file

Last edited by agama; 03-22-2012 at 10:50 AM.. Reason: typo
# 5  
Old 03-22-2012
Hi,

Yes, this is really close. I just try it on my complete file, and I see something that I forget to mention. sometime, after the records, I have a comment that I need to keep.

like
Code:
SCPTreturnFanStaticPressureSetpoint#US: 	text("%f", *0.0040217+0(0:954)); 	! in. of H2O

righ now the ! in. of H2O is on a single line.. like this

Code:
 SCPTductStaticPressureLimit#US
 text("%f", *0.0040217+0(0:954));
 	! in. of H2O

Is it possible to keep this comment ?

I think not an easy task ?

Jonathan
# 6  
Old 03-22-2012
It's possible, but given that requirement I think it needs to be parsed a bit differently, so still awk, but a shift in gears:

Code:
awk '
    function printit( buffer,           a, b )
    {
        a = buffer;                 # first segment will end up in a
        b = buffer;                 # second segment will end up in b
        sub( ":.*", "", a );
        sub( "[^ ]+:.*text\\(", "text(", b );
        printf( "%s\n%s\n", a, b ); # print each segment on its own line
    }

    NF < 1 || /^!/ {next; }         # blank/comment lines -- just ditch

    /[^ ]+:.*text\(.*\);/ {     # full line
        printit( $0 );
        buffer = "";
        next;
    }

    /[^ ]+:.*text\(/ {          # partial line, start buffering
        buffer = $0;
        while( getline > 0  && ! index( $0, ";" ) )
            buffer = buffer " " $0;
        buffer = buffer " " $0;
        printit( buffer );
        next;
    }
'  input-file  >output-file

This User Gave Thanks to agama For This Post:
dumarjo
# 7  
Old 03-22-2012
Wow,

it's AWK is very flexible.

I undertand most of ! (that's not bad hey ? Smilie)

And it's works !

i'll really appreciate your help agama

regards

Jonathan

---------- Post updated at 02:09 PM ---------- Previous update was at 11:04 AM ----------

Hi,

I try to make this scritp usable with 2 or 3 files. (the same script) take the result of this 3 files and do some logic with it.

I just modified a littlebit to put a, and b in a array, and add END that display the array.

here something that I have so far

Code:
#!/bin/awk -f
function printit( buffer, file,          a, b )
{
	a = buffer;                 # first segment will end up in a
	b = buffer;                 # second segment will end up in b
	sub( ":.*", "", a );
	sub( "[^ ]+:.*text\\(", "text(", b );
#instead of printing the output, I stor it in an array with the cpName as Index
#	printf( "%s\n%s\n", a, b ); # print each segment on its own line
	file[a] = b;
}

	NF < 1 || /^!/ {next; }         # blank/comment lines -- just ditch

	/[^ ]+:.*text\(.*\);/ {     # full line
		printit( $0, standard);
		buffer = "";
		next;
	}

	/[^ ]+:.*text\(/ {          # partial line, start buffering
		buffer = $0;
		while( getline > 0  && ! index( $0, ";" ) )
			buffer = buffer " " $0;
		buffer = buffer " " $0;
		printit( buffer, stardard);
		next;
	}



END{
	printf("standard.fmt result\n");
	for ( i in standard)
	{
		if( i != "")
		{
			printf( "%s[%s]\n", i, standard[i]);
		}
	}

}

thanx
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pass awk field to a command line executed within awk

Hi, I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date). All my attempts failed this far. Here's an example. It works fine with timestamp hard-codded into the command echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies

2. Shell Programming and Scripting

Multiple command execution inside awk command during xml parsing

below is the output xml string from some other command and i will be parsing it using awk cat /tmp/alerts.xml <Alert id="10102" name="APP-DS-ds_ha-140018-componentFailure-S" alertDefinitionId="13982" resourceId="11427" ctime="1359453507621" fixed="false" reason="If Event/Log Level(ANY) and... (2 Replies)
Discussion started by: vivek d r
2 Replies

3. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

4. Shell Programming and Scripting

awk command in script gives error while same awk command at prompt runs fine: Why?

Hello all, Here is what my bash script does: sums number columns, saves the tot in new column, outputs if tot >= threshold val: > cat getnon0file.sh #!/bin/bash this="getnon0file.sh" USAGE=$this" InFile="xyz.38" Min="0.05" # awk '{sum=0; for(n=2; n<=NF; n++){sum+=$n};... (4 Replies)
Discussion started by: catalys
4 Replies

5. Shell Programming and Scripting

awk command for simple join command but based on 2 columns

input1 a_a a/a 10 100 a1 a_a 20 200 b1 b_b 30 300 input2 a_a a/a xxx yyy a1 a1 lll ppp b1 b_b kkk ooo output a_a a/a 10 100 xxx yyy (2 Replies)
Discussion started by: ruby_sgp
2 Replies

6. UNIX for Dummies Questions & Answers

Basic awk question...getting awk to act on $1 of the command itself

I have a script problem that I am not able to solve due my very limited understanding of unix/awk. This is the contents of test.sh awk '{print $1}' From the prompt if I enter: ./test.sh Hello World I would expect to see "Hello" but all I get is a blank line. Only then if I enter "Hello... (2 Replies)
Discussion started by: JasonHamm
2 Replies

7. Shell Programming and Scripting

awk/sed Command : Parse parameter file / send the lines to the ksh export command

Sorry for the duplicate thread this one is similar to the one in https://www.unix.com/shell-programming-scripting/88132-awk-sed-script-read-values-parameter-files.html#post302255121 Since there were no responses on the parent thread since it got resolved partially i thought to open the new... (4 Replies)
Discussion started by: rajan_san
4 Replies

8. Shell Programming and Scripting

assign a command line argument and a unix command to awk variables

Hi , I have a piece of code ...wherein I need to assign the following ... 1) A command line argument to a variable e.g origCount=ARGV 2) A unix command to a variable e.g result=`wc -l testFile.txt` in my awk shell script When I do this : print "origCount" origCount --> I get the... (0 Replies)
Discussion started by: sweta_doshi
0 Replies
Login or Register to Ask a Question