dynamically change awk Field Separator FS


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting dynamically change awk Field Separator FS
# 1  
Old 08-24-2009
dynamically change awk Field Separator FS

Hi All,

I was wondering if anyone knew how to dynamically change the FS in awk to accept vairiable containing a field separator. the current code is as below and does not work when i introduce the dynamic FS change :-(

validate_source_file()
{
source_file=$1
datetimestamp=`date +"%Y%m%d%H%M%S"`
invalid_file=$2_$datetimestamp.dat
field_count=$3
file_delimeter=$4


#awk 'BEGIN{ FS="|"; } {
awk 'BEGIN{ FS=file_delimeter; } {
if ( NF != field_count ) {
invalid_rec_count+=1 ;
#
# print the header for the bad file once....then start printing the invalid records to the same file...
#
print $0 > invalid_file ;

}
else {
valid_rec_count+=1 ;
}
}
END { printf "%-10s\t\n%-10s\t\n%-10s\t\n","Total Lines Read ["NR"]","Valid Records Found ["valid_rec_count"]","Invalid Records Found ["invalid_rec_count"]" ;

if (invalid_rec_count>1) {
print " "
print "Please refer to file "invalid_file" for full invalid record set."
}
}
' invalid_file="$invalid_file" field_count="$field_count" file_delimiter="$file_delimeter" $source_file
}


so when I call it i would want to pass a 4th parameter like:

validate_source_file "$filenamedir" "$source_tabin_name" "$field_count" ","

or

validate_source_file "$filenamedir" "$source_tabin_name" "$field_count" "|"


Any ideas appreciated!

Regards
Satnam
# 2  
Old 08-24-2009
You have to pass variables into awk with -v

Code:
awk -v fdelim=$file_delimeter 'BEGIN{ FS=fdelim; } { 
        if ( NF != field_count ) {
< snip >

You can probably do it with -F as well and just get rid of the BEGIN statement:

Code:
awk -F $file_delimeter '{ 
        if ( NF != field_count ) {

# 3  
Old 08-24-2009
Works a treat!

Top stuff..solution well fine


cheers for responding! :-)

regards
Satnam
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk field separator not working

Hi, can some some help to get me the right results, I have few text files, need to grep few columns from each file and get the results in one row with comma separated. my code is #folder=/nz/kit/log/backupsvr folder=/export/home/nz/valai/tmpfiles/ echo $folder for entry in `ls... (4 Replies)
Discussion started by: ValaiG
4 Replies

2. Shell Programming and Scripting

awk field separator help -

Hi Experts , file : - How to construct the awk filed separator so that $1, $2 $3 , can be assigned to the each "" range. I am trying : awk -F"]" '{print $1}' but it is printing the entire file. Not first field. The desired output needed for first field... (9 Replies)
Discussion started by: rveri
9 Replies

3. Shell Programming and Scripting

awk field separator

I need to set awk field separator to ";", but I need to avoid ";EXT". so that echo a;b;c;EXTd;e;f | awk -F";" '{print $3}' would give "c;EXTd" (2 Replies)
Discussion started by: locoroco
2 Replies

4. Shell Programming and Scripting

awk - show field separator

I am using this code to insert something into a csv file: awk -F";" -v url=$url -v nr=$nr 'NR==nr{$2=url$2}1' file Why do I get the output field1 field2 instead of field1;field2 I have given -F";", so the field separator should surely be ";". (1 Reply)
Discussion started by: locoroco
1 Replies

5. UNIX for Dummies Questions & Answers

change field separator only from nth field until NF

Hi ! input: 111|222|333|aaa|bbb|ccc 999|888|777|nnn|kkk 444|666|555|eee|ttt|ooo|ppp With awk, I am trying to change the FS "|" to "; " only from the 4th field until the end (the number of fields vary between records). In order to get: 111|222|333|aaa; bbb; ccc 999|888|777|nnn; kkk... (1 Reply)
Discussion started by: beca123456
1 Replies

6. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

7. Shell Programming and Scripting

awk (nawk) field separator

Hi; i have a file and i want to get; - If the last word in line 14 is NOT equal to "Set."; then print 2nd, 3rd, 4th and 5th values of 3rd line. and my code is: nawk 'NR==14 {if ($NF!="Set.") (NR==3{print $2,$3,$4,$5}) }' file.txt but no result?? :confused::(:confused::( (4 Replies)
Discussion started by: gc_sw
4 Replies

8. Shell Programming and Scripting

Field separator in awk

Hi I need to check if field separator I am using in awk statement is " : ", for example: TIME=12:59 HOUR=`echo "$TIME" | awk '{FS=":"; print $1}'` MINUTES=`echo "$TIME" | awk '{FS=":"; print $2}'` Is there a way to check within the above awk statement ? Thanks for help -A (2 Replies)
Discussion started by: aoussenko
2 Replies

9. UNIX for Dummies Questions & Answers

Can't figure out what field separator to use in awk....

Hi Friends, Scripting newb here. So I'm trying to create a geektool script that uses awk and printf to output certain fields from top (namely command, cpu%, rsize, pid and time, in that order). Here's the input from the top process that I'm putting into awk: PID COMMAND %CPU ... (3 Replies)
Discussion started by: thom.mattson
3 Replies

10. UNIX for Dummies Questions & Answers

Change field separator of grep from : to space

Hi, All, I wonder how to change the field separator of grep from : to space when use grep to display. for example when I use grep -e 'pattern1' -e 'pattern2' -n filename to find patterns, it use : to separate patterns, but I want to use blank space. is there an option I can set to... (2 Replies)
Discussion started by: Jenny.palmy
2 Replies
Login or Register to Ask a Question