Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk {'FS="," ... vs. awk -F"," ...

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk {'FS="," ... vs. awk -F"," ...
# 1  
Old 12-03-2008
Question awk {'FS="," ... vs. awk -F"," ...
I used the following command
Code:
awk '{FS=","} $3>2007' $workfile5t > $workfile6t

But it appears to skip over the first record; even though field 3 is not > 2007, it is output.
However, the following appears to function properly
Code:
awk -F"," '$3>2007' $workfile5t > $workfile6t

Confused about the rule for the field separators.
# 2  
Old 12-03-2008
Code:
awk 'BEGIN {FS=","} $3>2007' $workfile5t > $workfile6t

# 3  
Old 12-03-2008
Question Where should BEGIN be placed if processing multiple files?
See this section of my larger program:
Code:
   awk -v wrk1=$workfile1ns -v wrk2x=$workfile2x -v wrk4=$workfile4 -v wrk5=$workfile5 -v planf=$workfile3 '
# put all plans in an array
      FILENAME==planf { FS=","
         ua1_pl[$1]=$2
         print "### 40a ## Set plan "$1" to "$2
         }
# put all jobs in an array
      FILENAME==wrk2x { FS=","
         j_num[$1]=$2
         j_yr[$1]=$3
         j_mon[$1]=$4
         j_month[$1]=$5
         print "### 40b ## Set job "$1"="$2" to month/year "$4"/"$3
         }
# put all the data together
      FILENAME==wrk1 { FS=","
# check on plan data
         if ( ua1_pl[$1] >"")
           { w_plan=ua1_pl[$1]
         } else
           { w_plan="***UA1 CODE UNDEFINED ["$1"]"
#           print "### 40c ## Undefined UA1 code "$1","$2
         }
# check on job number data
#         print $1,$2,j_num[$2],j_yr[$2]
         if ( j_num[$2] > "")
           { w_jnum=j_num[$2] ; w_jyr=j_yr[$2]
           w_jmon=j_mon[$2]
         }
# write out all the variables
         print $1","$2","w_plan","w_jnum","w_jyr","w_jmon > wrk4
         print ","w_jnum","w_jyr","w_jmon","w_plan > wrk5
   }' "$workfile3" "$workfile2x" "$workfile1ns"

I have to read through over a million records, and match up against a couple of smaller files. Thus, my thought to load the two smaller files into arrays and then read through the entire third file. I do array lookups on each record of the third file; thus creating one file containing all the important details.

In the example above, since I forced the FS to be "," I suppose I could simply change the awk -v ... to awk -F"," -v ... and be all set. But, what if the files had different delimiters? Where would I put the BEGIN before the FS="," commands?
# 4  
Old 12-03-2008
assuming ALL 3 files have different FieldSeparators - adjust the FS assignments in the 'BEGIN' accordingly:
Code:
   awk -v wrk1=$workfile1ns -v wrk2x=$workfile2x -v wrk4=$workfile4 -v wrk5=$workfile5 -v planf=$workfile3 '
      BEGIN {
         FSf1=","
         FSf2="|"
         FSf3="#"
      }

      FNR == 1 {
          if (FILENAME==planf) FS=FSf3
          if (FILENAME==wrk2x ) FS=FSf2
          if (FILENAME==wrk1 ) FS=FSf1
          $1=$1
      }
# put all plans in an array
      FILENAME==planf { 
         ua1_pl[$1]=$2
         print "### 40a ## Set plan "$1" to "$2
         }
# put all jobs in an array
      FILENAME==wrk2x { 
         j_num[$1]=$2
         j_yr[$1]=$3
         j_mon[$1]=$4
         j_month[$1]=$5
         print "### 40b ## Set job "$1"="$2" to month/year "$4"/"$3
         }
# put all the data together
      FILENAME==wrk1 { 
# check on plan data
         if ( ua1_pl[$1] >"")
           { w_plan=ua1_pl[$1]
         } else
           { w_plan="***UA1 CODE UNDEFINED ["$1"]"
#           print "### 40c ## Undefined UA1 code "$1","$2
         }
# check on job number data
#         print $1,$2,j_num[$2],j_yr[$2]
         if ( j_num[$2] > "")
           { w_jnum=j_num[$2] ; w_jyr=j_yr[$2]
           w_jmon=j_mon[$2]
         }
# write out all the variables
         print $1","$2","w_plan","w_jnum","w_jyr","w_jmon > wrk4
         print ","w_jnum","w_jyr","w_jmon","w_plan > wrk5
   }' "$workfile3" "$workfile2x" "$workfile1ns"

# 5  
Old 12-03-2008
Joeyg,

If you change the field separator within the code, you have to rearrange the line with $0=$0.

Regards
# 6  
Old 12-03-2008
Question
vgersh99 - thanks, that makes sense

Franklin52 - what is meant by your instruction $0=$0?
The solution proposed by vgersh99 has a $1=$1.
# 7  
Old 12-03-2008
If you change the field separator within the code, the the command is read after the first record is read.
The command $0=$0 or $=$1 is to rearrange the fields with the new field separator in the buffer ($0).

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

3. Shell Programming and Scripting

Awk,sed : change every 2nd field ":" to "|"

Hi Experts, I have a string with colon delimited, want 2nd colon to be changed to a pipe. data: 101:8:43:4:72:14:41:69:85:3:137:4:3:0:4:0:9:3:0:3:12:3: I am trying with sed, but can change only 1 occurance: echo "101:8:43:4:72:14:41:69:85:3:137:4:3:0:4:0:9:3:0:3:12:3:" | sed 's/:/|/2'... (5 Replies)
Discussion started by: rveri
5 Replies

4. Post Here to Contact Site Administrators and Moderators

Suggestion: adding two new groups "sed" and "awk"

Majority of the questions are pertaining file/string parsing w.r.t sed or awk It would be nice to have these two as their own sub category under shell-programming-scripting which can avoid lot of duplicate posts. (1 Reply)
Discussion started by: jville
1 Replies

5. Shell Programming and Scripting

how to use "cut" or "awk" or "sed" to remove a string

logs: "/home/abc/public_html/index.php" "/home/abc/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" "/home/xyz/public_html/index.php" how to use "cut" or "awk" or "sed" to get the following result: abc abc xyz xyz xyz (8 Replies)
Discussion started by: timmywong
8 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. Shell Programming and Scripting

Simplify Bash Script Using "sed" Or "awk"

Input file: 2 aux003.net3.com error12 6 awn0117.net1.com error13 84 aux008 error14 29 aux001.ha.ux.isd.com error12 209 aux002.vm.ux.isd.com error34 21 alx0027.vm.net2.com error12 227 dux001.net5.com error123 22 us008.dot.net2.com error121 13 us009.net2.com error129Expected Output: 2... (4 Replies)
Discussion started by: sQew
4 Replies

8. Shell Programming and Scripting

cat $como_file | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g'

hi All, cat file_name | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g' Can this be done by using sed or awk alone (4 Replies)
Discussion started by: harshakusam
4 Replies

9. Shell Programming and Scripting

help for saving vertical datas to horizontal with "awk" or "cut"

hi, i have a file having datas like that ./a.txt 12344 12345 12346 12347 ..... ..... ... i want to save this datas to another file like that ./b.txt 12344 12345 12346 12347 ... ... ... i think awk can make this but how? :) waiting for ur help. (3 Replies)
Discussion started by: mercury
3 Replies

10. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question