Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Multiple bash variables passed into nawk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multiple bash variables passed into nawk
# 1  
Old 02-12-2015
Multiple bash variables passed into nawk
I have a file that has 2 fields called b_file:

Code:
11977 DAR.V3.20150209.1.CSV
3295 DAR.V3.20150209.1.CSV
1721 DAR.V2.20150210.1.CSV

I need to search a sftplog using the field 1, but want to maintain the relationship between field 1 and 2. I am passing field 1 as a parameter in a bash loop.

Code:
for i in `cat |nawk '{ print $0 }' /tmp/b_file`
do

I am searching a sftplog for the ID for the user, but this is only giving me the field after "user".

Code:
nawk -v st=$i '$5 ~ st && /closed/ && /user/ && !/ORG/  && !/LE/ { for (x=1;x<=NF;x++) if ($x~"user") print $(x+1) } ' /var/adm/sftplog >> /tmp/dar3.out

I want the output in /tmp/dar3.out to be
Code:
tom DAR.V3.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV
steve DAR.V2.20150210.1.CSV

where tom was 11977, charlie was 3295, and steve was 1721.

complete code:
Code:
for i in `cat |nawk '{ print $0 }' /tmp/b_file`
do
nawk -v st=$i '$5 ~ st && /closed/ && /user/ && !/ORG/  && !/LE/ { for (x=1;x<=NF;x++) if ($x~"user") print $(x+1) } ' /var/adm/sftplog >> /tmp/dar3.out
done

Open to alternative methods if recommended. Any assistance is much appreciated.
Last edited by Scrutinizer; 02-12-2015 at 01:16 PM.. Reason: additional code tags for data samples
# 2  
Old 02-12-2015
You have shown the output you want, now show the input you have please.
# 3  
Old 02-12-2015
Hello smenago,

You was not clear about requirement, based on my assumptions which are (You have 2 files named b_file and c_file as follows.)
Code:
cat b_file
11977 DAR.V3.20150209.1.CSV
3295 DAR.V3.20150209.1.CSV
1721 DAR.V2.20150210.1.CSV
 
cat c_file
tom 11977
charlie 3295
steve 1721

Then as per your expected output I can suggest following.
Code:
awk 'FNR==NR{A[$1]=$2;next} ($2 in A){print $1 OFS A[$2]}' b_file c_file

Output will be as follows.
Code:
tom DAR.V3.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV
steve DAR.V2.20150210.1.CSV

If you are happy with above awk command then you can redirect it's output to a file named > /tmp/dar3.out.

Hope this helps, if my assumptions are not correct, please do let us know the expected output with os details.


NOTE: On Solaris/SunOS systems, you need to change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk


EDIT: Sorry Corona, seems we have posted on same time, please do remove this post if needed until OP comes back with answer.


Thanks,
R. Singh
Last edited by RavinderSingh13; 02-12-2015 at 12:18 PM.. Reason: Adding comment for Corona
# 4  
Old 02-12-2015
The input from the sftplog looks like this

Code:
Feb 11 12:01:46 ftp02 sftp-server[11977]: [ID 800047 local1.info] session closed for local user tom from [65.200.105.28]
Feb 11 20:49:57 ftp02 sftp-server[3295]: [ID 800047 local1.info] session closed for local user charlie from [65.200.105.28]

Last edited by Scrutinizer; 02-12-2015 at 01:14 PM.. Reason: additional code tags for data
# 5  
Old 02-12-2015
Bash/shell is even enough:
Code:
while read uNo fileNm
do
 echo "$( <procedure to translate $uNo to user>) $fileNm
done <in_file >out_file

But if the user is in file c, just join:
Code:
sort -u -o c c
sort -u -o b b
join c b | cut -f 2,3

I think you can adjust the output of join with arguments and avoid the cut. See the man page. You can also join with any tool with associative arrays (hash maps), like ksh, bash, awk; putting the smaller set into an array and the looking up the larger one line by line.
# 6  
Old 02-12-2015
Hello smenago,

Could you please try following and let me know if this helps.
Code:
awk 'FNR==NR{A[$1]=$2;next} ($2 in A){sub(/.*user /,X,$5);sub(/ from/,X,$5);print $5 OFS A[$2]}' b_file FS="[][]" c_file

Output will be as follows.
Code:
tom  DAR.V3.20150209.1.CSV
charlie  DAR.V3.20150209.1.CSV


Thanks,
R. Singh
# 7  
Old 02-13-2015
Hi R.Singh,

I tried the following, but received a lot more of the content of the source file.

Code:
/usr/xpg4/bin/awk 'FNR==NR{A[$1]=$2;next} ($2 in A){sub(/.*user /,X,$5);sub(/ from/,X,$5);print $5 OFS A[$2]}' b_file FS="[][]" /var/adm/sftplog

Here is a sample of what was received.

Code:
brcp  DAR.V3.20150209.1.CSV
 received client version 3 DAR.V3.20150209.1.CSV
 realpath "." DAR.V3.20150209.1.CSV
 opendir "outbound/" DAR.V3.20150209.1.CSV
 closedir "outbound/" DAR.V3.20150209.1.CSV
 stat name "outbound/DAR.V3.20150209.1.CSV" DAR.V3.20150209.1.CSV
 open "outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666 DAR.V3.20150209.1.CSV
 close "outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0 DAR.V3.20150209.1.CSV
 stat name "outbound/DDR.V2.20150209.1.CSV" DAR.V3.20150209.1.CSV
 open "outbound/DDR.V2.20150209.1.CSV" flags READ mode 0666 DAR.V3.20150209.1.CSV
 close "outbound/DDR.V2.20150209.1.CSV" bytes read 16393726 written 0 DAR.V3.20150209.1.CSV
brcp  DAR.V3.20150209.1.CSV
jpmcc  DAR.V3.20150209.1.CSV
 received client version 3 DAR.V3.20150209.1.CSV
 realpath "." DAR.V3.20150209.1.CSV
 realpath "/outbound" DAR.V3.20150209.1.CSV
 stat name "/outbound" DAR.V3.20150209.1.CSV
 stat name "/outbound/." DAR.V3.20150209.1.CSV
 opendir "/outbound/." DAR.V3.20150209.1.CSV
 closedir "/outbound/." DAR.V3.20150209.1.CSV
 open "/outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666 DAR.V3.20150209.1.CSV
 close "/outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0 DAR.V3.20150209.1.CSV
jpmcc  DAR.V3.20150209.1.CSV
ORG116  DAR.V2.20150210.1.CSV
 received client version 3 DAR.V2.20150210.1.CSV
 realpath "." DAR.V2.20150210.1.CSV

My original had this to isolate the the specific record on the file. Can I include this anywhere to get the record I need?

Code:
$5 ~ st && /closed/ && /user/ && !/ORG/  && !/LE/

---------- Post updated 02-13-15 at 12:04 PM ---------- Previous update was 02-12-15 at 12:12 PM ----------

Hi R.Singh,
The file that contains the records to be evaluated in an another file is

Code:
ftp01$ less b_file
11977 DAR.V3.20150209.1.CSV
3295 DAR.V3.20150209.1.CSV

The datafile that needs to be evaluated to extract the records needed is formatted like so. Please let me know if you want me to provide in an attachment.

Code:
mdadmin@ftp01$ grep 11977 /var/adm/sftplog
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] session opened for local user tom from [111.11.11.111]
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] received client version 3
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] realpath "."
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] opendir "outbound/"
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] closedir "outbound/"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DAR.V3.20150209.1.CSV"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DDR.V2.20150209.1.CSV"
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DDR.V2.20150209.1.CSV" flags READ mode 0666
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DDR.V2.20150209.1.CSV" bytes read 16393726 written 0
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] session closed for local user tom from [111.11.11.111]

mdadmin@ftp01$ grep 3295 /var/adm/sftplog
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] session opened for local user charlie from [111.11.11.111]
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] received client version 3
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] opendir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] closedir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] open "/outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] close "/outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] session closed for local user charlie from [111.11.11.111]

Just to confirm, I would like to maintain the relationship between the two fields in b_file and print in a new file.

Code:
tom DAR.V3.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV

Last edited by Scrutinizer; 02-12-2015 at 01:14 PM.. Reason: Additional code tags for data sample
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Multiple variables to be passed in a loop

Hi, I need to pass the multiple values of src1 to another variable. I managed to print it but not sure how to assign it to a variable in a loop. src1=01,02,03 echo $src1|awk 'BEGIN {FS=","} {for(i=1;i<=NF;i++) print $i}' I need to pass the value as src2=01 src2=02 src2=03 Thanks... (4 Replies)
Discussion started by: shash
4 Replies

2. Shell Programming and Scripting

Shell Variables passed to awk to return certain rows

Hi Forum. I have the following test.txt file and need to extract certain rows based on "starting position", "length of string" and "string to search for": 1a2b3d 2a3c4d ..... My script accepts 3 parameters: (starting col pos, length to search for, string to search for) and would like to... (4 Replies)
Discussion started by: pchang
4 Replies

3. Shell Programming and Scripting

awk processing of passed variables

Currently have this: set current=192.168.0.5 set servicehost = `echo $current | awk -F. '{print $4}'` echo $numberoffields 5 ..but would like to reduce # of variables and eliminate echo to have something like this: set servicehost = `awk -v s="$current" -F. 'BEGIN{print $2}'`But... (3 Replies)
Discussion started by: Mid Ocean
3 Replies

4. Shell Programming and Scripting

Multiple Variables for BASH script

Hello, I am new to the whole "scripting" thing. Below is the script that I have so far and where i need the Variables to go (VAR#) #!/bin/bash #Sample Script VAR1= echo "Choose an option: 1) Create a file. 2) Delete a file. 3) Move a file." read VAR1 case $VAR1 in 1) echo "Pick... (4 Replies)
Discussion started by: eclerget
4 Replies

5. Shell Programming and Scripting

nawk variables

Hi - The following nawk is not working and trying to understand why! nawk -v t="internal_order" '/SAP_RM_ADMIN_SCHEMA/ && ("" toupper(t)) || /SAP_RM_ADMIN_SCHEMA/ && ("" tolower(t))' PBFD100.ksh My intention is to retrieve the line containing SAP_RM_ADMIN_SCHEMA.internal_order but its just not... (6 Replies)
Discussion started by: anduzzi
6 Replies

6. Shell Programming and Scripting

How to call multiple variables in bash !!

Hi all , I have a file with below data , bash#cat file.txt user1 amount1 status1 user2 amount2 status2 user3 amount3 status3 user4 amount4 status4 . . . Now i have a command to be executed with above values like below , ./errorcheck -u user1 -a amount1 -s status1 ... (3 Replies)
Discussion started by: gnanasekar_beem
3 Replies

7. Shell Programming and Scripting

Can I use nawk -v with two or more variables?

Hi, Is it possible in awk/nawk to pass two or more variables in the -v flag? That is: X=1 Y=2 nawk -v X=$X Y=$Y..... Thanks in advance. (7 Replies)
Discussion started by: daytripper1021
7 Replies

8. Shell Programming and Scripting

AWK: Retrieving names of variables passed with -v

I'm an experienced awk user, but this one has me stumped. I have an awk script which is called from a UNIX command line as you'd expect: myscript.awk -v foo=$1 -v bar=$2 filename My question is this: is there a mechanism for determining the names of the -v variables within a script? ... (3 Replies)
Discussion started by: John Mac
3 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. Shell Programming and Scripting

nawk and variables

Hi guy's Im trying to pass variables into nawk and then match them on a value within a record but it don't seem to be working. If i put in the dates i want to see then it works fine.. #!/usr/bin/ksh -x YEST=$(/usr/local/bin/perl -e... (8 Replies)
Discussion started by: plimpix
8 Replies
Login or Register to Ask a Question