Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Multiple bash variables passed into nawk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multiple bash variables passed into nawk
# 8  
Old 02-15-2015
Hi all,
Any assistance or suggestions on the above is much appreciated. I have read more and tried different code, but cannot obtain the desired result.

Thanks.
# 9  
Old 02-15-2015
As much as I would like to help - I can't. Neither the verbal specification nor the data samples nor the code snippets given - all spread over several posts - give me a chance to understand what you're after.

Mayhap a step back might help; start over with a new, precise, careful specification seconded by accurate data samples, both input and desired output, and improved code snippets that would really work...?
# 10  
Old 02-15-2015
From the file content below which is from a sftplog I need extract the user and the file name into a file. I would like to use nawk to do so, but am open to any other menthods. The only way to maintain the relationship between user and file is by the session ID which is in brackets.
  1. The user is identified after the word
    Code:
    user

    .
  2. The file appears after the string
    Code:
    close "/outbound/

    or
    Code:
    close "outbound/

  3. The session IDs below are
    Code:
    11977

    and
    Code:
    3295

Code:
mdadmin@ftp01$ grep 11977 /var/adm/sftplog
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] session opened for local user tom from [111.11.11.111]
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] received client version 3
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] realpath "."
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] opendir "outbound/"
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] closedir "outbound/"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DAR.V3.20150209.1.CSV"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DDR.V2.20150209.1.CSV"
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DDR.V2.20150209.1.CSV" flags READ mode 0666
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DDR.V2.20150209.1.CSV" bytes read 16393726 written 0
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] session closed for local user tom from [111.11.11.111]

mdadmin@ftp01$ grep 3295 /var/adm/sftplog
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] session opened for local user charlie from [111.11.11.111]
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] received client version 3
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] opendir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] closedir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] open "/outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] close "/outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] session closed for local user charlie from [111.11.11.111]

The desired output is

Code:
tom DAR.V3.20150209.1.CSV
tom DDR.V2.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV

# 11  
Old 02-16-2015
You could try something like this:
Code:
awk '/ session .* user /{u=$(NF-2)} / close /{m=split($0,F,/["\/]/); print u, F[m-1]}' logfile

But this requirement seeds to be different from the one in post #1
# 12  
Old 02-18-2015
Yes. I tried to simply my issue by putting the session ID and file name in a file and then using the contents of that file to extract the user name for the session ID. I was then struggling to use an array across two files to make sure the user was attributed to the correct file name. In the process I confused everyone so I eliminated the first step.



Unfortunately, The above suggestion is not producing all of the file names.



This is the closest I have come.


1. Create a file with session ID and file_name -

---------- Post updated at 10:09 AM ---------- Previous update was at 09:59 AM ----------

This is the closest I have come.


1. Create a file with session ID and file_name -

Code:
ftp02$ less /tmp/b_file
11977 DAR.V3.20150209.1.CSV
11977 DDR.V2.20150209.1.CSV
3295 DAR.V3.20150209.1.CSV

2. Loop through isolated record to get user id

Code:
for i in `cat |nawk '{ print $1 }' /tmp/b_file`
do
nawk -v st=$i '$5 ~ st && /closed/ && /user/ && !/ORG/  && !/LE/ { for (x=1;x<=NF;x++) if ($x~"user") print $(x+1) } ' /var/adm/sftplog >> /tmp/log.out
done

3. Here is the sample of sftlog.
Code:
mdadmin@ftp01$ grep 11977 /var/adm/sftplog
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] session opened for local user tom from [111.11.11.111]
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] received client version 3
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] realpath "."
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] opendir "outbound/"
Feb 11 04:29:13 ftp01 sftp-server[11977]: [ID 800047 local1.info] closedir "outbound/"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DAR.V3.20150209.1.CSV"
Feb 11 04:29:14 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] stat name "outbound/DDR.V2.20150209.1.CSV"
Feb 11 04:30:00 ftp01 sftp-server[11977]: [ID 800047 local1.info] open "outbound/DDR.V2.20150209.1.CSV" flags READ mode 0666
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] close "outbound/DDR.V2.20150209.1.CSV" bytes read 16393726 written 0
Feb 11 04:31:27 ftp01 sftp-server[11977]: [ID 800047 local1.info] session closed for local user tom from [111.11.11.111]

mdadmin@ftp01$ grep 3295 /var/adm/sftplog
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] session opened for local user charlie from [111.11.11.111]
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] received client version 3
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] realpath "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound"
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] stat name "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] opendir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] closedir "/outbound/."
Feb 11 05:14:11 ftp01 sftp-server[3295]: [ID 800047 local1.info] open "/outbound/DAR.V3.20150209.1.CSV" flags READ mode 0666
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] close "/outbound/DAR.V3.20150209.1.CSV" bytes read 9627450 written 0
Feb 11 05:14:22 ftp01 sftp-server[3295]: [ID 800047 local1.info] session closed for local user charlie from [111.11.11.111]

3. With the above the contents of log.out will be
Code:
tom
tom
charlie

I want it to be -
Code:
tom DAR.V3.20150209.1.CSV
tom DDR.V2.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV

How can I get the file_name or field $2 lncluded in log.out?

---------- Post updated 02-18-15 at 09:56 AM ---------- Previous update was 02-17-15 at 10:09 AM ----------

Hi all,
Is this clear, or am I still confusing everyone/anyone? I realize I am potentially annoying everyone with this problem. Any assistance is tremendously appreciated.

Sean
# 13  
Old 02-18-2015
Try

Code:
akshay@Aix:/tmp$ cat parse.awk
FNR==NR{
	a[$1]= $1 in a ? a[$1] SUBSEP $2 : $2
	next
}
{
	for( i in a )
	{ 
	 if( $5 == "sftp-server["i"]:" && /closed/ && /user/ && !/ORG/ && !/LE/ )
	 {
		split(a[i],A,SUBSEP)
		for(j in A)
		{
			print $14,A[j]
		} 
	 }
	}
}

Code:
akshay@Aix:/tmp$ awk -f parse.awk b_file sftp_log 
tom DAR.V3.20150209.1.CSV
tom DDR.V2.20150209.1.CSV
charlie DAR.V3.20150209.1.CSV

This User Gave Thanks to Akshay Hegde For This Post:
smenago
# 14  
Old 02-18-2015
Solved
Akshay,
Thanks so much!! This is solved.
Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Multiple variables to be passed in a loop

Hi, I need to pass the multiple values of src1 to another variable. I managed to print it but not sure how to assign it to a variable in a loop. src1=01,02,03 echo $src1|awk 'BEGIN {FS=","} {for(i=1;i<=NF;i++) print $i}' I need to pass the value as src2=01 src2=02 src2=03 Thanks... (4 Replies)
Discussion started by: shash
4 Replies

2. Shell Programming and Scripting

Shell Variables passed to awk to return certain rows

Hi Forum. I have the following test.txt file and need to extract certain rows based on "starting position", "length of string" and "string to search for": 1a2b3d 2a3c4d ..... My script accepts 3 parameters: (starting col pos, length to search for, string to search for) and would like to... (4 Replies)
Discussion started by: pchang
4 Replies

3. Shell Programming and Scripting

awk processing of passed variables

Currently have this: set current=192.168.0.5 set servicehost = `echo $current | awk -F. '{print $4}'` echo $numberoffields 5 ..but would like to reduce # of variables and eliminate echo to have something like this: set servicehost = `awk -v s="$current" -F. 'BEGIN{print $2}'`But... (3 Replies)
Discussion started by: Mid Ocean
3 Replies

4. Shell Programming and Scripting

Multiple Variables for BASH script

Hello, I am new to the whole "scripting" thing. Below is the script that I have so far and where i need the Variables to go (VAR#) #!/bin/bash #Sample Script VAR1= echo "Choose an option: 1) Create a file. 2) Delete a file. 3) Move a file." read VAR1 case $VAR1 in 1) echo "Pick... (4 Replies)
Discussion started by: eclerget
4 Replies

5. Shell Programming and Scripting

nawk variables

Hi - The following nawk is not working and trying to understand why! nawk -v t="internal_order" '/SAP_RM_ADMIN_SCHEMA/ && ("" toupper(t)) || /SAP_RM_ADMIN_SCHEMA/ && ("" tolower(t))' PBFD100.ksh My intention is to retrieve the line containing SAP_RM_ADMIN_SCHEMA.internal_order but its just not... (6 Replies)
Discussion started by: anduzzi
6 Replies

6. Shell Programming and Scripting

How to call multiple variables in bash !!

Hi all , I have a file with below data , bash#cat file.txt user1 amount1 status1 user2 amount2 status2 user3 amount3 status3 user4 amount4 status4 . . . Now i have a command to be executed with above values like below , ./errorcheck -u user1 -a amount1 -s status1 ... (3 Replies)
Discussion started by: gnanasekar_beem
3 Replies

7. Shell Programming and Scripting

Can I use nawk -v with two or more variables?

Hi, Is it possible in awk/nawk to pass two or more variables in the -v flag? That is: X=1 Y=2 nawk -v X=$X Y=$Y..... Thanks in advance. (7 Replies)
Discussion started by: daytripper1021
7 Replies

8. Shell Programming and Scripting

AWK: Retrieving names of variables passed with -v

I'm an experienced awk user, but this one has me stumped. I have an awk script which is called from a UNIX command line as you'd expect: myscript.awk -v foo=$1 -v bar=$2 filename My question is this: is there a mechanism for determining the names of the -v variables within a script? ... (3 Replies)
Discussion started by: John Mac
3 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. Shell Programming and Scripting

nawk and variables

Hi guy's Im trying to pass variables into nawk and then match them on a value within a record but it don't seem to be working. If i put in the dates i want to see then it works fine.. #!/usr/bin/ksh -x YEST=$(/usr/local/bin/perl -e... (8 Replies)
Discussion started by: plimpix
8 Replies
Login or Register to Ask a Question