passing a regex as variable to awk and using that as regular expression for search
Hi All,
I have a sftp session log where I am transferring multi files by issuing "mput abc*.dat". The contents of the logfile is below -
#################################################
#################################################
This log has been captured in a file called sftp_log. Now I am out of the sftp session and I have this sftp_log for my referance. I want to check the log and find out if all the files (resembling abc*.dat) are transferred as per size. So, I want to find the lines where the abc*.dat files were long listed. I have the abc*.dat captured in a variable named TRANSFERRING_FNAME. So I used this variable to be passed in awk and search the desired lines by below command -
But it is not returning anything. Actually I needed only below lines from the sftp_log -
###################################################
###################################################
From these lines I want to extract the file name and their corresponding size like below -
############################################
############################################
And the value in $TRANSFERRING_FNAME can vary so we can't manupulate on hard coded value like 'abc'. Could anyone please advise.
Thanks & Regards,
Bijitesh
Last edited by Franklin52; 02-17-2012 at 03:30 AM..
Reason: Please use code tags for code and data samples, thank you
What are the exact contents of this TRANSFERRING_FNAME? abc*.dat is not regular expression, it's a glob -- try to interpret it as a regex and it will work wrong. It will match abc.dat, abcadat, abcccc.dat, abccccccadat, and so forth, since * means "one or more of the previous character" and . means "any single character".
Thanks Corona688 for correcting me. I am not good at regex and thought abc*.dat as regex. But TRANSFERRING_FNAME variable contains abc*.dat only. Actually it is taken from the understanding of working "ls -lrt abc*.dat" which means all files - abc.dat, abca.dat, abcccc.dat, abcccccca.dat, abc140212095613.dat, abc140212095639.dat, abc14022012.dat etc. The script first does multiple put in a sftp session as "mput $TRANSFERRING_FNAME". And then from this clue, i.e. $TRANSFERRING_FNAME I have to search the sftp_log. And it is obvious that I can't modify the contents of $TRANSFERRING_FNAME as '^abc.*\.dat'. Please help.
That's a slight fallacy known as useless use of ls *, it's not ls that understands what * means -- it's the shell itself. That's why * works the same way with every command.
mput is independent of the shell however, and does its own processing of *, but it works the same way as the shell.
You're going to need to modify your shell script I think. Perhaps you can tell it the prefix you want, instead of giving it the entire "abc*.dat", and the shell can replace that itself, and you can feed that into awk in the form you want as well.
To modify your shell script I'll need to actually see it of course.
---------- Post updated at 10:01 AM ---------- Previous update was at 09:55 AM ----------
This kludge might also work, but it's not pretty. It replaces all . by \., all * by .*, prepends ^, and appends $ to turn simple globs into regular expressions. If your awk doesn't have gsub, use nawk.
Thanks a lot Corona688 for this valuable advise. I'll check if this is possible. One thing is that the TRANSFERRING_FNAME would be assigned with a value that the users passes as the argument to the shell script. So it can contain anything starting from absolute value abc140212095639.dat to abc*.dat or abc?.dat.
Hi All,
I would like to search a regular expression by passing as an i/p variableto AWK.
For Example ::
162.111.101.209.9516
162.111.101.209.41891
162.111.101.209.9516
162.111.101.209.9517
162.111.101.209.41918
162.111.101.209.9517
162.111.101.209.41937
162.111.101.209.41951... (7 Replies)
I am trying to test if I can replace a regular expression in 'nawk' with a variable.
Please let me know why it is not working. I am using ksh88i on solaris8
I am trying use this test as a building block to filter active external DNS connections.
Ideally I want to pass variable defined... (4 Replies)
Hi,
Server - MEDIAWIKI - MYSQL - CENTOS 5 - PHP5
I have a database import of close to a million pages into my wiki, mediawiki site,
the format that were left with is not pretty, and I need to find a way to clean this up and present it nicely...
I think regex is the best option as I can... (1 Reply)
Hello,
Awk seem treat the pattern as regular expression, how can awk search not using regular expression? e.g. just represent for "", not "A" or "a" . I don't want to add backslash . (2 Replies)
Hi All,
Below is a sample code:
print "Enter the Name: ";
my $Name = <>;
print "Word is $Name";
open (FH,"AIDNameList.txt");
while (<FH>)
{
my $line;
print "Word is $Name";
for $line(<FH>)... (12 Replies)
Hi ,
I have a file with "|" (pipe) as a delimeter. I am looking for the record count where 5th field is a number with 15 digit length only.
all the records with above requirement is valid rest all are invalid. I need count of valid records and invalid records.
Can anyone please help (9 Replies)
I have a large file encoded in Unicode that I need to convert to CSV. In general, I know how to do this by regular expression substitutions using sed or Perl, but one problem I am having is that I need to put a quotation mark at the end of each line to protect the last field. The usual regex... (1 Reply)
I want to block all special characters except alphanumerics.. and "."(dot ) character
currently am using //
I want to even block only single dot or multiple dots..
ex:
. or .............. should be blocked.
please provide me the reg ex.
---------- Post updated at 05:11 AM... (10 Replies)
Hi,
I have this script:
awk -v va=45 '$0~va{print}' flo2
That returns: "4526745 1234 " (this is the only line of the file "flo2".
However, I would like to get "va" to match the begining of the line, so that is "va" is different than 45 (eg. 67, 12 ...) I would not have any output. That... (3 Replies)
Anyone know how I will use awk's variable in a regular expression?
This line of code of mine is working, the value PREMS should be a variable:
awk '$1 ~ /PREMS/ { if(length(appldata)+2 >= length($1)) print $0; }' appldata=$APPLDATA /tmp/file.tmp
The value of APPLDATA variable is PREMS.
... (2 Replies)