Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Make awk gsub take value of for loop
Top Forums Shell Programming and Scripting Make awk gsub take value of for loop Post 303014440 by bedtime on Monday 12th of March 2018 12:47:24 PM
Old 03-12-2018
Quote:
Originally Posted by RudiC
This is my personal feeling: It is well known that any dedicated compiled program, be it C, C++, Pascal, or other, usually benefit from increased execution speed compared to e.g. scripts. But there is a tradeoff in terms of flexibility vs. e.g. awk, perl esp. when it comes to text analysis and processing, and adapting / modelling algorithms, for which those were specifically targeted / designed.
I'd be very interested in any results comparing execution times of your C++ with an equivalent awk script, as they both will use the same regex system calls.
I hadn't seen this post, else I'd have responded sooner...

I'm not sure how to set a timer within the awk program to just time the replacing time itself, so the results will be based on the programs opening/searching for the key/replacing text/closing.

The word 'ad' was used as it has a long definition and is close to the beginning of the file (less search time).

The timer program:
Code:
#!/bin/mksh
#
# Run:
#
# $ ./bench.sh '<program> <parameters>' <number of iterations> 
#
# Ex.:
#
# $ ./bench.sh 'lat ad' 1000
#

echo "\nRunning \""$1"\" for" $2 "iterations.\n\nPlease wait...\n"

i=0

time while [ $i -le $2 ]; do /home/user/scripts/$1; i=$(($i + 1)); done

The results:

Awk (regex):
1000x @ 2m08.22s real 2m04.66s user 0m05.01s system

C++ (regex):
1000x @ 15.24s real 0m13.44s user 0m01.53s system (8.4x faster than Awk)

C++ (custom search and replace - I'll post this code on the forum)
1000x @ 3.37s real 0m02.54s user 0m00.62s system (4.5x faster than C++ regex)


Timed within the programs themselves and only timing the search and replace process:

C++ (regex)
1000x @ 12.26s real 0m05.82s user 0m00.00s system
Ram used: 3mb

C++ (custom)
1000x @ 1.66s real 0m01.66s user 0m00.00s system
Ram used: > 1mb
This User Gave Thanks to bedtime For This Post:
RudiC
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

use var in gsub of awk

Hi all, This problem has cost me half a day, and i still do not know how to do. Any help will be appreciated. Thanks advance. I want to use a variable as the first parameters of gsub function of awk. Example: { ... arri]=gsub(i,tolower(i),$1) (which should be ambraced by //) ... } (1 Reply)
Discussion started by: summer_cherry
1 Replies

2. Shell Programming and Scripting

Help with AWK and gsub

Hello, I have a variable that displays the following results from a JVM.... 1602100K->1578435K I would like to collect the value of 1578435 which is the value after a garbage collection. I've tried the following command but it looks like I can't get the > to work. Any suggestions as... (4 Replies)
Discussion started by: npolite
4 Replies

3. Shell Programming and Scripting

awk gsub

Hi all I want to do a simple substitution in awk but I am getting unexpected output. My function accepts a time and then prints out a validation message if the time is valid. However some times may include a : and i want to strip this out if it exists before i get to the validation. I have shown... (4 Replies)
Discussion started by: pxy2d1
4 Replies

4. Shell Programming and Scripting

Awk Gsub Query

Hi, Can some one please explain the following line please throw some light on the ones marked in red awk '{print $9}' ${FTP_LOG} | awk -v start=${START_DATE} 'BEGIN { FS = "." } { old_line1=$0; gsub(/\-/,""); if ( $3 >= start ) print old_line1 }' | awk -v end=${END_DATE} 'BEGIN { FS="." } {... (3 Replies)
Discussion started by: crosairs
3 Replies

5. Shell Programming and Scripting

Awk gsub error.

I want to replace comma with space and "*646#" with space. I am using the following code: nawk -F"|" '{gsub(","," ",$3); gsub(/\*646\#/"," ",$3);print}' OFS="|" file I am getting following error: Help is appreciated (5 Replies)
Discussion started by: pinnacle
5 Replies

6. Shell Programming and Scripting

awk gsub with variables?

Hey, I would like to replace a string by a new one. Teh problem is that both strings should be variables to be flexible, because I am having a lot of files (with the same structure, but in different folders) for i in daysim_* do cd $i/5/ folder=`pwd |awk '{print $1}'` awk '{ if... (3 Replies)
Discussion started by: ergy1983
3 Replies

7. Shell Programming and Scripting

Awk; gsub in fields 3 and 4

I want to transform a log file into input for a database. Here's the log file: Tue Aug 4 20:17:01 PDT 2009 Wireless users: 339 Daily Average: 48.4285 = Tue Aug 11 20:17:01 PDT 2009 Wireless users: 295 Daily Average: 42.1428 = Tue Aug 18 20:17:01 PDT 2009 Wireless users: 294 Daily... (6 Replies)
Discussion started by: Bubnoff
6 Replies

8. Shell Programming and Scripting

awk gsub

Hi, I want to print the first column with original value and without any double quotes The output should look like <original column>|<column without quotes> $ cat a.txt "20121023","19301229712","100397" "20121023","19361629712","100778" "20121030A","19361630412","100838"... (3 Replies)
Discussion started by: ysrini
3 Replies

9. UNIX for Dummies Questions & Answers

awk gsub with variables

Hello, I'm trying to substitute a string with leading zero for all the records except the trailer record using awk command and with variables. The input file test_med1.txt has data like below 1234ABC...........................9200............LF... (2 Replies)
Discussion started by: somu_june
2 Replies

10. Shell Programming and Scripting

Using multiple gsub() function under a loop in awk

Hi ALL, I want to replace string occurrence in my file "Config" using a external file named "Mapping" using awk. $cat Config ! Configuration file for RAVI ! Configuration file for RACHANA ! Configuration file for BALLU $cat Mapping ravi:ram rachana:shyam ballu:hameed The... (5 Replies)
Discussion started by: useless79
5 Replies

LEARN ABOUT DEBIAN

dpkg-ruby

DPKG-RUBY(1)						      General Commands Manual						      DPKG-RUBY(1)

NAME

       dpkg-ruby - Utility to read a dpkg style db file, dpkg-awk clone

SYNOPSIS

       dpkg-ruby  [(-f|--file) filename] [(-d|--debug) ##] [(-s|--sort) list] [(-n|--numeric) list] [(-rs|--rec_sep) ??] '<fieldname>:<regex>' ...
       -- <out_fieldname> ..

DESCRIPTION

       dpkg-ruby Parses a dpkg status file(or other similarly formated file) and outputs the resulting records.  It can use  regex  on	the  field
       values to limit the returned records, and it can also be told which fields to output.  As another option, it can sort the matched fields.

OPTIONS

       -f filename
       --file filename
	      The file to parse.  The default is /var/lib/dpkg/status.

       -d [#]
       --debug [#]
	      Each time this is specified, it increased the debug level.

       -s field(s)
       --sort field(s)
	      A space or comma separated list of fields to sort on.

       -n field(s)
       --numeric field(s)
	      A space or comma separated list of fields that should be interpreted as numeric in value.

       -rs ??
       --rec_sep ??
	      Output this string at the end of each output paragraph.

       -h
       --help Display some help.

       fieldname
	      The fields from the file, that are matched with the regex given.	The fieldnames are case insensitive.

       out_fieldname
	      The  fields  from the file, that are outputted for each record.  If the first field listed is begins with ^, then the list that fol-
	      lows are fields NOT to be outputted.

BUGS

       Be warned that the author has only a shallow understanding of the dpkg packaging system, so there are probably tons of bugs  in	this  pro-
       gram.

       This  program  comes with no warranties.  If running this program causes fire and brimstone to rain down upon the earth, you will be on our
       own.

       This program accesses the dpkg database directly in places, querying for data that cannot be gotten via dpkg.

AUTHOR

       Fumitoshi UKAI <ukai@debian.or.jp>.  This manual page are based on (or almost copy from :) dpkg-awk(1) manual written by Adam  Heath  <doo-
       gie@debian.org>

DEBIAN
								 Debian Utilities						      DPKG-RUBY(1)
All times are GMT -4. The time now is 02:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy