Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Text string parsing in awk
Top Forums Shell Programming and Scripting Text string parsing in awk Post 302892655 by Don Cragun on Friday 14th of March 2014 03:10:53 AM
Old 03-14-2014
Clearly the results vary considerably from system to system. I upped my input test file to 1,000,000 lines and ran each set of code 11 times. I threw out the 1st run for each awk script. The remaining results on my system are:
Code:
    Original    	  Don Cragun   		  Chubler_XL   
================	===============		===============
real	0m26.82s	real	0m5.91s		real	0m6.59s
user	0m25.79s	user	0m5.19s		user	0m6.07s
sys	0m0.68s		sys	0m0.48s		sys	0m0.47s
		
real	0m26.62s	real	0m5.72s		real	0m6.63s
user	0m25.69s	user	0m5.17s		user	0m6.06s
sys	0m0.66s		sys	0m0.47s		sys	0m0.47s
		
real	0m26.95s	real	0m5.85s		real	0m6.63s
user	0m25.79s	user	0m5.18s		user	0m6.07s
sys	0m0.67s		sys	0m0.47s		sys	0m0.46s
		
real	0m26.80s	real	0m5.73s		real	0m6.73s
user	0m25.81s	user	0m5.16s		user	0m6.11s
sys	0m0.68s		sys	0m0.48s		sys	0m0.47s
		
real	0m26.79s	real	0m5.82s		real	0m6.65s
user	0m25.89s	user	0m5.18s		user	0m6.08s
sys	0m0.68s		sys	0m0.47s		sys	0m0.47s
		
real	0m27.20s	real	0m5.76s		real	0m6.64s
user	0m26.01s	user	0m5.18s		user	0m6.05s
sys	0m0.69s		sys	0m0.47s		sys	0m0.47s
		
real	0m27.12s	real	0m5.74s		real	0m6.61s
user	0m26.04s	user	0m5.17s		user	0m6.05s
sys	0m0.68s		sys	0m0.47s		sys	0m0.47s
		
real	0m26.99s	real	0m5.78s		real	0m6.68s
user	0m26.11s	user	0m5.19s		user	0m6.07s
sys	0m0.67s		sys	0m0.48s		sys	0m0.47s
		
real	0m27.00s	real	0m5.78s		real	0m6.65s
user	0m25.98s	user	0m5.17s		user	0m6.09s
sys	0m0.66s		sys	0m0.47s		sys	0m0.47s
		
real	0m26.71s	real	0m5.84s		real	0m6.64s
user	0m25.85s	user	0m5.21s		user	0m6.07s
sys	0m0.67s		sys	0m0.48s		sys	0m0.47s

On OS X, awk using index() 4 times and substr() 5 times seems to be a little slower than using match() 2 times and substr() 2 times. Both are considerably faster than using split() and a for loop to append fields to create the concatenation of the 4th through the NFth fields.
This User Gave Thanks to Don Cragun For This Post:
Michael Stora
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help parsing a string

Hi, I'm writing a shell script that outputs, among other things, some of the information that is outputted by the mysqladmin status command. The output of the command looks like this: Uptime: 816351 Threads: 19 Questions: 80719739 Slow queries: 1419 Opens: 15903523 Flush tables: 1 Open tables:... (6 Replies)
Discussion started by: achieve
6 Replies

2. UNIX for Dummies Questions & Answers

Parsing string

I am passing argument 1-13 to a sh file. I want to parse the string and the get the numbers on either side of "-" in two different variables. I am not familiar with unix .. how can i do this? (3 Replies)
Discussion started by: rolex.mp
3 Replies

3. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

4. Shell Programming and Scripting

String parsing with awk/sed/?

If I have a string that has some name followed by an ID#(ex.B123456) followed by some more #'s and/or letters, would it be possible to just grab the ID portion of this string? If so how? I am pretty new with these text tools so any help is appreciated. Example: "Name_One-B123456A-12348A" (2 Replies)
Discussion started by: airon23bball
2 Replies

5. Shell Programming and Scripting

choose random text between constant string.. using awk?

Hallo I have maybe a little bit advanced request.... I need to choose one random part betwen %.... so i have this.. % text1 text1 text1 text1 text1 text1 text1 text1 text1 % text2 text2 text2 text2 text2 % text3 text3 text3 tetx3 % this choose text between % awk ' /%/... (8 Replies)
Discussion started by: sandwich
8 Replies

6. Shell Programming and Scripting

how to extract a paticular string from the text file with awk.

hello forum members I have txt file which consists the following information. Server: abababa.xyz.ap.mxmx.com Address: 111.143.211.202 Name: rmxd.ipc.ap.mxmx.com Address: 144.111.99.9 from the abovefile i have to extract only string "rmxd.ipc.ap.mxmx.com" through awk command.... (1 Reply)
Discussion started by: rajkumar_g
1 Replies

7. Shell Programming and Scripting

Parsing a long string string problem for procmail

Hi everyone, I am working on fetchmail + procmail to filter mails and I am having problem with parsing a long line in the body of the email. Could anyone help me construct a reg exp for this string below. It needs to match exactly as this string. GetRyt... (4 Replies)
Discussion started by: cwiggler
4 Replies

8. Shell Programming and Scripting

awk + gsub to search multiple input values & replace with located string + extra text

Hi all. I have the following command that is successfully searching for any one of the strings on all lines of a file and replacing it with the instructed value. cat inputFile | awk '{gsub(/aaa|bbb|ccc|ddd/,"1234")}1' > outputFile This does in fact replace any occurrence of aaa, bbb,... (2 Replies)
Discussion started by: dazhoop
2 Replies

9. Shell Programming and Scripting

Complex text parsing with speed/performance problem (awk solution?)

I have 1.6 GB (and growing) of files with needed data between the 11th and 34th line (inclusive) of the second column of comma delimited files. There is also a lot of stray white space in the file that needs to be trimmed. They have DOS-like end of lines. I need to transpose the 11th through... (13 Replies)
Discussion started by: Michael Stora
13 Replies

10. Shell Programming and Scripting

awk to change specific string to new value if found in text file

I am trying to use awk to change a specific string in a field, if it is found, to another value. In the tab-delimited file the text in bold in $3 contains the string 23, which is always right before a ., if it is present. I am trying to change that string to X, keeping the formatting and the... (3 Replies)
Discussion started by: cmccabe
3 Replies

LEARN ABOUT LINUX

times

TIMES(P)						     POSIX Programmer's Manual							  TIMES(P)

NAME

       times - write process times

SYNOPSIS

       times

DESCRIPTION

       The  times  utility  shall  write  the accumulated user and system times for the shell and for all of its child processes, in the following
       POSIX locale format:

	      "%dm%fs %dm%fs
%dm%fs %dm%fs
", <shell user minutes>,
		  <shell user seconds>, <shell system minutes>,
		  <shell system seconds>, <children user minutes>,
		  <children user seconds>, <children system minutes>,
		  <children system seconds>

       The four pairs of times shall correspond to the members of the <sys/times.h> tms structure (defined  in	the  Base  Definitions	volume	of
       IEEE Std 1003.1-2001, Chapter 13, Headers) as returned by times(): tms_utime, tms_stime, tms_cutime, and tms_cstime, respectively.

OPTIONS

       None.

OPERANDS

       None.

STDIN

       Not used.

INPUT FILES

       None.

ENVIRONMENT VARIABLES

       None.

ASYNCHRONOUS EVENTS

       Default.

STDOUT

       See the DESCRIPTION.

STDERR

       The standard error shall be used only for diagnostic messages.

OUTPUT FILES

       None.

EXTENDED DESCRIPTION

       None.

EXIT STATUS

       Zero.

CONSEQUENCES OF ERRORS

       Default.

       The following sections are informative.

APPLICATION USAGE

       None.

EXAMPLES

	      $ times
	      0m0.43s 0m1.11s
	      8m44.18s 1m43.23s

RATIONALE

       The times special built-in from the Single UNIX Specification is now required for all conforming shells.

FUTURE DIRECTIONS

       None.

SEE ALSO

       Special Built-In Utilities

COPYRIGHT

       Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol-
       ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of
       Electrical  and	Electronics  Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE
       and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can  be  obtained
       online at http://www.opengroup.org/unix/online.html .

IEEE
/The Open Group						       2003								  TIMES(P)
All times are GMT -4. The time now is 11:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy