Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: String manipulation for sheer speed...
Top Forums UNIX for Dummies Questions & Answers String manipulation for sheer speed... Post 302813239 by agama on Sunday 26th of May 2013 10:16:10 PM
Old 05-26-2013
Have you tried reading the data into an array which would allow you to avoid the substring overhead which I believe is costing you. Your original code ran in about 4 seconds for me using Bash and less than a second with Kshell. The code below, which reads and processes the array, runs much faster under both:

Kshell .07s
Bash .17s

Maybe I'm missing something but this seems to perform pretty well.

Code:
# "decimalstring" is an array of the first 8192 bytes from the input file
# (the variable name is now misleading, but I left it the same as in the original code)
decimalstring=(  $( hexdump -n8192 -s0 -v -e '1/1 "%u "' input-file )  )

n=0
m=0

printf "\nStart!"

# Start position of slow loop.
for (( i = 0; i < 8192; i++ ))
do
        if (( ${decimalstring[$i]} >= 127 ))
        then
                ((m++))
        else
                ((n++))
        fi
done
# End position of slow loop.

printf "\n\nDone!, m=$m, n=$n...\n\n"


The raw output:
Code:
spot:[/home/scooter/src/test]time bash tt18

Start!

Done!, m=2063, n=6129...


real    0m0.17s
user    0m0.17s
sys     0m0.01s
spot:[/home/scooter/src/test]time ksh tt18

Start!

Done!, m=2063, n=6129...


real    0m0.07s
user    0m0.06s
sys     0m0.01s

Last edited by agama; 05-26-2013 at 11:22 PM.. Reason: code tag broken
This User Gave Thanks to agama For This Post:
wisecracker
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

string manipulation

Hello, I have a korn shell string variable str1 = "A,B,Z" I would like to create another korn shell string variable str2 = "letter = 'A' or letter = 'B' or letter = 'Z' " Please help! Thanks in advance an UNIX newbie! (13 Replies)
Discussion started by: hai1973
13 Replies

2. Shell Programming and Scripting

String Manipulation Help

Hey Guys, Right i know how to alter a word to begin with a capital letter, i know how to remove unwanted characters and replace them with the relevant character however i don't now if there is a way to do them all in one line. Code: echo -n ${string:0:1} | tr a-z A-Z #convert first letter... (4 Replies)
Discussion started by: shadow0001
4 Replies

3. Shell Programming and Scripting

string manipulation

i have a file that contains a pattern like this: ajay 1234 newyork available kumar 2345 denver singh 2345 newyork ajay 3456 denver kumar 3456 newyork singh 3456 delhi available ajay 4567 miami kumar 4567 miami singh 4567 delhi i want to search for each line... (5 Replies)
Discussion started by: ajay41aj
5 Replies

4. Shell Programming and Scripting

string manipulation

if I have two string variable, how do I add one to anther. like a= "a" b="b" c=$a+$b but that doesn't work. Is there anyway to solve it.http://www.qtl.co.il/img/copy.pnghttp://www.google.com/favicon.icohttp://www.babylon.com/favicon.icohttp://www.morfix.com/favicon.ico (2 Replies)
Discussion started by: programAngel
2 Replies

5. Shell Programming and Scripting

Help With String Manipulation

Hi Guru's, I need some help with data manipulation using shell scripting. I know how to replace the whole string but not part of the string. The value after aa= should be replaced with the value in the mail leaving ,OU=111,OU=222,DC=333 as is. Below are the inputs and expected outputs. Input:... (17 Replies)
Discussion started by: Samingla
17 Replies

6. Shell Programming and Scripting

string manipulation

Hi, I have the followoing details in one file: opt/tra/domain/test/new/filename1 training/ear/help I need to manipulate the string in the following manner: filename1= opt/tra/domain/test/new/filename1 help=training/ear/help last string is the name and equal sign and then... (2 Replies)
Discussion started by: ckchelladurai
2 Replies

7. Shell Programming and Scripting

Deleting part of a string : string manipulation

i have something like this... echo "teCertificateId" | awk -F'Id' '{ print $1 }' | awk -F'te' '{ print $2 }' Certifica the awk should remove 'te' only if it is present at the start of the string.. anywhere else it should ignore it. expected output is Certificate (7 Replies)
Discussion started by: vivek d r
7 Replies

8. Shell Programming and Scripting

String manipulation

Hi , I am getting a string like aaa,bbb,sdsdad,sdfsdf,sdfsdfdsf,rtyrtyr,45654654,ddfdfdfgdfg,dfgdfgdg........... Now what I need is to format it. So after each nth comma I need one newline. So the above will look like when n=3 aaa,bbb,sdsdad, sdfsdf,sdfsdfdsf,rtyrtyr,... (4 Replies)
Discussion started by: Anupam_Halder
4 Replies

9. Shell Programming and Scripting

String Manipulation

I'm making a little game in Perl, and I am trying to remove the first instance of a character in an arbitrary string. For example, if the string is "cupcakes"and the user enters another string that contains letters from "cupcake" e.g: "sake"the original string will now look like this (below)... (3 Replies)
Discussion started by: whyte_rhyno
3 Replies

10. OS X (Apple)

String manipulation

i have a string that am looking to extract all characters following 3 consecutiv numbers. Example my string is J1705PEAN038TDMN, i need to get TDMN My string can have multiple 3 consecutive numbers, i need what follows last occurance (9 Replies)
Discussion started by: gigagigosu
9 Replies

LEARN ABOUT PHP

sqlite_udf_encode_binary

SQLITE_UDF_ENCODE_BINARY(3)											       SQLITE_UDF_ENCODE_BINARY(3)

sqlite_udf_encode_binary - Encode binary data before returning it from an UDF

SYNOPSIS

       string sqlite_udf_encode_binary (string	$data)

DESCRIPTION

       sqlite_udf_encode_binary(3)  applies  a	binary	encoding to the $data so that it can be safely returned from queries (since the underlying
       libsqlite API is not binary safe).

	If there is a chance that your data might be binary unsafe (e.g.: it contains a NUL byte in the middle rather than at the end,	or  if	it
       has and 0x01 byte as the first character) then you must call this function to encode the return value from your UDF.

	PHP does not perform this encode/decode operation automatically as it would severely impact performance if it did.

       Note

	       Do  not	use  sqlite_escape_string(3)  to  quote  strings  returned  from  UDF's as it will lead to double-quoting of the data. Use
	      sqlite_udf_encode_binary(3) instead!

PARAMETERS

	      o $data
		- The string being encoded.

RETURN VALUES

	The encoded string.

SEE ALSO

       sqlite_udf_decode_binary(3), sqlite_escape_string(3), sqlite_create_function(3), sqlite_create_aggregate(3).

PHP Documentation Group 											       SQLITE_UDF_ENCODE_BINARY(3)
All times are GMT -4. The time now is 04:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy