Even though string operations on shell variables are relatively fast, stripping three characters off of a huge string takes a while. Consider splitting your huge string into shorter strings and feed them into your existing code in a loop. Note also that the standards say that octal values passed to the printf %b format specifier need to be in the format:
Quote:
"\0ddd", where ddd is a zero, one, two, or three-digit octal number that shall be
converted to a byte with the numeric value specified by the octal number
and your code is not supplying the leading 0 for octal values larger than 077.
I don't have dash installed on my system, but sh, bash, and ksh on OS X El Capitan Version 10.11.6 all produce the output you specified when running the following modified version of your script:
Note that I changed the output file name form /tmp/binary to /tmp/binary2 so you can compare the results of the two scripts directly if you'd like to compare run-times of your script against this script and compare the output files produced.
When testing your script (using ksh instead of dash and with the 2nd operand to printf '%b' modified as shown in the script above, I get the same output as you got with both scripts. But, the script above ran in about 10% of the time needed to run your script. I would expect a considerably greater run time improvement for considerably longer input data.
Note also that although you were passing an argument to the binary function, the function you have defined does not use any positional parameters. Therefore, I have removed that operand from the function invocation.
Last edited by Don Cragun; 08-28-2016 at 08:16 AM..
Reason: Fix typo: s/iexdump/hexdump/ as noted in post #3.
This User Gave Thanks to Don Cragun For This Post:
This is at least an order of magnitude faster than my test code.
Works perfectly in dash, except for your typo for hexdump near the end of the script.
Off to try some big binary files now. I will keep you informed over the next few days.
Shell Check makes minor warnings on line 20...
printf '%s\n' $split_octal | while read octal
...to be:-
printf '%s\n' "$split_octal" | while read -r octal
Bit I don't see why it needs changing as the file is always going to be multiple 3 digit octal values only, along with newlines of course, so I have left as is...
Thanks a lot I have something to get my teeth into now.
If only arrays were allowed...
Last edited by Don Cragun; 08-28-2016 at 08:18 AM..
Isn't that approach a bit overcomplicated? Why not save some lines of code, some variables, and reduce the amount of data shoved to and fro?
Call it like binary "$aplit_octal".
This is at least an order of magnitude faster than my test code.
Works perfectly in dash, except for your typo for hexdump near the end of the script.
Off to try some big binary files now. I will keep you informed over the next few days.
Shell Check makes minor warnings on line 20...
printf '%s\n' $split_octal | while read octal
...to be:-
printf '%s\n' "$split_octal" | while read -r octal
Bit I don't see why it needs changing as the file is always going to be multiple 3 digit octal values only, along with newlines of course, so I have left as is...
Thanks a lot I have something to get my teeth into now.
If only arrays were allowed...
Hi wisecracker,
Thank you for pointing out the iexdump (which has now been fixed in my post). I am not sure how that happened; I have hexdump in the code I copied and tried to paste.
It is good that you left the expansion $split_octal as is. With quotes around that expansion, the function won't work.
Quote:
Originally Posted by RudiC
Isn't that approach a bit overcomplicated? Why not save some lines of code, some variables, and reduce the amount of data shoved to and fro?
Call it like binary "$split_octal".
Yes. This is better. I hadn't noticed that the function was being called with an operand until after I had posted the script. One might also consider changing it to:
and invoke it with:
or with:
where input_file contains text similar to the string assigned to split_octal without the quotes (which would be handy if the data being processed is sometimes in a separate file and sometimes in a shell variable).
This User Gave Thanks to Don Cragun For This Post:
Hi,
I have a VM with following configration .
3.10.0-693.1.1.el7.x86_64 #1 SMP Thu Aug 3 08:15:31 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
My current POSIX is :--
Your environment variables take up 2011 bytes
POSIX upper limit on argument length (this system): 2093093
POSIX smallest... (15 Replies)
Hi All,
I need the answer of below question?
1) How to write multiple cronjobs in shellscript? Is there any way or we cant write in shellscript...
Regards,
Priyanka (2 Replies)
Hello *nix specialists,
Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
I want to create a program that creates 2 child process, and each of them creates 2 threads, and each thread prints its thread id. I0ve allread done that the outuput isn't the outuput i want.
When a run the following comand "$./a.out | sort -u | wc -l" I have the folowing output
2
$:
It should... (3 Replies)
Hi ,
I am using UNIX network programming Vol1 (by R Stevens) book to learn about IPC.
I would be using HP-UX,Solaris and Linux at my work.
I have sections for POSIX and for System V in that book.
I am quite confused in indentifying those OSs as POSIX or SYstem V.
Can anyone please... (1 Reply)
Hi,
We have smb client running on two of the linux boxes and smb server on another linux system. During a backup operation which uses smb, read of a file was allowed while write to the same file was going on.Also simultaneous writes to the same file were allowed.Following are the settings in the... (1 Reply)
I'm in the Solaris environment. I want to write data to a file, but I don't want it to be easily read from the C shell. For example, here's my code:
main ()
{
FILE *fo;
fo = fopen ("filename", "w");
fprintf (fo, "This is a test.\n");
fclose (fo);
}
Anyone can open up... (3 Replies)
HI,
When i am configuring php in SUN Solaris. I am getting the below error.
configure: error: Your system seems to lack POSIX threads.
Do i need to install POSIX? If so can somebody let me know where can i download POSIX for Solaris 8?
Thanks, (2 Replies)
I wanted study and write a unix like system. who can help me.
-------------
Removed the garbled characters... not sure why they were there... (2 Replies)
can somebody explain about the ANSI C vs POSIX. say i was using open and fopen, i know that open is POSIX, and fopen is ANSI C. i read that that POSIX is a system call and ANSI C is like a standard library function. wouldn't the fopen function has to call on open function anyway to open any kind... (2 Replies)