02-05-2011
The most important factor to writing fast shell scripts is to minimize the number of external programs invoked. It is a big deal to create a new process, go find the file containing sed or awk, open it, read it in, transfer control it, wait for it to exit, reclaim the process' resources and deliver the return code to the shell. When I need to manipulate a string I would rather code a dozen internal shell operations that invoke a single copy of sed. This is especially true of string manipulation in a loop.
Once a decision hs been made to invoke an external program, some are faster than others, but this is a relatively minor consideration. sed can tackle fewer jobs than awk. It's a much smaller program than awk. awk always tries to crack a line into fields whether or not this is useful. So sed can outperform awk on the tasks that sed can easily do. tr is smaller still and can outperform sed on the even smaller setset of tasks that tr can handle. But I rarely spend a lot of time worrying about stuff like this. They are all external programs. I try to avoid as many as possible.
Of all of the shell scripts I have posted on this site, the most frequently used is datecalc. And datecalc invokes no external programs at all.
datecalc
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi,
Iam doing the following using sed in a script , it is NOT working
line_old= 3754|Yes|Yes
line_new= 3754|Yes|Yes|Yes|Yes
sed -e 's/$line_old/$line_new/' data.$$ > tmp.$$
mv tmp.$$ data.$$
Regards (5 Replies)
Discussion started by: baanprog
5 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I have a test file as follows:
1G102119 ^ AA1179291 ^ 06oct2006 09:50:35^ 73.4^ 2^ 13^ 0^ 1493
1G102119 ^ AA1179291 ^ 06oct2006 09:49:45^ 73.4^ 2^ 13^ 0^ 1493
1G102119 ^ AA1179291 ^ 06oct2006 09:48:58^ 73.4^ 2^ 17^ 0^ 2
1G102119 ... (9 Replies)
Discussion started by: shashi_kiran_v
9 Replies
3. UNIX for Dummies Questions & Answers
I am working on sed ... to replace a string... but not able to save.
i need to repalce a string in httpd.conf in numerous directories.
I am doing this
find /opt/apache/*/conf/ -name httpd.conf -exec sed 's/LogLevel debug/LogLevel error/g' {} \;
even tried with
find... (4 Replies)
Discussion started by: logic0
4 Replies
4. Shell Programming and Scripting
Hi All,
I need to modify ifcfg-eth0 file. The file currently contains the following:
ONBOOT=no
The desired output of the file is:
ONBOOT=no
DEVICE=eth0
BOOTPROTO=static
IPADDR=234.235.34.56
NETMASK=255.255.255.0
GATEWAY=234.235.34.1
I know sed can help me in this, and... (4 Replies)
Discussion started by: nua7
4 Replies
5. UNIX for Advanced & Expert Users
Hi,
I need to redirect internal internet requests to a auth client site siting on the gateway. Currently users that are authenticated to access the internet have there mac address listed in the FORWARD chain. All other users need to be redirected to a internal site for authentication.
Can... (1 Reply)
Discussion started by: mshindo
1 Replies
6. UNIX for Dummies Questions & Answers
Hi All,
Need some assistance n my script.
My file is LBXBC040904071724 and output should be LBX0904071704
tempFile=`echo $file | cut -c 4-7`
tempFile1=`echo $file | sed -e s/$tempFile//`
min=`expr substr $tempFile1 12 2`
cycleno=`expr substr $tempFile 3 2`
newFile=`echo $tempFile1 |... (2 Replies)
Discussion started by: jda
2 Replies
7. Shell Programming and Scripting
hello,
I am not able to redirect the output to the same file, where I am searching and replacing a pattern.
D:\>cat abc.txt
abc
D:\>sed "s\abc\xyz\g" abc.txt > abc.txt
D:\>cat abc.txt
D:\>
If I dont redirect the output to abc.txt, the command is working fine, even if I append the... (10 Replies)
Discussion started by: chetanpunekar
10 Replies
8. Shell Programming and Scripting
I dont get something about sed
If i have a text file inside contain a:a:a:a:a
sed "s/"$title:$author:$price:$qtyAvailable:$qtySold"/"$Ntitle:$author:$price:$qtyAvailable:$qtySold"/"
This work!! i can change a to be something else
but
If i have a text file inside contain Tom Tom:La... (2 Replies)
Discussion started by: GQiang
2 Replies
9. Shell Programming and Scripting
sed -i '' 's:'<string>/Users/testuser/Desktop/test.sh</string>':'something':g' log.txt
The log file has this in
<string>/Users/testuser/Desktop/test.sh</string> and I want to change it to something
This code isn't working any ideas? Its doing my head in! (4 Replies)
Discussion started by: digitalviking
4 Replies
10. Solaris
Hi, This is a strange issue: We have an sftp server. Users can ssh to it from internal LAN without any issue, but they can not ssh to it externally via firewall. Here is what I got:
OS is Solaris 9. No hosts.allow and hosts.deny files.
Please help. Thank you in advance! (7 Replies)
Discussion started by: aixlover
7 Replies
PYP(1) General Commands Manual PYP(1)
NAME
pyp - The Pyed Piper: A Modern Python Alternative to awk, sed and Other Unix Text Manipulation Utilities
SYNOPSIS
pyp [options] files ...
DESCRIPTION
pyp, the Pyed Piper, is a command line tool for text manipulation. It is similar to awk and sed in functionality, but its subcommands are
Python based, and thus more familiar to many programmers.
It can operate both on a per-line base and on the complete input stream. Different features can be pipelined in a single command by using
the pipe character familiar from shell commands.
pyp backs up its input for reruns with modified commands, and can save commands as macros. On the downside, the rerun feature makes it
unsuitable for continuous pipe operation.
OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is
included below. For a complete description, use --manual.
-h, --help
Show this help message and exit.
-m, --manual
Prints out extended help.
-l, --macro_list
Lists all available macros.
-s MACRO_SAVE_NAME, --macro_save=MACRO_SAVE_NAME
Saves current command as macro. use "#" for adding
comments EXAMPLE:
pyp -s "great_macro # prints first letter" "p[1]".
-f MACRO_FIND_NAME, --macro_find=MACRO_FIND_NAME
Searches for macros with keyword or user name.
-d MACRO_DELETE_NAME, --macro_delete=MACRO_DELETE_NAME
Deletes specified public macro.
-g, --macro_group
Specify group macros for save and delete; default is user.
-t TEXT_FILE, --text_file=TEXT_FILE
Specify text file to load. For advanced users,
you should typically cat a file into pyp.
-x, --execute
Execute all commands.
-c, --turn_off_color
Prints raw, uncolored output.
-u, --unmodified_config
Prints out generic PypCustom.py config file.
-b BLANK_INPUTS, --blank_inputs=BLANK_INPUTS
Generate this number of blank input lines; useful for
generating numbered lists with variable 'n'.
-n, --no_input
Use with command that generates output with no input;
same as --dummy_input 1.
-k, --keep_false
Print blank lines for lines that test as False.
default is to filter out False lines from the output.
-r, --rerun
Rerun based on automatically cached data from the last run.
Use this after executing "pyp", pasting input into the shell,
and hitting CTRL-D.
SEE ALSO
awk(1), grep(1), sed(1).
AUTHOR
pyp was written by Toby Rosen <tobyrosen@gmail.com>.
This manual page was written by Khalid El Fathi <khalid@elfathi.fr>, for the Debian project (and may be used by others).
March 19, 2012 PYP(1)