Visit Our UNIX and Linux User Community


On how to select the right tool for a given task


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting On how to select the right tool for a given task
# 8  
Old 10-17-2008
sed is a programmable text filter, nothing more, nothing less. Why its use should be limited to files when its normal modus operandi is to work in a pipeline eludes me.

csh is not the standard shell (mostly because three is no such thing as the standard shell at all), but it is a standard shell of course. Conceded it is outdated, its bugridden and its use is - rightfully so - discouraged. But i don't know any Unix derivate without a C-shell. SAP even has all its start- stop- and installation-scripts use csh and the SAP admin user is configured to have csh as default shell.

bakunin

Last edited by bakunin; 10-18-2008 at 06:53 AM..
# 9  
Old 10-18-2008
Quote:
Originally Posted by cfajohnson
On Solaris, /bin/sh is not a POSIX shell, but there is always one installed. It may be /usr/pkg4/bin/sh, and there is usually ksh and possibly bash.
The POSIX-compliant utilities on Solaris are under /usr/xpg4. The conformant shell is /usr/xpg4/bin/sh.

BTW, OpenSolaris and Solaris are in the process of changing the default shell to ksh93. Code is already in OpenSolaris.
# 10  
Old 10-18-2008
Quote:
Originally Posted by bakunin
sed is a programmable text filter, nothing more, nothing less. Why its use should be limited to files when its normal modus operandi is to work in a pipeline eludes me.

For efficiency in both time and resources.
Quote:
csh is not the standard shell (mostly because three is no such thing as the standard shell at all),

There certainly is a standard shell. It is specified in the Open Group Base Specifications.
Quote:
but it is a standard shell of course.

Though it is included in most, if not all, *nixen, csh is not required by the UNIX specification.

Scripts written for a standard shell will not run in csh.
Quote:
Conceded it is outdated, its bugridden and its use is - rightfully so - discouraged. But i don't know any Unix derivate without a C-shell. SAP even has all its start- stop- and installation-scripts use csh and the SAP admin user is configured to have csh as default shell.
# 11  
Old 10-18-2008
I really have to agree with cfajohnson. Putting sed in a loop where you process, say, 50,000 records means that your script must fork() and exec() 50,000 times. And that's assuming that you limited yourself to a single invocation of sed. If you used 10 sed's per record, now you need 500,000 forks() and exec() pairs. That is one million very expensive system calls added to a script. And it doesn't need to be 10 sed's... 10 invocations of stuff like cut, sed, awk, tr, basename, dirname, cat, etc will do the trick.

It is not unusual to see scripts that invoke 10 or more external programs like sed in an inner loop like that. When I am asked to speed up a slow script, this is the first thing I look for. It usually takes me about 10 minutes to rewrite the script in a way that saves millions and millions of system calls.
This User Gave Thanks to Perderabo For This Post:
# 12  
Old 10-18-2008
Quote:
Originally Posted by Perderabo
I really have to agree with cfajohnson. Putting sed in a loop where you process, ...
This is all true, but not the point: cfajohnson claimed that sed is (or should be used) only for files. I challenged that claim by citing that using sed in a pipeline would be a counterexample for an equally sensible use of sed.

Additionally you are correct in assessing the use of any external program in a loop to be expensive in terms of system calls. That doesn't mean it can always be avoided though. Sometimes there is simply no other choice because shell expansion is limited in its possibilities. If your problem can be solved by it, good, if not, you have to resort to some external program with greater capabilities even at the expense of system calls.

bakunin
# 13  
Old 10-18-2008
Quote:
Originally Posted by bakunin
This is all true, but not the point: cfajohnson claimed that sed is (or should be used) only for files. I challenged that claim by citing that using sed in a pipeline would be a counterexample for an equally sensible use of sed.

My point is that sed (or any external command) should not be used to chop a single string, even if it means several instructions where a single regex and an external program would have done the job. You can execute many shell statements in the time it takes to call an external command.

In a pipeline, where multiline input is coming from other commands, and not from the echoing of a single string, sed is fine.
# 14  
Old 10-19-2008
i fully understand cfajohnson's point of view that "sed (or any external command) should not be used to chop a single string".
I am arguing that that's not true because
1) for some situations, using sed's regexp is easier, especially to a person versed in the art of regexp. in this situation, development time is more important than execution time.
2) processor time/resources are so cheap nowadays that its insignificant to even mention.

Previous Thread | Next Thread
Test Your Knowledge in Computers #714
Difficulty: Medium
Machine learning is a well known method for mitigating the Von Neumann performance bottleneck
True or False?

9 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

[HELP] Easy task

I have a simple task for my school work. I'm new with unix, so i need help. I need to write a scenario. Task is. From created txt file read first 3 words and create a 3 catalogs with those 3 words. 2 of those new catalogs should be transferred to other directory. If someone could help me just... (1 Reply)
Discussion started by: justynykas
1 Replies

2. Shell Programming and Scripting

Task

Hi experts, I have a problem with the below shell task: I need to modify the file creatin a paired row , per each row which matches filter (e.g. number of nonempty columns = 5) Output should look like this: second row is original one from the input, first row(red) is pairing row, it's... (29 Replies)
Discussion started by: hernand
29 Replies

3. Shell Programming and Scripting

task

Hi all, I'm newbie and stuck here. Thanks for any help. Input(txt file) a b X c d Y e f Z g h W Requested output: a b X Y c d Y X e f Z W g h W Z Please use code tags when posting data and code samples! (10 Replies)
Discussion started by: hernand
10 Replies

4. Shell Programming and Scripting

last task for my script

hi, infile- create table salary ( occupation_code char(40), earnings decimal(10,2), occ_yearend integer ); outfile- salary:create table salary salary:( occupation_code char(40), salary: earnings decimal(10,2), salary: occ_yearend integer salary:); Thanks. (4 Replies)
Discussion started by: dvah
4 Replies

5. Boot Loaders

Reboot and Select Proper Boot device or insert Boot media in select Boot device and press a key

Hello, I have kubuntu on my laptop and now I decided to switch to Windows 7. I made the bios settings properly (first choice is boot from cd\vd) but I see the error " reboot and select proper Boot device or insert Boot media in select Boot device and press a key " I have tried CD and... (0 Replies)
Discussion started by: rpf
0 Replies

6. Programming

Help with task daemon

believe it or not but this is my first c program (i've worked with java, C#, php though) I am trying to make a daemon that checks if mplayer is running(it's for a projection room) and if it is not then to run mplayer with a file.. So far it's not working and I don't know why Help and comments... (5 Replies)
Discussion started by: james2432
5 Replies

7. Shell Programming and Scripting

Parse an XML task list to create each task.xml file

I have an task definition listing xml file that contains a list of tasks such as <TASKLIST <TASK definition="Completion date" id="Taskname1" Some other <CODE name="Code12" <Parameter pname="Dog" input="5.6" units="feet" etc /Parameter> <Parameter... (3 Replies)
Discussion started by: MissI
3 Replies

8. Shell Programming and Scripting

comment and Uncomment single task out of multiple task

I have a file contains TASK gsnmpproxy { CommandLine = $SMCHOME/bin/gsnmpProxy.exe } TASK gsnmpdbgui { CommandLine = $SMCHOME/bin/gsnmpdbgui.exe I would like to comment and than uncomment specific task eg TASK gsnmpproxy Pls suggest how to do in shell script (9 Replies)
Discussion started by: madhusmita
9 Replies

9. UNIX for Dummies Questions & Answers

process vs task

Hi, I am new to this forum and unix too. I have just started learning unix. As I was going through the first chapter, I read that unix is multitasking, multiprogramming, multiprocessing and multiuser OS. My question is: Is there any difference between a TASK and a PROCESS. How are PROCESS... (2 Replies)
Discussion started by: hana
2 Replies

Featured Tech Videos