08-16-2011
Quote:
Originally Posted by
LeNouveau
I would like to use properly an input pipe, like this :
cat myFile.txt | myCommand.sh
That is a
useless use of cat, which may account for a little (but probably not most) of the lost performance.
Quote:
On a big file, with a simple grep, I can spend 2400 times more time ! oO
(from 0,023sec to 1m)
So, what did I miss ? ^^'
I don't know -- I can't see the contents of this myCommand.sh from here. There may well be things that can be improved if you show us what you actually did. Frequent mistakes include the use of cut/awk/sed and so forth many times per loop instead of shell builtins, burdensomely long pipe chains, etc, etc, etc.
In any case, grep is a purpose-built utility made in raw C -- of course it's faster.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi guys,
I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help.
I have 2 input files (example given below)
Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I want to check a particular word is in standard error output or not. Can I acheive it in single command?
For example,
Delete file_name 2>error.log
cat error.log
Output:
XXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXX
Successfully deleted
XXXXXXXXXXXXXXXXX
where delete is... (2 Replies)
Discussion started by: poova
2 Replies
3. Shell Programming and Scripting
Hi everyone,
Is there any way to use the pipe "|" operator to pass more than one input? For instance do:
(command1) (command2) | command3
with command 3 taking 2 parameters?
or maybe using ``s?
Thanks!
Anthony (4 Replies)
Discussion started by: anthony.cros
4 Replies
4. Shell Programming and Scripting
Hi All ,
Is there any way to replace the pipe ( | ) with the broken pipe (0xA6) in unix (1 Reply)
Discussion started by: saj
1 Replies
5. Shell Programming and Scripting
Hello,
I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall:
I regularly need to delete files from many directories.
A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies
6. Shell Programming and Scripting
Hi guys,
I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file
for eg:
file 1:
xx|yy|"xyz|zzz"|zzz|12...
using below awk command
awk 'BEGIN {FS=OFS="|" } print $3
i would get xyz
But i want as :
xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies
7. Shell Programming and Scripting
Dear Friends,
I am looking for a shell script to merge input files into one file .. here is my idea:
1st paramter would be outfile file (all input files content)
read all input files and merge them to input param 1
ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies
8. Shell Programming and Scripting
Hello,
I have an ffmpeg bash script which is working nice and
I need to do the same for other sources.
To create new scripts and to deal with multiple bash files sounds not logical. It is a bit hard to manage for me..
I wondered if it was possible to make my input file as variable.
Then I... (1 Reply)
Discussion started by: baris35
1 Replies
9. Shell Programming and Scripting
Hello,
Normally below script works, could you please comment out what could be the reason of failure if there are spaces in input filename:
script.sh
#!/bin/bash
cd /home/hts/.hts/tvh/
file="$1 $2 $3 $4"
read -d $'\x04' name < "$file"
/usr/bin/ffmpeg -i ""$name"" -vcodec copy -preset... (1 Reply)
Discussion started by: baris35
1 Replies
10. UNIX for Beginners Questions & Answers
Dear UNIX forum members,
I am using macbook pro 13 (2015 edition) with MAC OS Mojave and am trying to write the shell script where when it is run through terminal it asks for an input (in the code below an input variable is domains) and then that input becomes capital letter or letters which... (3 Replies)
Discussion started by: Aurimas
3 Replies
LEARN ABOUT DEBIAN
shell-quote
SHELL-QUOTE(1p) User Contributed Perl Documentation SHELL-QUOTE(1p)
NAME
shell-quote - quote arguments for safe use, unmodified in a shell command
SYNOPSIS
shell-quote [switch]... arg...
DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands
or files with embedded white space or shell globbing characters safely. Here are a few examples.
EXAMPLES
ssh preserving args
When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and
passes them to "$SHELL -c". This doesn't work as intended:
ssh host touch 'hi there' # fails
It creates 2 files, hi and there. Instead, do this:
cmd=`shell-quote touch 'hi there'`
ssh host "$cmd"
This gives you just 1 file, hi there.
process find output
It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to
split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote:
eval set -- `find -type f -print0 | xargs -0 shell-quote --`
debug shell scripts
shell-quote is better than echo for debugging shell scripts.
debug() {
[ -z "$debug" ] || shell-quote "debug:" "$@"
}
With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can.
save a command for later
shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command
you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are
things the user can't pass through), you can do something like this:
user_switches=
while [ $# != 0 ]
do
case x$1 in
x--pass-through)
[ $# -gt 1 ] || die "need an argument for $1"
user_switches="$user_switches "`shell-quote -- "$2"`
shift;;
# process other switches
esac
shift
done
# later
eval "shell-quote some-command $user_switches my args"
OPTIONS
--debug
Turn debugging on.
--help
Show the usage message and die.
--version
Show the version number and exit.
AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions.
AUTHOR
Roderick Schertler <roderick@argon.org>
perl v5.8.4 2005-05-03 SHELL-QUOTE(1p)