08-16-2011
Quote:
Originally Posted by
LeNouveau
I would like to use properly an input pipe, like this :
cat myFile.txt | myCommand.sh
That is a
useless use of cat, which may account for a little (but probably not most) of the lost performance.
Quote:
On a big file, with a simple grep, I can spend 2400 times more time ! oO
(from 0,023sec to 1m)
So, what did I miss ? ^^'
I don't know -- I can't see the contents of this myCommand.sh from here. There may well be things that can be improved if you show us what you actually did. Frequent mistakes include the use of cut/awk/sed and so forth many times per loop instead of shell builtins, burdensomely long pipe chains, etc, etc, etc.
In any case, grep is a purpose-built utility made in raw C -- of course it's faster.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi guys,
I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help.
I have 2 input files (example given below)
Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I want to check a particular word is in standard error output or not. Can I acheive it in single command?
For example,
Delete file_name 2>error.log
cat error.log
Output:
XXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXX
Successfully deleted
XXXXXXXXXXXXXXXXX
where delete is... (2 Replies)
Discussion started by: poova
2 Replies
3. Shell Programming and Scripting
Hi everyone,
Is there any way to use the pipe "|" operator to pass more than one input? For instance do:
(command1) (command2) | command3
with command 3 taking 2 parameters?
or maybe using ``s?
Thanks!
Anthony (4 Replies)
Discussion started by: anthony.cros
4 Replies
4. Shell Programming and Scripting
Hi All ,
Is there any way to replace the pipe ( | ) with the broken pipe (0xA6) in unix (1 Reply)
Discussion started by: saj
1 Replies
5. Shell Programming and Scripting
Hello,
I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall:
I regularly need to delete files from many directories.
A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies
6. Shell Programming and Scripting
Hi guys,
I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file
for eg:
file 1:
xx|yy|"xyz|zzz"|zzz|12...
using below awk command
awk 'BEGIN {FS=OFS="|" } print $3
i would get xyz
But i want as :
xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies
7. Shell Programming and Scripting
Dear Friends,
I am looking for a shell script to merge input files into one file .. here is my idea:
1st paramter would be outfile file (all input files content)
read all input files and merge them to input param 1
ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies
8. Shell Programming and Scripting
Hello,
I have an ffmpeg bash script which is working nice and
I need to do the same for other sources.
To create new scripts and to deal with multiple bash files sounds not logical. It is a bit hard to manage for me..
I wondered if it was possible to make my input file as variable.
Then I... (1 Reply)
Discussion started by: baris35
1 Replies
9. Shell Programming and Scripting
Hello,
Normally below script works, could you please comment out what could be the reason of failure if there are spaces in input filename:
script.sh
#!/bin/bash
cd /home/hts/.hts/tvh/
file="$1 $2 $3 $4"
read -d $'\x04' name < "$file"
/usr/bin/ffmpeg -i ""$name"" -vcodec copy -preset... (1 Reply)
Discussion started by: baris35
1 Replies
10. UNIX for Beginners Questions & Answers
Dear UNIX forum members,
I am using macbook pro 13 (2015 edition) with MAC OS Mojave and am trying to write the shell script where when it is run through terminal it asks for an input (in the code below an input variable is domains) and then that input becomes capital letter or letters which... (3 Replies)
Discussion started by: Aurimas
3 Replies
LEARN ABOUT SUSE
unbuffer
UNBUFFER(1) General Commands Manual UNBUFFER(1)
NAME
unbuffer - unbuffer output
SYNOPSIS
unbuffer program [ args ]
INTRODUCTION
unbuffer disables the output buffering that occurs when program output is redirected from non-interactive programs. For example, suppose
you are watching the output from a fifo by running it through od and then more.
od -c /tmp/fifo | more
You will not see anything until a full page of output has been produced.
You can disable this automatic buffering as follows:
unbuffer od -c /tmp/fifo | more
Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the
-p flag. Example:
process1 | unbuffer -p process2 | process3
CAVEATS
unbuffer -p may appear to work incorrectly if a process feeding input to unbuffer exits. Consider:
process1 | unbuffer -p process2 | process3
If process1 exits, process2 may not yet have finished. It is impossible for unbuffer to know long to wait for process2 and process2 may
not ever finish, for example, if it is a filter. For expediency, unbuffer simply exits when it encounters an EOF from either its input or
process2.
In order to have a version of unbuffer that worked in all situations, an oracle would be necessary. If you want an application-specific
solution, workarounds or hand-coded Expect may be more suitable. For example, the following example shows how to allow grep to finish pro-
cessing when the cat before it finishes first. Using cat to feed grep would never require unbuffer in real life. It is merely a place-
holder for some imaginary process that may or may not finish. Similarly, the final cat at the end of the pipeline is also a placeholder
for another process.
$ cat /tmp/abcdef.log | grep abc | cat
abcdef
xxxabc defxxx
$ cat /tmp/abcdef.log | unbuffer grep abc | cat
$ (cat /tmp/abcdef.log ; sleep 1) | unbuffer grep abc | cat
abcdef
xxxabc defxxx
$
BUGS
The man page is longer than the program.
SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995.
AUTHOR
Don Libes, National Institute of Standards and Technology
1 June 1994 UNBUFFER(1)