Sponsored Content
Full Discussion: How to use an input pipe ?
Top Forums Shell Programming and Scripting How to use an input pipe ? Post 302547853 by Corona688 on Tuesday 16th of August 2011 06:35:38 PM
Old 08-16-2011
Quote:
Originally Posted by LeNouveau
I would like to use properly an input pipe, like this :
cat myFile.txt | myCommand.sh
That is a useless use of cat, which may account for a little (but probably not most) of the lost performance.
Quote:
On a big file, with a simple grep, I can spend 2400 times more time ! oO
(from 0,023sec to 1m)

So, what did I miss ? ^^'
I don't know -- I can't see the contents of this myCommand.sh from here. There may well be things that can be improved if you show us what you actually did. Frequent mistakes include the use of cut/awk/sed and so forth many times per loop instead of shell builtins, burdensomely long pipe chains, etc, etc, etc.

In any case, grep is a purpose-built utility made in raw C -- of course it's faster.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

2. UNIX for Dummies Questions & Answers

Standard error output to Pipe input - solved

Hi, I want to check a particular word is in standard error output or not. Can I acheive it in single command? For example, Delete file_name 2>error.log cat error.log Output: XXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXX Successfully deleted XXXXXXXXXXXXXXXXX where delete is... (2 Replies)
Discussion started by: poova
2 Replies

3. Shell Programming and Scripting

pass more than one input with pipe ("|")

Hi everyone, Is there any way to use the pipe "|" operator to pass more than one input? For instance do: (command1) (command2) | command3 with command 3 taking 2 parameters? or maybe using ``s? Thanks! Anthony (4 Replies)
Discussion started by: anthony.cros
4 Replies

4. Shell Programming and Scripting

Replace pipe with Broken Pipe

Hi All , Is there any way to replace the pipe ( | ) with the broken pipe (0xA6) in unix (1 Reply)
Discussion started by: saj
1 Replies

5. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

6. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

7. Shell Programming and Scripting

Read input files and merge them in given order and write them to input one param or one file

Dear Friends, I am looking for a shell script to merge input files into one file .. here is my idea: 1st paramter would be outfile file (all input files content) read all input files and merge them to input param 1 ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies

8. Shell Programming and Scripting

While read pipe input issue

Hello, I have an ffmpeg bash script which is working nice and I need to do the same for other sources. To create new scripts and to deal with multiple bash files sounds not logical. It is a bit hard to manage for me.. I wondered if it was possible to make my input file as variable. Then I... (1 Reply)
Discussion started by: baris35
1 Replies

9. Shell Programming and Scripting

Space in input filename with pipe

Hello, Normally below script works, could you please comment out what could be the reason of failure if there are spaces in input filename: script.sh #!/bin/bash cd /home/hts/.hts/tvh/ file="$1 $2 $3 $4" read -d $'\x04' name < "$file" /usr/bin/ffmpeg -i ""$name"" -vcodec copy -preset... (1 Reply)
Discussion started by: baris35
1 Replies

10. UNIX for Beginners Questions & Answers

Help on passing an input variable to the voronota program (after third pipe)

Dear UNIX forum members, I am using macbook pro 13 (2015 edition) with MAC OS Mojave and am trying to write the shell script where when it is run through terminal it asks for an input (in the code below an input variable is domains) and then that input becomes capital letter or letters which... (3 Replies)
Discussion started by: Aurimas
3 Replies
SHELL-QUOTE(1p) 					User Contributed Perl Documentation					   SHELL-QUOTE(1p)

NAME
shell-quote - quote arguments for safe use, unmodified in a shell command SYNOPSIS
shell-quote [switch]... arg... DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands or files with embedded white space or shell globbing characters safely. Here are a few examples. EXAMPLES
ssh preserving args When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and passes them to "$SHELL -c". This doesn't work as intended: ssh host touch 'hi there' # fails It creates 2 files, hi and there. Instead, do this: cmd=`shell-quote touch 'hi there'` ssh host "$cmd" This gives you just 1 file, hi there. process find output It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote: eval set -- `find -type f -print0 | xargs -0 shell-quote --` debug shell scripts shell-quote is better than echo for debugging shell scripts. debug() { [ -z "$debug" ] || shell-quote "debug:" "$@" } With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can. save a command for later shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are things the user can't pass through), you can do something like this: user_switches= while [ $# != 0 ] do case x$1 in x--pass-through) [ $# -gt 1 ] || die "need an argument for $1" user_switches="$user_switches "`shell-quote -- "$2"` shift;; # process other switches esac shift done # later eval "shell-quote some-command $user_switches my args" OPTIONS
--debug Turn debugging on. --help Show the usage message and die. --version Show the version number and exit. AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions. AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.8.4 2005-05-03 SHELL-QUOTE(1p)
All times are GMT -4. The time now is 12:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy