Sponsored Content
Full Discussion: Piping multiple commands
Top Forums UNIX for Dummies Questions & Answers Piping multiple commands Post 302682651 by alister on Monday 6th of August 2012 03:48:21 PM
Old 08-06-2012
Quote:
Originally Posted by jrymer
Is there a safer more efficient way to go about this?

Code:
#!/bin/bash

G1=(/home/dirone)

find ${G1} -type f -name \*.sff | xargs python /usr/local/bin/sff_extract.py

If the goal is to treat every conceivable pathname verbatim ...

xargs treats spaces, single quotes, double quotes, and perhaps also underscores specially. If any of those characters occurred in a filename, your xargs solution would misbehave.

There are xargs options to work with logical lines, but those modes of operation sacrifice a great deal of efficiency for an imperfect workaround (quotes would no longer be a problem, but leading/trailing spaces could be).

Some find/xarg implementations support delimiting pathnames with a null byte.

While the following is slightly more efficient since piping isn't required, the primary advantage is robustness: any legal filename, no matter how bizarre, will be passed intact:
Code:
find "${G1}" -type f -name \*.sff -exec python /usr/local/bin/sff_extract.py {} +

Regards,
Alister
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to do multiple commands:

NEVERMIND figured it out! (1 Reply)
Discussion started by: llsmr777
1 Replies

2. Shell Programming and Scripting

Piping through commands read as variables from file

This is going to be part of a longer script with more features, but I have boiled it down to the one thing that is presently stumping me. The goal is a script which checks for updates to web pages that can be run as a cron job. The script reads (from a tab-delim file) a URL, an MD5 digest, and an... (1 Reply)
Discussion started by: fitzwilliam
1 Replies

3. Shell Programming and Scripting

multiple commands

Hi, I have multiple commands that i need to run daily on my Solaris servers and watch the output, and actually i do it as multi-tasking. I do not want to put all of them in the file, and run it, and get the entire output at one time. What i want is that it should ask for before it runs next... (5 Replies)
Discussion started by: john_prince
5 Replies

4. Solaris

Help with executing multiple remote commands after multiple hops

Hi SSHers, I have embedded this below code in my shell script.. /usr/bin/ssh -t $USER@$SERVER1 /usr/bin/ssh $USER2@S$SERVER2 echo uptime:`/opt/OV/bin/snmpget -r 0 -t 60 $nodeName system.3.0 | cut -d: -f3-5` SSH to both these servers are public-key authenticated, so things run... (13 Replies)
Discussion started by: LinuxUser2008
13 Replies

5. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of... (20 Replies)
Discussion started by: amarn
20 Replies

6. UNIX for Dummies Questions & Answers

Piping commands

Hi I am tryin to undertand piping command1|command2 from what i learn output of cammand 2 is an intput for command 1 right? If so . What dose next sequence do cat f1 >> f2 | grep '^' I think it takes context of f1 and Concatenate's it to f2 and then looks for ....i don't know..... (7 Replies)
Discussion started by: iliya24
7 Replies

7. Shell Programming and Scripting

Multiple Commands in PS

I need to find in file where the record starts with "A,U,I,R" and their file name Record will look like below A|1|138||XXXX|XX|XX U|2|XX|XX|XX|XX R|3|EK|XX|XX|XX D|4|TY|CC|CC|CC find ./*.dat -exec egrep "^A|U|I|R" \; -exec cut -d'|' -f1,2 \; -exec sort -u {} /dev/null \; But... (3 Replies)
Discussion started by: Niranjancse
3 Replies

8. UNIX for Advanced & Expert Users

Pass Multiple Commands and Open Multiple Xterms via PSS

Hello, I'm attempting to open multiple xterms and run a command as an SAP user via sudo using PSSH. So far, I'm able to run PSSH to a file of servers with no check for keys, open every xterm in to the servers in the file list, and SUDO to the SAP(ADM) user, but it won't do anything else... (11 Replies)
Discussion started by: icemanj
11 Replies

9. Shell Programming and Scripting

Proper chaining and piping of commands

can someone please help me fix the below one liner? what i want to do is run a command, and if that command is successful, pipe the output of the command to another command. i'd prefer not to use any temp files for this. who -blahblah ; if ; then echo exit; fi | egrep username (2 Replies)
Discussion started by: SkySmart
2 Replies

10. UNIX for Beginners Questions & Answers

Piping commands using xargs

Need help in piping commands using xargs I have several .tar.gz files that I need to list the folder content in a subdirectory. For example, a.tar.gz b.tar.gz c.tar.gz The following command works great for each .tar.gz file but it's a pain to run the tar command for each file. tar -tf... (11 Replies)
Discussion started by: april
11 Replies
XARGS(1)						    BSD General Commands Manual 						  XARGS(1)

NAME
xargs -- construct argument list(s) and execute utility SYNOPSIS
xargs [-0oprt] [-E eofstr] [-I replstr [-R replacements] [-S replsize]] [-J replstr] [-L number] [-n number [-x]] [-P maxprocs] [-s size] [utility [argument ...]] DESCRIPTION
The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments. Any arguments specified on the command line are given to utility upon each invocation, followed by some number of the arguments read from the standard input of xargs. This is repeated until standard input is exhausted. Spaces, tabs and newlines may be embedded in arguments using single (`` ' '') or double (``"'') quotes or backslashes (``''). Single quotes escape all non-single quote characters, excluding newlines, up to the matching single quote. Double quotes escape all non-double quote char- acters, excluding newlines, up to the matching double quote. Any single character, including newlines, may be escaped by a backslash. The options are as follows: -0 Change xargs to expect NUL (``'') characters as separators, instead of spaces and newlines. This is expected to be used in concert with the -print0 function in find(1). -E eofstr Use eofstr as a logical EOF marker. -I replstr Execute utility for each input line, replacing one or more occurrences of replstr in up to replacements (or 5 if no -R flag is speci- fied) arguments to utility with the entire line of input. The resulting arguments, after replacement is done, will not be allowed to grow beyond replsize (or 255 if no -S flag is specified) bytes; this is implemented by concatenating as much of the argument contain- ing replstr as possible, to the constructed arguments to utility, up to replsize bytes. The size limit does not apply to arguments to utility which do not contain replstr, and furthermore, no replacement will be done on utility itself. Implies -x. -J replstr If this option is specified, xargs will use the data read from standard input to replace the first occurrence of replstr instead of appending that data after all other arguments. This option will not affect how many arguments will be read from input (-n), or the size of the command(s) xargs will generate (-s). The option just moves where those arguments will be placed in the command(s) that are executed. The replstr must show up as a distinct argument to xargs. It will not be recognized if, for instance, it is in the middle of a quoted string. Furthermore, only the first occurrence of the replstr will be replaced. For example, the following com- mand will copy the list of files and directories which start with an uppercase letter in the current directory to destdir: /bin/ls -1d [A-Z]* | xargs -J % cp -Rp % destdir -L number Call utility for every number lines read. If EOF is reached and fewer lines have been read than number then utility will be called with the available lines. -n number Set the maximum number of arguments taken from standard input for each invocation of utility. An invocation of utility will use less than number standard input arguments if the number of bytes accumulated (see the -s option) exceeds the specified size or there are fewer than number arguments remaining for the last invocation of utility. The current default value for number is 5000. -o Reopen stdin as /dev/tty in the child process before executing the command. This is useful if you want xargs to run an interactive application. -P maxprocs Parallel mode: run at most maxprocs invocations of utility at once. -p Echo each command to be executed and ask the user whether it should be executed. An affirmative response, 'y' in the POSIX locale, causes the command to be executed, any other response causes it to be skipped. No commands are executed if the process is not attached to a terminal. -r Compatibility with GNU xargs. The GNU version of xargs runs the utility argument at least once, even if xargs input is empty, and it supports a -r option to inhibit this behavior. The FreeBSD version of xargs does not run the utility argument on empty input, but it supports the -r option for command-line compatibility with GNU xargs, but the -r option does nothing in the FreeBSD version of xargs. -R replacements Specify the maximum number of arguments that -I will do replacement in. If replacements is negative, the number of arguments in which to replace is unbounded. -S replsize Specify the amount of space (in bytes) that -I can use for replacements. The default for replsize is 255. -s size Set the maximum number of bytes for the command line length provided to utility. The sum of the length of the utility name, the arguments passed to utility (including NULL terminators) and the current environment will be less than or equal to this number. The current default value for size is ARG_MAX - 4096. -t Echo the command to be executed to standard error immediately before it is executed. -x Force xargs to terminate immediately if a command line containing number arguments will not fit in the specified (or default) command line length. If utility is omitted, echo(1) is used. Undefined behavior may occur if utility reads from the standard input. If a command line cannot be assembled, or cannot be invoked, or if an invocation of utility is terminated by a signal, or an invocation of utility exits with a value of 255, the xargs utility stops processing input and exits after all invocations of utility finish processing. EXIT STATUS
The xargs utility exits with a value of 0 if no error occurs. If utility cannot be found, xargs exits with a value of 127, otherwise if utility cannot be executed, xargs exits with a value of 126. If any other error occurs, xargs exits with a value of 1. SEE ALSO
echo(1), find(1), execvp(3) STANDARDS
The xargs utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compliant. The -J, -o, -P, -R and -S options are non-standard FreeBSD extensions which may not be available on other operating systems. HISTORY
The xargs utility appeared in PWB UNIX. BUGS
If utility attempts to invoke another command such that the number of arguments or the size of the environment is increased, it risks execvp(3) failing with E2BIG. The xargs utility does not take multibyte characters into account when performing string comparisons for the -I and -J options, which may lead to incorrect results in some locales. BSD
March 16, 2012 BSD
All times are GMT -4. The time now is 07:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy