Sponsored Content
Top Forums Shell Programming and Scripting KSH: Opening Files based on a file list Post 302443266 by agama on Sunday 8th of August 2010 12:24:44 AM
Old 08-08-2010
Quote:
Originally Posted by drumminfool91
I'm trying to understand xargs, though. What does that do for me here? I only ask b/c I've never used before.
It is possible to have written the command like this:

Code:
cat $(egrep -l "pattern" *)

except that the result of the egrep might be more arguments than can be handled on a single command line. By piping the output to xargs, xargs will execute the command that you give it (cat in this case) with n number of arguments such that n does not exceed the limit from a command line perspective. It may invoke the command multiple times to accomplish its task.

With this said, I realise that my original command would be flawed as the '*' given to egrep might expand into too many arguments, causing an error, and thus the xargs in this case would do no good. This would probably be better:

Code:
ls | xargs egrep -l "system.*log|log.*system" | xargs cat > result

The ls command generates a list of filenames which is given to egrep via the first xargs. The result of the searches is then given to cat via the second xargs.

When writing a script it is usually best to assume the worst case and use xargs in this situation. If you're willing to live with the errors, and are pretty sure that there will not be more files in a directory than the max allowable args on a command line, then the first form (without xargs) is easiest and straightforward.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh: How to get latest file from a list of files in a directory

Hi, I need to get the latest file from a list of files in a particular directory. Please could anyone help me out to get the file. Thank you, - Jay. (1 Reply)
Discussion started by: Jayathirtha
1 Replies

2. Shell Programming and Scripting

create diffrent files based on other file and parameters list

I would like ot create shell script/ bash to create diffrent files based on a file and parameters list. Here is the detail example: I have a textfile and four static parameter files (having ‘?'). mainfile.txt has below records (this count may be more than 50) A200001 A200101 B200001... (9 Replies)
Discussion started by: raghav525
9 Replies

3. Shell Programming and Scripting

Perl-opening a file then copying files

Good morning guys!! Im still practicing with Perl and now Im trying to open a file, and copy its contents to another file. Them I want to remeove the information out of the orginal file after it is copied over. The flow should be messages-->messages1-->messages2. Kind of like a log... (1 Reply)
Discussion started by: bigben1220
1 Replies

4. Shell Programming and Scripting

Moving log files based on month - help with ksh shell script

Hello, I'm trying to move the log files from the parent directory to respective monthly folder and I would be running this script on a weekly basis through cron. I'm new to this scripting and here is what i could come up and it runs without really doing anything. I even tried placing echo... (2 Replies)
Discussion started by: jusblues
2 Replies

5. Shell Programming and Scripting

scp list of files using FOR LOOP in ksh

hi i want to scp files from remote server B to my local server A... and i have a file containing list of all files to be scped from remote server B there is a passwordless connectivity set between remote server and my local server. need a ksh script.. with a for loop that goes through... (2 Replies)
Discussion started by: billpeter3010
2 Replies

6. UNIX for Dummies Questions & Answers

Help with ksh script to list files, cp it to another UNIX server

Hi, I'm quite new to ksh scripting, can someone help me with this. Requirements: I need to create a script that list the files from a user input date range. e. g. format of file: *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00*... (7 Replies)
Discussion started by: chococrunch6
7 Replies

7. Shell Programming and Scripting

Help with ksh-to read ip file & append lines to another file based on pattern match

Hi, I need help with this- input.txt : L B white X Y white A B brown M Y black Read this input file and if 3rd column is "white", then add specific lines to another file insert.txt. If 3rd column is brown, add different set of lines to insert.txt, and so on. For example, the given... (6 Replies)
Discussion started by: prashob123
6 Replies

8. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

9. Shell Programming and Scripting

List the files after sorting based on file content

Hi, I have two pipe separated files as below: head -3 file1.txt "HD"|"Nov 11 2016 4:08AM"|"0000000018" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" head -3 file2.txt "HD"|"Nov 15 2016 2:18AM"|"0000000019" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" I want to list the... (6 Replies)
Discussion started by: Prasannag87
6 Replies

10. Shell Programming and Scripting

How to find list of missing files based on the file format?

Hi All, In the file names we have dates. Based on the file format given by the user, if any file is not existed for a particular date with in a given interval we should consider that file is missing. I have the below files in the directory /bin/daily/voda_files. ... (9 Replies)
Discussion started by: nalu
9 Replies
XARGS(1)						    BSD General Commands Manual 						  XARGS(1)

NAME
xargs -- construct argument list(s) and execute utility SYNOPSIS
xargs [-0oprt] [-E eofstr] [-I replstr [-R replacements] [-S replsize]] [-J replstr] [-L number] [-n number [-x]] [-P maxprocs] [-s size] [utility [argument ...]] DESCRIPTION
The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments. Any arguments specified on the command line are given to utility upon each invocation, followed by some number of the arguments read from the standard input of xargs. This is repeated until standard input is exhausted. Spaces, tabs and newlines may be embedded in arguments using single (`` ' '') or double (``"'') quotes or backslashes (``''). Single quotes escape all non-single quote characters, excluding newlines, up to the matching single quote. Double quotes escape all non-double quote char- acters, excluding newlines, up to the matching double quote. Any single character, including newlines, may be escaped by a backslash. The options are as follows: -0 Change xargs to expect NUL (``'') characters as separators, instead of spaces and newlines. This is expected to be used in concert with the -print0 function in find(1). -E eofstr Use eofstr as a logical EOF marker. -I replstr Execute utility for each input line, replacing one or more occurrences of replstr in up to replacements (or 5 if no -R flag is speci- fied) arguments to utility with the entire line of input. The resulting arguments, after replacement is done, will not be allowed to grow beyond replsize (or 255 if no -S flag is specified) bytes; this is implemented by concatenating as much of the argument contain- ing replstr as possible, to the constructed arguments to utility, up to replsize bytes. The size limit does not apply to arguments to utility which do not contain replstr, and furthermore, no replacement will be done on utility itself. Implies -x. -J replstr If this option is specified, xargs will use the data read from standard input to replace the first occurrence of replstr instead of appending that data after all other arguments. This option will not affect how many arguments will be read from input (-n), or the size of the command(s) xargs will generate (-s). The option just moves where those arguments will be placed in the command(s) that are executed. The replstr must show up as a distinct argument to xargs. It will not be recognized if, for instance, it is in the middle of a quoted string. Furthermore, only the first occurrence of the replstr will be replaced. For example, the following com- mand will copy the list of files and directories which start with an uppercase letter in the current directory to destdir: /bin/ls -1d [A-Z]* | xargs -J % cp -Rp % destdir -L number Call utility for every number lines read. If EOF is reached and fewer lines have been read than number then utility will be called with the available lines. -n number Set the maximum number of arguments taken from standard input for each invocation of utility. An invocation of utility will use less than number standard input arguments if the number of bytes accumulated (see the -s option) exceeds the specified size or there are fewer than number arguments remaining for the last invocation of utility. The current default value for number is 5000. -o Reopen stdin as /dev/tty in the child process before executing the command. This is useful if you want xargs to run an interactive application. -P maxprocs Parallel mode: run at most maxprocs invocations of utility at once. -p Echo each command to be executed and ask the user whether it should be executed. An affirmative response, 'y' in the POSIX locale, causes the command to be executed, any other response causes it to be skipped. No commands are executed if the process is not attached to a terminal. -r Compatibility with GNU xargs. The GNU version of xargs runs the utility argument at least once, even if xargs input is empty, and it supports a -r option to inhibit this behavior. The FreeBSD version of xargs does not run the utility argument on empty input, but it supports the -r option for command-line compatibility with GNU xargs, but the -r option does nothing in the FreeBSD version of xargs. -R replacements Specify the maximum number of arguments that -I will do replacement in. If replacements is negative, the number of arguments in which to replace is unbounded. -S replsize Specify the amount of space (in bytes) that -I can use for replacements. The default for replsize is 255. -s size Set the maximum number of bytes for the command line length provided to utility. The sum of the length of the utility name, the arguments passed to utility (including NULL terminators) and the current environment will be less than or equal to this number. The current default value for size is ARG_MAX - 4096. -t Echo the command to be executed to standard error immediately before it is executed. -x Force xargs to terminate immediately if a command line containing number arguments will not fit in the specified (or default) command line length. If utility is omitted, echo(1) is used. Undefined behavior may occur if utility reads from the standard input. If a command line cannot be assembled, or cannot be invoked, or if an invocation of utility is terminated by a signal, or an invocation of utility exits with a value of 255, the xargs utility stops processing input and exits after all invocations of utility finish processing. EXIT STATUS
The xargs utility exits with a value of 0 if no error occurs. If utility cannot be found, xargs exits with a value of 127, otherwise if utility cannot be executed, xargs exits with a value of 126. If any other error occurs, xargs exits with a value of 1. SEE ALSO
echo(1), find(1), execvp(3) STANDARDS
The xargs utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compliant. The -J, -o, -P, -R and -S options are non-standard FreeBSD extensions which may not be available on other operating systems. HISTORY
The xargs utility appeared in PWB UNIX. BUGS
If utility attempts to invoke another command such that the number of arguments or the size of the environment is increased, it risks execvp(3) failing with E2BIG. The xargs utility does not take multibyte characters into account when performing string comparisons for the -I and -J options, which may lead to incorrect results in some locales. BSD
March 16, 2012 BSD
All times are GMT -4. The time now is 04:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy