awk includes pattern matching capabilities similar to grep. So in lieu of grepping "State" to stdout and then printing the third field and then evaluating it, the proposal does all three actions in one go:
This is not correct, there are no line length limitations to a for loop. A for loop is not a utility, it is not even a builtin utility. A for loop is part of the shell syntax, so neither the limitations of passing parameters to a subprocess apply
Could you quote your findings? I couldn't find anything which would back it up in the man pages of bash or zsh (I didn't check it for other shells). Well, even then, some limit likely applies (simply because the available memory is finite), but I agree that this limit might possibly be larger than the one which usually applies for the command line.
Still, exploiting this limit migh not be wise. I did some experimentation with this in bash and zsh, basically pasting the content of a huge file to the command line, with the result that after a certain size, my system started swapping (obviously because the shell tried to generate the whole command line in-memory, which makes sense) and in fact blocked the whole process (I killed it after a couple minutes). I would conclude that for processing large amounts of data, a solution based on xargs is safer than a command line expansion, even if it is "only" a for loop.
Could you quote your findings? I couldn't find anything which would back it up in the man pages of bash or zsh (I didn't check it for other shells). Well, even then, some limit likely applies (simply because the available memory is finite), but I agree that this limit might possibly be larger than the one which usually applies for the command line.
Still, exploiting this limit migh not be wise. I did some experimentation with this in bash and zsh, basically pasting the content of a huge file to the command line, with the result that after a certain size, my system started swapping (obviously because the shell tried to generate the whole command line in-memory, which makes sense) and in fact blocked the whole process (I killed it after a couple minutes). I would conclude that for processing large amounts of data, a solution based on xargs is safer than a command line expansion, even if it is "only" a for loop.
It is hard to prove a negative. The point is that the limit ARG_MAX in a C program applies to the arguments and environment variables that need to be passed to a new process image which happens when using C functions like the exec family of functions (including, but not limited to execl(), execle(), execv(), and fexecve()) and posix_spawn(). If your shell isn't written in C, whatever language you're using will still need to ask the kernel to create a new process image at some point, and when it does that it will be making the same system calls that the C library invokes to make the exec family of functions and the posix_spawn() function do their jobs so they would still be subject to the error return that would result in errno being set to E2BIG in a C program. It does not apply to shell compound commands like:
and if statements, while loops, until loops, and function definition commands (none of which are run by the shell in a new process).
As far as the LINE_MAX limit, the standard is very clear in its description of the input files (i.e. shell scripts) read by the sh utility (which apply to any standards-conforming shell):
Quote:
The input file shall be a text file, except that line lengths shall be unlimited.
In practice, shell scripts are usually text files meeting LINE_MAX limitations, but that isn't due to shell restrictions; it is because the editors used to create shell scripts frequently fail if lines are longer than LINE_MAX. And, long lines are harder for humans writing scripts to understand.
I'm not saying you should never use xargs, but xargs has its own set of problems. And, I would never suggest using:
instead of:
(again with IFS set just for the read command to be just a <newline> character), which is much less memory intensive (if file can be large), but produces the same results (unless the shell runs out of memory while gathering arguments for the for loop).
These 3 Users Gave Thanks to Don Cragun For This Post:
I am searching for a process that should be up and running. Im using the following command
ps -ef | grep elasticsearch
to get
elastic+ 1673 1 0 Jan29 ? 05:08:56 /bin/java -Xms4g -Xmx4g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC... (1 Reply)
Hi All
I am working on AIX 7.1 and I am trying to show an output that I get from "cat" a log file to email. However in email I get the below output:
In the script I have defined the colors as:
#!/bin/sh
echo "\033
Below is the script I have created to send this output:
... (9 Replies)
Hi Folks,
I have a situation here, where no command is giving any output, and it's not even showing any error message also.
What could be the reason? (3 Replies)
Hello
I am working on one script where I am trying to display all the directories which is inside the workspace but somehow it is giving me weird output and this is occurring only with one directory other also having the result.html file inside the directory.
for i in `ls -1 | egrep -iv... (2 Replies)
Hi,
This is Solaris-10 box and in few of file-system (root file-system of non global zones), usage/available is not showing correct size. I am not able to figure out, what is eating up this space.
Global Server - bdrpod01
Non Global zone - bdrpod01-zputq01
root@bdrpod01:/root# df -h... (2 Replies)
Running solaris 9, on issuing the follwing command
df -h | awk '$5 > 45 {print}'
Filesystems with utilisation > 45% are being displayed as well as those between
5 and-9%!!! (3 Replies)
Hi, I have the following file called addresses, (it is a large file i have only copy and pasted few of the data below) and I am wanting to write a command so it will Find the ratio of mobile (07....) to land line (01....) telephone numbers?
then find the most popular first name and list the... (1 Reply)
Hi,
I have a very frustrating issue! I hope you guys can assist
When a disk is presented out the iSCSI target display a lower disk capacity
SOLARIS VERSION is SOLARIS 10 05/09 Kernel Patch 139555-31
ISCSI Patch 119090-31, 141878-11
Unix Commands To discover Target
bash-3.00# i... (0 Replies)
I'm using the below command to list files older than 2 hours but it returns redundant output, am I missing something.
# find . -mmin +120 -exec ls -l {} \;
total 0
-rw-r--r-- 1 root system 0 Oct 13 09:52 test1
-rw-r--r-- 1 root system 0 Oct 13 09:52 test2
-rw-r--r-- 1 root ... (5 Replies)