I have encountered a problem with grep command. The max characters that grep can supported is 2048 as defined in LINE_MAX in hp-ux. I try setting $TK_GREP_LINE_MAX but this is not workable in HP-UX, anyone has experienced with setting the max characters supported by grep command.
Well, grep can be replace with sed, but for curiosity, can anyone give their kind advice on how it can be done using grep
What are you actually trying to do? I don't know anything about LINE_MAX, but I might be able to help with the more general question?
If LINE_MAX is the limit of the pattern you can define, that doesn't surprise me, since a very long pattern would be a huge job to process. It does surprise me if it's the limit of a line it can process within a file, and in fact, that makes no sense at all. It can't skip to the next line without first finding the next \n, so it would have to process that far anyway... Maybe it just stops processing and reads to the next \n? That's just plain weird...
Hmmm... the man page for grep on Linux doesn't mention LINE_MAX, but (..google...) the one for sun does... hmmm... undefined behaviour... nice
Another option would be a perl one-liner. See the -p switch in the perlrun man page.
I am trying to do a selection of lines in the file that fullfilled a matching prefix i.e. [ABC]. Initially, i don't know about the limitation on the number of characters permitted by grep command until i trying to grep a line with 7000 characters and i found that it only return 2048. So i perform a google and found that the command is actually limited by the macro define in ulimit.h under the system include folder.
The man page does not mention about it but it does breifly touch on environmental varaibles hence i am trying to find out if anyone has experiences with manipulation of the environmental variable in grep.
Please post the script. Truncation to 2048 characters should not happen with grep in a modern HP-UX. Is this a clean plain text file with no extra control codes and a proper record terminator of line-feed ?
For those who are interested, LINE_MAX is mentioned in "man 5 limits" and /usr/include/limits.h .
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
Hi.
I don't use HPUX much, but I keep a login for comparisons.
Here is a script that was run on an HP. I downloaded a version of grep that was written in perl. I created a 3-line data file. The first and last lines are quite short, adding up to 11 characters (with newlines), and the middle line is several thousands of characters long. That second line contains a "9", which will be the string for which we will search:
Code:
#!/usr/bin/env bash
# @(#) s1 Demonstrate perl version of grep.
# Found at:
# http://cpansearch.perl.org/src/CWEST/ppt-0.14/src/grep/tcgrep
echo
set +o nounset
LC_ALL=C ; LANG=C ; export LC_ALL LANG
echo "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version "=o" $(_eat $0 $1) grep ./tcgrep
set -o nounset
echo
FILE=${1-data1}
echo " Data file, first and last line, counts $FILE:"
head -1 $FILE
tail -1 $FILE
wc $FILE
echo
echo " Results standard grep:"
time grep 9 $FILE |
wc
echo
echo " Results perl grep:"
time ./tcgrep 9 $FILE |
wc
exit 0
producing:
Code:
$ ./s1
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: HP-UX, B.11.00, 9000/712
Distribution : GenericSysName [HP Release B.11.00] (see /etc/issue)
GNU bash 2.05b.0
grep - ( /usr/bin/grep Nov 7 1997 )
./tcgrep - (local: ./tcgrep Jul 29 17:19 )
Data file, first and last line, counts data1:
First
Last
3 3 6905 data1
Results standard grep:
1 1 6894
real 0m0.091s
user 0m0.060s
sys 0m0.030s
Results perl grep:
1 1 6894
real 0m1.349s
user 0m1.190s
sys 0m0.140s
There are several conclusions to be drawn here. The system grep has returned more than 2048 characters. Both the system and the perl grep extracted the same line, and the character count is the same. I agree that perl can handle very long lines, but it uses more resources.
This is evidence that HPUX grep (for this combination of versions) did what was expected.
The URL for the perl version of grep points to CPAN, a large repository of perl code. The options in tcgrep are generally not the same as all system versions of grep. However, it ran correctly directly "out of the box".
for example i have a directory
home/solaris/unix/samplefiles/
with defaults files in it.. those default files have 1 word in common "UNIX". how can i list the files without "UNIX" words in it using grep command
thanks,
im using unix solaris, korn shell.. (1 Reply)
Hello,
I am running below command as root user
#nodetool cfstats tests | grep "Memtable switch count"
Memtable switch count: 12
Where as when I try to run same command as another user it gives different result.
#su -l zabbix -s /bin/bash -c "nodetool cfstats tests | grep "Memtable switch... (10 Replies)
grep -i -f panel_genes.txt hg19_refGene.txt > match.txt
seems to be pulling names the do not exist in the input file (panel_genes.txt) - the output is attached as well (match.txt)
For example, RNF185 or ZNF146 are not genes in the input. I am trying to match the input file genes only and am... (9 Replies)
Hi
I have lots of file in on folder and i want to egrep from only few files.
List of files......
Polt_KJ_430_OutputRBS_istUt_CR2.log
Polt_KN_4122_OutputRBS_ncChk_CR.log
Polt_LN_2230_OutputRNC_Hth_CLKLKL.log
Solt_KJ_430_OutputRBS_istUt_CR2.log
Solt_KN_4122_OutputRBS_ncChk_CR3.log... (2 Replies)
Hi,
I have around 500 Text files and Each file will be having either
String1 or String2.
I want to list the file only which has String1 and Sting2 in a single command.. (5 Replies)
Hi, im taking an entry Unix class, and as part of my lab assignment I have to copy all files in the /home/david/lab3 directory that have the file extension .save to your lab3/temp directory. I'm having trouble getting the grep to do anything worth while
I've been trying to do:
cp... (6 Replies)
i want to search in the current directory all the files that contain one word for example "hello"
i want to achieve it with the grep command but not with the grep * (2 Replies)
I am doing "ps -f" to see my process.
but I get lines that one of it represents the ps command itself.
I want to grep it out using -v flag, but than I get another process that belongs to the GREP itself :
I would like to exclude
# ps -f
UID PID PPID C STIME TTY TIME CMD... (2 Replies)
hi all
i have directory
/usr
under this directory i have subdirectories
tmp1,tmp2,tmp3
like this
/usr/tmp1
/usr/tmp2
/usr/tmp3
and so on
i want to search string in files (i don't know the name of the files)and i want to serch it in all the directories under the /usr
how shell i do... (3 Replies)
