Visit Our UNIX and Linux User Community


Grep Command


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Grep Command
# 1  
Old 07-29-2009
Grep Command

Hi,

I have encountered a problem with grep command. The max characters that grep can supported is 2048 as defined in LINE_MAX in hp-ux. I try setting $TK_GREP_LINE_MAX but this is not workable in HP-UX, anyone has experienced with setting the max characters supported by grep command.

Well, grep can be replace with sed, but for curiosity, can anyone give their kind advice on how it can be done using grep

Thanks a lot.

Regards
# 2  
Old 07-29-2009
What are you actually trying to do? I don't know anything about LINE_MAX, but I might be able to help with the more general question?

If LINE_MAX is the limit of the pattern you can define, that doesn't surprise me, since a very long pattern would be a huge job to process. It does surprise me if it's the limit of a line it can process within a file, and in fact, that makes no sense at all. It can't skip to the next line without first finding the next \n, so it would have to process that far anyway... Maybe it just stops processing and reads to the next \n? That's just plain weird...

Hmmm... the man page for grep on Linux doesn't mention LINE_MAX, but (..google...) the one for sun does... hmmm... undefined behaviour... nice

Another option would be a perl one-liner. See the -p switch in the perlrun man page.
# 3  
Old 07-29-2009
Hi pileofrogs,

I am trying to do a selection of lines in the file that fullfilled a matching prefix i.e. [ABC]. Initially, i don't know about the limitation on the number of characters permitted by grep command until i trying to grep a line with 7000 characters and i found that it only return 2048. So i perform a google and found that the command is actually limited by the macro define in ulimit.h under the system include folder.

The man page does not mention about it but it does breifly touch on environmental varaibles hence i am trying to find out if anyone has experiences with manipulation of the environmental variable in grep.

Cheers
# 4  
Old 07-30-2009
Please post the script. Truncation to 2048 characters should not happen with grep in a modern HP-UX. Is this a clean plain text file with no extra control codes and a proper record terminator of line-feed ?

For those who are interested, LINE_MAX is mentioned in "man 5 limits" and /usr/include/limits.h .
# 5  
Old 08-02-2009
Perl is probably your best bet. It will consume as much system resources as are available - by design. So it'll handle strings as long as you want.

Last edited by Beaknit; 08-02-2009 at 02:21 PM.. Reason: Typo
# 6  
Old 08-02-2009
Hi.

I don't use HPUX much, but I keep a login for comparisons.

Here is a script that was run on an HP. I downloaded a version of grep that was written in perl. I created a 3-line data file. The first and last lines are quite short, adding up to 11 characters (with newlines), and the middle line is several thousands of characters long. That second line contains a "9", which will be the string for which we will search:
Code:
#!/usr/bin/env bash

# @(#) s1       Demonstrate perl version of grep.
# Found at:
# http://cpansearch.perl.org/src/CWEST/ppt-0.14/src/grep/tcgrep

echo
set +o nounset
LC_ALL=C ; LANG=C ; export LC_ALL LANG
echo "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version "=o" $(_eat $0 $1) grep ./tcgrep
set -o nounset
echo

FILE=${1-data1}

echo " Data file, first and last line, counts $FILE:"
head -1 $FILE
tail -1 $FILE
wc $FILE

echo
echo " Results standard grep:"
time grep 9 $FILE |
wc 

echo
echo " Results perl grep:"
time ./tcgrep 9 $FILE |
wc 

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: HP-UX, B.11.00, 9000/712
Distribution        : GenericSysName [HP Release B.11.00] (see /etc/issue)
GNU bash 2.05b.0
grep - ( /usr/bin/grep Nov 7 1997 )
./tcgrep - (local: ./tcgrep Jul 29 17:19 )

 Data file, first and last line, counts data1:
First
Last
3 3 6905 data1

 Results standard grep:
1 1 6894

real    0m0.091s
user    0m0.060s
sys     0m0.030s

 Results perl grep:
1 1 6894

real    0m1.349s
user    0m1.190s
sys     0m0.140s

There are several conclusions to be drawn here. The system grep has returned more than 2048 characters. Both the system and the perl grep extracted the same line, and the character count is the same. I agree that perl can handle very long lines, but it uses more resources.

This is evidence that HPUX grep (for this combination of versions) did what was expected.

The URL for the perl version of grep points to CPAN, a large repository of perl code. The options in tcgrep are generally not the same as all system versions of grep. However, it ran correctly directly "out of the box".

Best wishes ... cheers, drl

Previous Thread | Next Thread
Test Your Knowledge in Computers #65
Difficulty: Easy
In the TCP/IP model, end-to-end connectivity is provided from host-to-host in the transport layer.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help on grep command

for example i have a directory home/solaris/unix/samplefiles/ with defaults files in it.. those default files have 1 word in common "UNIX". how can i list the files without "UNIX" words in it using grep command thanks, im using unix solaris, korn shell.. (1 Reply)
Discussion started by: daveaztig14
1 Replies

2. Shell Programming and Scripting

Grep command giving different result for different users for same command

Hello, I am running below command as root user #nodetool cfstats tests | grep "Memtable switch count" Memtable switch count: 12 Where as when I try to run same command as another user it gives different result. #su -l zabbix -s /bin/bash -c "nodetool cfstats tests | grep "Memtable switch... (10 Replies)
Discussion started by: Pushpraj
10 Replies

3. Shell Programming and Scripting

Grep command

grep -i -f panel_genes.txt hg19_refGene.txt > match.txt seems to be pulling names the do not exist in the input file (panel_genes.txt) - the output is attached as well (match.txt) For example, RNF185 or ZNF146 are not genes in the input. I am trying to match the input file genes only and am... (9 Replies)
Discussion started by: cmccabe
9 Replies

4. Shell Programming and Scripting

help on grep command...

Hi I have lots of file in on folder and i want to egrep from only few files. List of files...... Polt_KJ_430_OutputRBS_istUt_CR2.log Polt_KN_4122_OutputRBS_ncChk_CR.log Polt_LN_2230_OutputRNC_Hth_CLKLKL.log Solt_KJ_430_OutputRBS_istUt_CR2.log Solt_KN_4122_OutputRBS_ncChk_CR3.log... (2 Replies)
Discussion started by: asavaliya
2 Replies

5. Shell Programming and Scripting

Grep Command

Hi, I have around 500 Text files and Each file will be having either String1 or String2. I want to list the file only which has String1 and Sting2 in a single command.. (5 Replies)
Discussion started by: balasubramani04
5 Replies

6. Shell Programming and Scripting

Help with using grep command with copy command

Hi, im taking an entry Unix class, and as part of my lab assignment I have to copy all files in the /home/david/lab3 directory that have the file extension .save to your lab3/temp directory. I'm having trouble getting the grep to do anything worth while I've been trying to do: cp... (6 Replies)
Discussion started by: Critical jeff
6 Replies

7. Shell Programming and Scripting

can anyone help with shell script command about searching word with grep command?

i want to search in the current directory all the files that contain one word for example "hello" i want to achieve it with the grep command but not with the grep * (2 Replies)
Discussion started by: aintour
2 Replies

8. UNIX for Advanced & Expert Users

how to exclude the GREP command from GREP

I am doing "ps -f" to see my process. but I get lines that one of it represents the ps command itself. I want to grep it out using -v flag, but than I get another process that belongs to the GREP itself : I would like to exclude # ps -f UID PID PPID C STIME TTY TIME CMD... (2 Replies)
Discussion started by: yamsin789
2 Replies

9. UNIX for Dummies Questions & Answers

grep command

hi all i have directory /usr under this directory i have subdirectories tmp1,tmp2,tmp3 like this /usr/tmp1 /usr/tmp2 /usr/tmp3 and so on i want to search string in files (i don't know the name of the files)and i want to serch it in all the directories under the /usr how shell i do... (3 Replies)
Discussion started by: naamas03
3 Replies

10. Shell Programming and Scripting

grep command

What is the meaning of this grep -v $object grant_BU.sql>temp (1 Reply)
Discussion started by: debasis.mishra
1 Replies

Featured Tech Videos