egrep chars limitation ?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting egrep chars limitation ?
# 1  
Old 06-30-2010
Error egrep chars limitation ?

hello

i have i file which contains more than 5000 lines with number (between 4-10chars by line)

i have tried this
Code:
egrep -v "`cat keep.tmp`" listResult.tmp  > result.tmp

cat keep.tmp
5187|2915|2884|5206|2908|2871|2700|2864|3254|3093|2885|3862|5801|3810|2886|4087|5657|2818|2743|2893|2681|2860|2807|2797|2858|2872|2768|2355|2770|2904|2619|2921|4848|2605|
5008|2611|5275|2754|2679|2601|2597|2568|2615|2899|6300|5969|2693|2574|2548|4262|2607|2983|2688|5592|2707|4946|2652|2683|2665|2660|3116|2675|2680|2876|2344|2875|3111|4299|
5381|2699|6089|3065|2727|2691|3512|2723|6204|2814|2795|2708|2719|2801|2828|2735|6507|2731|2736|2863|2537|2538|2539|2541|2542|2543|2544|2546|2547|2549|2550|2551|2553|2554|
2555|2557|2558|2559|2560|2561|2562|2563|2564|2565|2566|2567|2569|2570|2571|2572|2573|2575|2576|2577|2578|2579|2580|2581|2583|2584|2586|2587|2588|2589|2590|2591|2592|2593|
2594|2595|2596|2598|2599|2600|2603|2604|2606|2608|2609|2610|2612|2613|2614|2617|2618|2620|2621|2622|2623|2624|2625|2626|2628|2629|2630|2631|2632|2633|2634|2635|2639|2636|
2637|2638|2643|2644|2645|2646|2647|2648|2516|2517|2521|2524|2525|2526|2532|2533|2535|2556|2654|1810|2657|2702|2709|2722|2726|2728|2729|2732|2737|2739|2740|2742|2744|2745|
2747|2748|2749|2750|2751|2753|2757|2759|2760|2761|2763|2764|2765|2767|2780|2783|2784|2785|2791|2792|2793|2800|2809|2813|2817|2819|2831|2833|2835|2837|2838|2839|2840|2841|
2843|2844|2846|2847|2849|2850|2862|2662|2690|2703|2724|2730|2734|2741|2766|2769|2773|2775|2776|2777|2781|2786|2788|2789|2790|2794|2798|2804|2806|2815|2816|2820|2823|2825|
2829|2830|2832|2834|2836|2851|2852|2861|2822|2842|2733|2752|277.....

but it returns now lines.
I expect to have datas on the result.tmp file

is there any limitation ?
If its the case, how can i manipulate this file to do an egrep (with good performance) ?

Server : SunOS 5.9 Generic_112233-06 sun4u sparc SUNW,Ultra-250

thanks
ade05fr
# 2  
Old 06-30-2010
what are you trying to do??

Code:
egrep -v "keep.tmp" listResult.tmp  > result.tmp

if you want to search for keep.tmp in listResult.tmp this will work
# 3  
Old 06-30-2010
I think you want
Code:
egrep -v -f [patternfile]  [the data file to search]
egrep -v -f keep.tmp  listResult.tmp > newfile

However, IMO, that approach is REALLY slow. awk would be a lot better.

What is the record layout for listResult.tmp?
# 4  
Old 06-30-2010
As posted the command line is likely to be too long, but it also does not work because the command is evaluated before the cat would run.

This syntax works but may still give trouble with the very long list of numbers on some Operating Systems (especially AIX).
Code:
keep="`cat keep.tmp`"
egrep -v "${keep}" listResult.tmp > result.tmp


It is probably better to list the numbers in a flat file (one line per number) and use "egrep -F":
For example:
Code:
egrep -F -v -f keep.list listResult.tmp > result.tmp

# 5  
Old 07-01-2010
Hello

firstly thanks a lot for your replies.
Now here is my feedback :

1- here is the content of the listResult.tmp file
Code:
cat listResult.tmp 
5187
2915
2884
5206
...

you think awk is more efficient for this kind of execution? I thought it would be better to use standard shell script.

2- the -F option is not available in my os.
Code:
egrep -F -v -f OneEgrepU.tmp listResult.tmp > result.tmp
egrep: illegal option -- F
usage: egrep [ -bchilnsv ] [ -e exp ] [ -f file ] [ strings ] [ file ] ...

3- when i tried one of your solutions i have an empty result.
Code:
 egrep  -v -f OneEgrepU.tmp listResult.tmp > result.tmp

===> generated result.tmp empty

here is the content of the different files
Code:
cat listResult.tmp | more                                                      
5187
2915
2884
5206
2908
2871
2700
2864
...

 cat OneEgrepU.tmp | more 
5187|2915|2884|5206|2908|2871|2700|2864|3254|3093|2885|3862|5801|3810|2886|4087|5657|2818|2743|2893|2681|2860|2807|2797|2858|2872|2768|2355|2770|2904|2619|2921|4848|2605|5008|2611|5275|2754|2679|2601|2597|2568|2615|2899|6300|5969|2693|2574|2548|4262|2607|2983|2688|5592|...

maybe its a problem of the use of egrep in this case ?

Last edited by ade05fr; 07-01-2010 at 02:55 AM..
# 6  
Old 07-01-2010
You're trying to search '5187|2915|2884|5206|2908|2871|2700|2...' in listResult.tmp which as you can see is not there. Hence an empty result.tmp.

If what you're trying to achieve is to find all numbers in listResult.tmp which are not in OneEgrepU.tmp, then you'll need to remove the '|' character and replace it with a newline. So what you need is each number in OneEgrepU.tmp to be on its own line.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to split data with a delimiter having chars and special chars

Hi Team, I have a file a1.txt with data as follows. dfjakjf...asdfkasj</EnableQuotedIDs><SQL><SelectStatement modified='1' type='string'><! The delimiter string: <SelectStatement modified='1' type='string'><! dlm="<SelectStatement modified='1' type='string'><! The above command is... (7 Replies)
Discussion started by: kmanivan82
7 Replies

2. UNIX for Dummies Questions & Answers

Limitation in addition

whats wrong with this addition? Whats the maximum number of digits can be handled? pandeeswaran@ubuntu:~/Downloads$ const=201234454654768979799999 pandeeswaran@ubuntu:~/Downloads$ let new+=const pandeeswaran@ubuntu:~/Downloads$ echo $new -2152890657037557890 pandeeswaran@ubuntu:~/Downloads$ (4 Replies)
Discussion started by: pandeesh
4 Replies

3. Shell Programming and Scripting

find 4 chars on 2nd line, 44 chars over

I know this should be simple, but I've been manning sed awk grep and find and am stupidly stumped :( I'm trying to use sed (or awk, find, etc) to find 4 characters on the second line of a file.txt 44-47 characters in. I can find lots of sed things for lines, but not characters. (4 Replies)
Discussion started by: unclecameron
4 Replies

4. UNIX for Dummies Questions & Answers

search ")" with egrep - egrep: syntax error

Hi Guys, we have a shell script which basically query the Database which retrieves huge data and use the data with "egrep" . Now there is some data which contains characters like "abc)" and the same is used like below : "egrep (.+\|GDPRAB16\|GDPR/11702 96 abc)\|$ temp.txt" now while... (7 Replies)
Discussion started by: sagarjani
7 Replies

5. UNIX for Dummies Questions & Answers

Egrep cheat sheet anywhere? Looking for meaning of egrep -c

Hi I've been searching google and have not found what egrep -c means. Does anyone know where I can get a cheat sheet or what that -c means? thanks, Linda (2 Replies)
Discussion started by: leelm
2 Replies

6. Shell Programming and Scripting

Awk limitation

Hi All, I have an awk code below. I have an input file which has a line which has a last field with about 4000 characters and it pop up an error stated below. It is too much for awk to take ? Awk Code: {if( $NF == "2007" && $1 == "**" ) LINE = $0;} END{printf("%20s\n",LINE); } Error:... (28 Replies)
Discussion started by: Raynon
28 Replies

7. Shell Programming and Scripting

How to convert C source from 8bit chars to 16bit chars?

I was using the following bash command inside the emacs compile command to search C++ source code: grep -inr --include='*.h' --include='*.cpp' '"' * | sed "/include/d" | sed "/_T/d" | sed '/^ *\/\//d' | sed '/extern/d' Emacs will then position me in the correct file and at the correct line... (0 Replies)
Discussion started by: siegfried
0 Replies

8. Shell Programming and Scripting

Is this a bug or a limitation?

Hi, I'm having a problem with a while loop syntax that doesn't seem to loop correctly. TODAY=`date +%d%m%Y` while read hostname #for hostname in $(cat $CONFIG) do OUTFILE=/tmp/health_check.$hostname.$TODAY if then touch $OUTFILE func_header else rm $OUTFILE ... (2 Replies)
Discussion started by: gilberteu
2 Replies

9. HP-UX

HP-UX 11i - File Size Limitation And Number Of Folders Limitation

Hi All, Can anyone please clarify me the following questions: 1. Is there any file size limitation in HP-UX 11i, that I can able to create upto certain size of file (say 2 GB) and not more then that???? 2. At max. how many files we can able to keep inside a folder???? 3. How many... (2 Replies)
Discussion started by: sundeep_mohanty
2 Replies

10. Shell Programming and Scripting

find limitation

Hi , i'm trying to use "find "command with "-size "option but i encounter 2gb file limitation. Can you confirm this limitation ? Is there a simple way to do the same thing ? My command is : <clazz01g-notes01>/base/base01 # find /base/base01 -name '*.nsf' -size +5242880000c -exec ls... (2 Replies)
Discussion started by: Nicol
2 Replies
Login or Register to Ask a Question