Performance issue in Grepping large files Post: 302819813

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix File System performance with large directories

Hi, how does the Unix File System perform with large directories (containing ~30.000 files)? What kind of structure is used for the organization of a directory's content, linear lists, (binary) trees? I hope the description 'Unix File System' is exact enough, I don't know more about the file...

2. Shell Programming and Scripting

Grepping issue..

I found another problem with my disk-adding script today. When looking for disks, I use grep. When I grep for the following disk sizes: 5242880 I also pick up these as well: 524288000 How do I specifically pick out one or the other, using grep, without resorting to the -v option? ...

3. Shell Programming and Scripting

Performance issue in UNIX while generating .dat file from large text file

Hello Gurus, We are facing some performance issue in UNIX. If someone had faced such kind of issue in past please provide your suggestions on this . Problem Definition: /Few of load processes of our Finance Application are facing issue in UNIX when they uses a shell script having below...

4. Shell Programming and Scripting

replace issue with large files

I have the following problem: I have two files: S containing sentences (one in each row) and W containing files (one in each row). It might look like this: S: a b c apple d. e f orange g. h banana i j. W: orange banana apple My task is to replace in S all words that appear in W...

5. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ...

6. Red Hat

Empty directory, large size and performance

Hi, I've some directory that I used as working directory for a program. At the end of the procedure, the content is deleted. This directory, when I do a ls -l, appears to still take up some space. After a little research, I've seen on a another board of this forum that it's not really taking...

7. Shell Programming and Scripting

Grepping large list of files

Hi All, I need help to know the exact command when I grep large list of files. Either using ls or find command. However I do not want to find in the subdirectories as the number of subdirectories are not fixed. How do I achieve that. I want something like this: find ./ -name "MYFILE*.txt"...

8. Shell Programming and Scripting

Grepping verbal forms from a large corpus

I want to extract verbal forms from a large corpus of English. I have identified a certain number of patterns. Each pattern has the following structure SPACE word_CATEGORY where word refers to the verbal form and CATEGORY refers to the class of the verb The categories are identified as per the...

9. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the...

LEARN ABOUT CENTOS

pcp.conf

PCP.CONF(5)							File Formats Manual						       PCP.CONF(5)

NAME

       pcp.conf - the Performance Co-Pilot configuration and environment file

SYNOPSIS

       /etc/pcp.conf

DESCRIPTION

       When  using  Performance Co-Pilot (PCP) tools and utilities and when calling PCP library functions, a standard set of environment variables
       are defined in /etc/pcp.conf.  These variables are generally used to specify the location of various PCP pieces in the file system and  may
       be  loaded  into  shell scripts by sourcing the /etc/pcp.env(5) shell script and queried by C/C++ programs using the pmGetConfig(3) library
       function.  If a variable is already defined in the environment, the values in pcp.conf do not override those values,  i.e.  the	values	in
       pcp.conf serve as installation defaults only.

       Both  the  pcp.env  and	pcp.conf  files  are expected to be found in /etc by default.  If required, the pcp.conf file may be relocated and
       PCP_CONF set in the environment to specify the full path to the new location.  The pcp.env file can not be relocated (this is the only hard
       coded path required by PCP).

       The syntax rules for pcp.conf are as follows :

       1.  the general syntax is
						       PCP_VARIABLE_NAME=variable value to end of line

       2.  lines that begin with # and all blank lines are ignored.

       3.  all variables must be prefixed with PCP_.  This is a security issue - variables that do not have this prefix will be silently ignored.

       4.  there  should  be  no  space  between the variable name and the literal = and no space between the = and the variable value (unless the
	   value actually starts with a space).  This is required because the pcp.conf file may be sourced directly by Makefiles as well as inter-
	   preted by the pcp.env script and the pmGetConfig function.

       5.  variable  values  may  contain  spaces  and should not be quoted.  The pcp.env script automatically quotes all variable values from the
	   character immediately following the = through to the end of the line.

       For further details and an explanation of the use of each variable, see the comments in the /etc/pcp.conf file itself.

ENVIRONMENT

       The PCP_CONF environment variable specifies an alternative path to the pcp.conf file.

SEE ALSO

       PCPIntro(1), PCPIntro(3), PMAPI(3), pmGetConfig(3) and pcp.env(5).

Performance Co-Pilot							PCP							       PCP.CONF(5)