Need optimized awk/perl/shell to give the statistics for the Large delimited file Post: 303023280

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Could someone give me an example of awk accessing array defined in Korn Shell?

As per title and much apprecieated!

2. UNIX for Dummies Questions & Answers

Trim String in 3rd Column in Tab Delimited File...SED/PERL/AWK?

Hey Everybody, I am having much trouble figuring this out, as I am not really a programmer..:mad: Datafile.txt Column0 Column1 Column2 ABC DEF xxxGHI I am running using WGET on a cronjob to grab a datafile, but I need to cut the first three characters from...

3. Shell Programming and Scripting

Large pipe delimited file that I need to add CR/LF every n fields

I have a large flat file with variable length fields that are pipe delimited. The file has no new line or CR/LF characters to indicate a new record. I need to parse the file and after some number of fields, I need to insert a CR/LF to start the next record. Input file ...

4. Shell Programming and Scripting

Extracting a portion of data from a very large tab delimited text file

Hi All I wanted to know how to effectively delete some columns in a large tab delimited file. I have a file that contains 5 columns and almost 100,000 rows 3456 f g t t 3456 g h 456 f h 4567 f g h z 345 f g 567 h j k lThis is a very large data file and tab delimited. I need...

5. Shell Programming and Scripting

Script Optimization - large delimited file, for loop with many greps

Since there are approximately 75K gsfiles and hundreds of stfiles per gsfile, this script can take hours. How can I rewrite this script, so that it's much faster? I'm not as familiar with perl but I'm open to all suggestions. ls file.list>$split for gsfile in `cat $split`; do csplit...

6. Shell Programming and Scripting

Awk getting statistics of a grid file,

Hi , I have the following file which is basically a grid (has more than 100000 rows) LLL1 PPP1 LLL1 PPP2 LLL1 PPP3 ............... LLL1 5500 ..... LLL2 PPP1 LLL2 PPP2 LLL2 PPP3 ............... LLL1 5500 ..... L100 PPP1 L100 PPP2 L100 PPP3 ............... 2100 5500...

7. Shell Programming and Scripting

awk read one delimited file, search another delimited file

Hello folks, I have another doozy. I have two files. The first file has four fields in it. These four fields map to different locations in my second file. What I want to do is read the master file (file 2 - 23 fields) and compare each line against each record in file 1. If I get a match in all four...

8. Shell Programming and Scripting

Removing dupes within 2 delimited areas in a large dictionary file

Hello, I have a very large dictionary file which is in text format and which contains a large number of sub-sections. Each sub-section starts with the following header : #DATA #VALID 1 and ends with a footer as shown below #END The data between the Header and the Footer consists of...

9. Shell Programming and Scripting

Perl script give answers by file

Hi, I am new in perl. I am running a perl installation script, its asking for paths and so many inputs. Can we provide that info by any file. so i can avoid the interactive installation.

LEARN ABOUT CENTOS

gensprep

gensprep(8)							 ICU 50.1.2 Manual						       gensprep(8)

NAME

       gensprep - compile StringPrep data from files filtered by filterRFC3454.pl

SYNOPSIS

       gensprep [ -h, -?, --help ] [ -v, --verbose ] [ -c, --copyright ] [ -s, --sourcedir source ] [ -d, --destdir destination ]

DESCRIPTION

       gensprep reads filtered RFC 3454 files and compiles their information into a binary form.  The resulting file, <name>.icu, can then be read
       directly by ICU, or used by pkgdata(8) for incorporation into a larger archive or library.

       The files read by gensprep are described in the FILES section.

OPTIONS

       -h, -?, --help
	      Print help about usage and exit.

       -v, --verbose
	      Display extra informative messages during execution.

       -c, --copyright
	      Include a copyright notice into the binary data.

       -s, --sourcedir source
	      Set the source directory to source.  The default source directory is specified by the environment variable ICU_DATA.

       -d, --destdir destination
	      Set the destination directory to destination.  The default destination directory is specified by the environment variable ICU_DATA.

ENVIRONMENT

       ICU_DATA  Specifies the directory containing ICU data. Defaults to /usr/share/icu/50.1.2/.  Some tools in ICU depend on the presence of the
		 trailing slash. It is thus important to make sure that it is present if ICU_DATA is set.

FILES

       The  following files are read by gensprep and are looked for in the source /misc for rfc3454_*.txt files and in source /unidata for Normal-
       izationCorrections.txt.

       rfc3453_A_1.txt	   Contains the list of unassigned codepoints in Unicode version 3.2.0....

       rfc3454_B_1.txt	   Contains the list of code points that are commonly mapped to nothing....

       rfc3454_B_2.txt	   Contains the list of mappings for casefolding of  code points when Normalization form NFKC is specified....

       rfc3454_C_X.txt	   Contains the list of code points that are prohibited for IDNA.

       NormalizationCorrections.txt
			   Contains the list of code points whose normalization has changed since Unicode Version 3.2.0.

VERSION

       50.1.2

COPYRIGHT

       Copyright (C) 2000-2002 IBM, Inc. and others.

SEE ALSO

       pkgdata(8)

ICU MANPAGE
							   18 March 2003						       gensprep(8)