File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this Post: 303025149

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number...

2. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab...

3. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and...

4. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA...

5. Shell Programming and Scripting

Splitting the Huge file into several files...

Hi I have to write a script to split the huge file into several pieces. The file columns is | pipe delimited. The data sample is as: 6625060|1420215|07308806|N|20100120|5572477081|+0002.79|+0000.00|0004|0001|.........

6. Shell Programming and Scripting

Problem running Perl Script with huge data files

Hello Everyone, I have a perl script that reads two types of data files (txt and XML). These data files are huge and large in number. I am using something like this : foreach my $t (@text) { open TEXT, $t or die "Cannot open $t for reading: $!\n"; while(my $line=<TEXT>){ ...

7. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . .

8. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment...

9. UNIX for Dummies Questions & Answers

File comparison of huge files

Hi all, I hope you are well. I am very happy to see your contribution. I am eager to become part of it. I have the following question. I have two huge files to compare (almost 3GB each). The files are simulation outputs. The format of the files are as below For clear picture, please see...

10. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ......

LEARN ABOUT DEBIAN

colorgccrc

COLORGCCRC(5)							File Formats Manual						     COLORGCCRC(5)

NAME

       colorgccrc - configuration file for colorgcc

DESCRIPTION

       A colorgccrc configuration file is used to configure the highlighting of the compiler output from colorgcc.

SYNTAX

       Each  line  consists  of  a  keyword  designating  a configuration variable.  The keyword is followed by `:' and then one or several values
       (depending on the keyword).  Lines beginning with a hash mark `#' are comments.

CONFIGURATION VARIABLES

       g++ | gcc | c++ | cc | g77 | gcj | gnat | gpc
	      Specifies the paths to the compilers.  Takes one value; a path to the compiler.

       nocolor
	      Specifies what terminal types colorization should be disabled on.  Takes one or several values, separated by whitespace.

       srcColor
	      Specifies the highlighting attributes source-code should be given.  Takes one or several color attributes.  See  the  section  COLOR
	      ATTRIBUTES for more information.

       introColor
	      Specifies  the  highlighting  attributes	for normal compiler output.  Takes one or several color attributes.  See the section COLOR
	      ATTRIBUTES for more information.

       warningFileNameColor | errorFileNameColor
	      Specifies the highlighting attributes for the filename in a  warning  or	an  error,  respectively.   Takes  one	or  several  color
	      attributes.  See the section COLOR ATTRIBUTES for more information.

       warningNumberColor | errorNumberColor
	      Specifies  the  highlighting  attributes	for  the  line-number  in a warning or an error, respectively.	Takes one or several color
	      attributes.  See the section COLOR ATTRIBUTES for more information.

       warningMessageColor | errorMessageColor
	      Specifies the highlighting attributes for the message-text in a warning or an error,  respectively.   Takes  one	or  several  color
	      attributes.  See the section COLOR ATTRIBUTES for more information.

COLOR ATTRIBUTES

       The following attributes are valid for highlighting.

       clear, reset

       bold, underline, underscore, blink, reverse, concealed

       black, red, green, yellow, blue, magenta, cyan, white

       on_black, on_red, on_green, on_yellow, on_blue, on_magenta, on_cyan, on_white

SEE ALSO

       gcc(1), colorgcc(1)

HISTORY

       Jan 15 2003: Initial version of this manual-page.

REPORTING BUGS

       Report bugs to <jmoyers@geeks.com>

AUTHORS

       Jamie Moyers <jmoyers@geeks.com> is the author of colorgcc.

       This  manual  page  was	written  by Joe Wreschnig <piman@sacredchao.net>, and modified by David Weinehall <tao@debian.org>, for the Debian
       GNU/Linux system (but may be used by others).

COPYRIGHT

       Copyright (C) 2003 Jamie Moyers
       This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
       LAR PURPOSE.

								   Jan 15, 2003 						     COLORGCCRC(5)