Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to extract data from a huge file?
Top Forums Shell Programming and Scripting How to extract data from a huge file? Post 302159593 by matrixmadhan on Friday 18th of January 2008 03:45:55 AM
Old 01-18-2008
The following code snippet, logically splits the data file based on the main tag

Code:
#! /opt/third-party/bin/perl

open(FILE, "<", "a");

while(<FILE>) {
  chomp;
  print "$_\n" if( $pr == 1 );
  if( /<dublin/ ) {
    $pr = 1;
    print "$_\n";
    next;
  }
  elsif ( /<\/dublin/ ) {
    print "\n\n\n";
    $pr = 0;
  }
}

close(FILE);

exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc... (2 Replies)
Discussion started by: Marcor
2 Replies

3. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

4. Shell Programming and Scripting

How to extract a subset from a huge dataset

Hi, All I have a huge file which has 450G. Its tab-delimited format is as below x1 A 50020 1 x1 B 50021 8 x1 C 50022 9 x1 A 50023 10 x2 D 50024 5 x2 C 50025 7 x2 F 50026 8 x2 N 50027 1 : : Now, I want to extract a subset from this file. In this subset, column 1 is x10, column 2 is... (3 Replies)
Discussion started by: cliffyiu
3 Replies

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

8. Shell Programming and Scripting

Extract few content from a huge list of files

I have a huge list of files (about 300,000) which have a pattern like this. .I 1 .U 87049087 .S Am J Emerg .M Allied Health Personnel/*; Electric Countershock/*; .T Refibrillation managed by EMT-Ds: .P ARTICLE. .W Some patients converted from ventricular fibrillation to organized... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is... (9 Replies)
Discussion started by: kartikirans
9 Replies

LEARN ABOUT DEBIAN

plhist

PLHIST(3plplot) 						    PLplot API							   PLHIST(3plplot)

NAME

       plhist - Plot a histogram from unbinned data

SYNOPSIS

       plhist(n, data, datmin, datmax, nbin, opt)

DESCRIPTION

       Plots a histogram from n data points stored in the array data.  This routine bins the data into nbin bins equally spaced between datmin and
       datmax, and calls plbin(3plplot) to draw the resulting histogram.  Parameter opt allows, among other things, the  histogram  either  to	be
       plotted in an existing window or causes plhist(3plplot) to call plenv(3plplot) with suitable limits before plotting the histogram.

       Redacted form: plhist(data, datmin, datmax, nbin, opt)

       This function is used in example 5.

ARGUMENTS

       n (PLINT, input)
	      Number of data points.

       data (PLFLT *, input)
	      Pointer to array with values of the n data points.

       datmin (PLFLT, input)
	      Left-hand edge of lowest-valued bin.

       datmax (PLFLT, input)
	      Right-hand edge of highest-valued bin.

       nbin (PLINT, input)
	      Number of (equal-sized) bins into which to divide the interval xmin to xmax.

       opt (PLINT, input)
	      Is  a  combination  of  several flags: opt=PL_HIST_DEFAULT: The axes are automatically rescaled to fit the histogram data, the outer
	      bins are expanded to fill up the entire x-axis, data outside the given extremes are assigned to the outer  bins  and  bins  of  zero
	      height  are  simply  drawn.   opt=PL_HIST_NOSCALING|...:	The existing axes are not rescaled to fit the histogram data, without this
	      flag, plenv(3plplot) is called to set the world coordinates.  opt=PL_HIST_IGNORE_OUTLIERS|...: Data outside the given  extremes  are
	      not  taken  into account. This option should probably be combined with opt=PL_HIST_NOEXPAND|..., so as to properly present the data.
	      opt=PL_HIST_NOEXPAND|...: The outer bins are drawn with equal size as the ones  inside.	opt=PL_HIST_NOEMPTY|...:  Bins	with  zero
	      height are not drawn (there is a gap for such bins).

AUTHORS

       Geoffrey  Furnish  and  Maurice LeBrun wrote and maintain PLplot.  This man page was automatically generated from the DocBook source of the
       PLplot documentation, maintained by Alan W. Irwin and Rafael Laboissiere.

SEE ALSO

       PLplot documentation at http://plplot.sourceforge.net/resources.

								   August, 2012 						   PLHIST(3plplot)
All times are GMT -4. The time now is 03:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy