Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to extract data from a huge file?
Top Forums Shell Programming and Scripting How to extract data from a huge file? Post 302159593 by matrixmadhan on Friday 18th of January 2008 03:45:55 AM
Old 01-18-2008
The following code snippet, logically splits the data file based on the main tag

Code:
#! /opt/third-party/bin/perl

open(FILE, "<", "a");

while(<FILE>) {
  chomp;
  print "$_\n" if( $pr == 1 );
  if( /<dublin/ ) {
    $pr = 1;
    print "$_\n";
    next;
  }
  elsif ( /<\/dublin/ ) {
    print "\n\n\n";
    $pr = 0;
  }
}

close(FILE);

exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Shell Programming and Scripting

How to extract a piece of information from a huge file

Hello All, I need some assistance to extract a piece of information from a huge file. The file is like this one : database information ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc ccccccccccccccccc os information cccccccccccccccccc cccccccccccccccccc... (2 Replies)
Discussion started by: Marcor
2 Replies

3. Shell Programming and Scripting

insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract if i use sed, i have to redirect the output to a seperate file like sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat the same is true for awk and... (10 Replies)
Discussion started by: deepaktanna
10 Replies

4. Shell Programming and Scripting

How to extract a subset from a huge dataset

Hi, All I have a huge file which has 450G. Its tab-delimited format is as below x1 A 50020 1 x1 B 50021 8 x1 C 50022 9 x1 A 50023 10 x2 D 50024 5 x2 C 50025 7 x2 F 50026 8 x2 N 50027 1 : : Now, I want to extract a subset from this file. In this subset, column 1 is x10, column 2 is... (3 Replies)
Discussion started by: cliffyiu
3 Replies

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

Extract header data from one file and combine it with data from another file

Hi, Great minds, I have some files, in fact header files, of CTD profiler, I tried a lot C programming, could not get output as I was expected, because my programming skills are very poor, finally, joined unix forum with the hope that, I may get what I want, from you people, Here I have attached... (17 Replies)
Discussion started by: nex_asp
17 Replies

8. Shell Programming and Scripting

Extract few content from a huge list of files

I have a huge list of files (about 300,000) which have a pattern like this. .I 1 .U 87049087 .S Am J Emerg .M Allied Health Personnel/*; Electric Countershock/*; .T Refibrillation managed by EMT-Ds: .P ARTICLE. .W Some patients converted from ventricular fibrillation to organized... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

10. UNIX for Advanced & Expert Users

File comaprsons for the Huge data files ( around 60G) - Need optimized and teh best way to do this

I have 2 large file (.dat) around 70 g, 12 columns but the data not sorted in both the files.. need your inputs in giving the best optimized method/command to achieve this and redirect the not macthing lines to the thrid file ( diff.dat) File 1 - 15 columns File 2 - 15 columns Data is... (9 Replies)
Discussion started by: kartikirans
9 Replies

LEARN ABOUT OPENSOLARIS

soladdapp

soladdapp(1M)						  System Administration Commands					     soladdapp(1M)

NAME

       soladdapp - add an application to the Solstice application registry

SYNOPSIS

       /usr/snadm/bin/soladdapp [-r registry] -n name -i icon -e executable
	    [args]

DESCRIPTION

       soladdapp  adds	an  application  to  the  Solstice  application  registry. After it is added, the application is displayed in the Solstice
       Launcher main window (see solstice(1M)).

OPTIONS

       -r registry	Define the full path name of the Solstice registry file.

       -n name		Define the name of the tool to be registered.

       -i icon		Define the full path name of the tool icon.

       -e executable	Define the full path name of the tool.

       args		Specify any arguments to use with the tool.

       When executed without options,  soladdapp uses  /opt/SUNWadm/etc/.solstice_registry (the default registry path).

RETURN VALUES

       0    on success

       1    on failure

       2    if the registry is locked

       3    if the entry is a duplicate.

EXAMPLES

       Example 1 A sample display of the soladdapp command.

       The following adds an application called Disk Manager to the Solstice application registry for display in the Solstice Launcher	main  win-
       dow.

	 # soladdapp -r /opt/SUNWadm/etc/.solstice_registry -n "Disk Manager"
	 -i /opt/SUNWdsk/etc/diskmgr.xpm -e /opt/SUNWdsk/bin/diskmgr

FILES

       /opt/SUNWadm/etc/.solstice_registry

	   The default registry path.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWsadml 		   |
       +-----------------------------+-----------------------------+

SEE ALSO

       soldelapp(1M), solstice(1M), attributes(5)

NOTES

       Globally registered applications are used by local and remote users sharing the software in a particular  /opt directory. They can be added
       only using soladdapp.

SunOS 5.11							    15 Sep 1995 						     soladdapp(1M)
All times are GMT -4. The time now is 03:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy