Sponsored Content
Top Forums Shell Programming and Scripting Extraction of various lines from a hugh file Post 302191801 by era on Monday 5th of May 2008 02:24:24 AM
Old 05-05-2008
As a matter of shell coding style, the parentheses are completely unnecessary, and stuff in backticks works badly if there's a file name with spaces in it.

I don't see why you couldn't use that shell script to wrap the Perl code; there's nothing much there which Perl does better than the shell, other than not having to read the country file over and over again (but you could optimize that in the shell script, too). But anyway, here goes. I'm afraid this is completely untested.

Code:
#!/usr/bin/perl

die "Usage: $0 dir yearfile countryfile" unless (@ARGV == 3);

open (Y, "$ARGV[1]") || die "$0: Could not open $ARGV[1]: $!\n";
open (C, "$ARGV[2]") || die "$0: Could not open $ARGV[2]: $!\n";
my @countries = <C>;
close C;
while ($year = <Y>) {
  for $country (@countries) {
    handle ("$ARGV[0]/$year/$country");
  }
}
close Y;

sub handle {
  my ($file) = @_;
  open (F, $file) || die "$0: Could not open $file: $!\n";
  while (<F>) {
    if (/^\[Querying/) {
      print; @wanted = qw(OrgName NetRange inetnum descr owner Country);
      $wanted = &wanted(@wanted);
    }
    if ($wanted && $_  =~ m/$wanted/i) {
      print;
      @wanted = grep { $_ ne $1 } @wanted;
      $wanted = @wanted ? &wanted(@wanted) : "";
    }
    close F;
  }
}  
sub wanted {
  return "^(" . join ("|", map { quotemeta $_ } @_) . "):";
}

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

help on file extraction

Hello, Im trying to extract a portion of a big file. Using unique pattern /occurrence , (ex. loginname1,logoff and loginname2,logoff ), I like to print the lines that contain the patterns and the lines between them. Also, create a file for every login occurrence. Thanks for everyone's... (1 Reply)
Discussion started by: apalex
1 Replies

2. UNIX for Dummies Questions & Answers

Flat File Extraction

Hi all, I'm new in the unix environment. I'm having a challenge in extracting data from a flat file and convert it to a CSV file format or I should be able to open it with MS Excel. The input data in my flat file looks like this: AV00001001155000063637143326711145412082EM SITHOLE... (3 Replies)
Discussion started by: Mthimbana
3 Replies

3. Shell Programming and Scripting

Data Extraction From a File

Hi All, I have a requirement where I have to search the file with some text say "Exception". This exception word can be repeated for more then 10 times. Suppose the "Exception" word is repeated at line numbers say x=10, 50, 60, 120. Now I want to extract all the lines starting from x-5 to... (3 Replies)
Discussion started by: rrangaraju
3 Replies

4. Shell Programming and Scripting

File extraction without awk

Hello everybody, Here is my problem : I cannot find a way to extract data from a particular file and more precisely I cannot extract the result of my awk script to an external file because I am currently working on HP-UX. I would like a simple script (without awk) which asks for a date like... (4 Replies)
Discussion started by: freyr
4 Replies

5. Shell Programming and Scripting

File Extraction

Hi, I have three files as below: AA.DAT20110505063903.Z AA.DAT20110405062903.Z AA.DAT20110305061903.Z All the above files are appended with Date and timestamp in compressed format. I need to extract AA.DAT20110505063903.Z(which is the latest file) from one server and uncompress it... (2 Replies)
Discussion started by: pyaranoid
2 Replies

6. Shell Programming and Scripting

problem with file content extraction

I need to extract some content of a file. Example file abc vi abc ooooooooo bbbbbbbbb vvv 1234 5 vvv 6789 3 xxxxxxxxxx xxxxxxxxxx i want to extract only the following content from file abc and store in another file say temp. 1234 5 6789 3 what should be my approach? (2 Replies)
Discussion started by: priya_ag04
2 Replies

7. Shell Programming and Scripting

data extraction from a file

Hi Freinds, I have a file1.txt in the following format File1.txt I want to get 2 files from the above file filextra.txt should have the lines which are ending with "<" and remaining lines in the filecompare.txt file. Please help. (3 Replies)
Discussion started by: i150371485
3 Replies

8. Shell Programming and Scripting

FILE_ID extraction from file name and save it in CSV file after looping through each folders

FILE_ID extraction from file name and save it in CSV file after looping through each folders My files are located in UNIX Server, i want to extract file_id and file_name from each file .and save it in a CSV file. How do I do that? I have folders in unix environment, directory structure is... (15 Replies)
Discussion started by: princetd001
15 Replies

9. Shell Programming and Scripting

CSV file data extraction

Hi I am writing a shell script to parse a CSV file , in which i am facing a problem to separate the columns . Could some one help me with it. IN301330/00001 pvavan kumar limited xyz@ttccpp.com IN302148/00002 PRECIOUS SECURITIES (P) LTD viash@yahoo.co.in IN300239/00000 CENTRE india... (8 Replies)
Discussion started by: nanduri
8 Replies

10. UNIX for Beginners Questions & Answers

Extraction of .gz file using 7zip fails

Hi, My target is to send a file created by Unix process to myself as an excel file. So I have used the below commands to achieve it. tr -d '\t' < PROGRAM_CREATED_FILE | sed -e 's/\\//g' | awk 'BEGIN{FS=">"; OFS="\t"} '{$1=$1}1' > file.xls gzip -9 file.xls echo "test mail" | sendxchange -a... (3 Replies)
Discussion started by: PikK45
3 Replies
Inline::Files(3)					User Contributed Perl Documentation					  Inline::Files(3)

NAME
Inline::Files - Multiple virtual files at the end of your code VERSION
This document describes version 0.68 of Inline::Files, released July 23, 2011. SYNOPSIS
use Inline::Files; my Code $here; # etc. # etc. # etc. __FOO__ This is a virtual file at the end of the data __BAR__ This is another virtual file __FOO__ This is yet another such file WARNING
It is possible that this module may overwrite the source code in files that use it. To protect yourself against this possibility, you are strongly advised to use the "-backup" option described in "Safety first". This module is still experimental. Regardless of whether you use "-backup" or not, by using this module you agree that the authors will b<under no circumstances> be responsible for any loss of data, code, time, money, or limbs, or for any other disadvantage incurred as a result of using Inline::Files. DESCRIPTION
Inline::Files generalizes the notion of the "__DATA__" marker and the associated "<DATA>" filehandle, to an arbitrary number of markers and associated filehandles. When you add the line: use Inline::Files; to a source file you can then specify an arbitrary number of distinct virtual files at the end of the code. Each such virtual file is marked by a line of the form: __SOME_SYMBOL_NAME_IN_UPPER_CASE__ The following text -- up to the next such marker -- is treated as a file, whose (pseudo-)name is available as an element of the package array @SOME_SYMBOL_NAME_IN_UPPER_CASE. The name of the first virtual file with this marker is also available as the package scalar $SOME_SYMBOL_NAME_IN_UPPER_CASE. The filehandle of the same name is magical -- just like "ARGV" -- in that it automatically opens itself when first read. Furthermore -- just like "ARGV" -- the filehandle re-opens itself to the next appropriate virtual file (by "shift"-ing the first element of @SOME_SYMBOL_NAME_IN_UPPER_CASE into $SOME_SYMBOL_NAME_IN_UPPER_CASE) whenever it reaches EOF. So, just as with "ARGV", you can treat all the virtual files associated with a single symbol either as a single, multi-part file: use Inline::Files; while (<FILE>) { print "$FILE: $_"; } __FILE__ File 1 here __FILE__ File 2 here __OTHER_FILE__ Other file 1 __FILE__ File 3 here or as a series of individual files: use Inline::Files; foreach $filename (@FILE) { open HANDLE, $filename; print "<<$filename>> "; while (<HANDLE>) { print; } } __FILE__ File 1 here __FILE__ File 2 here __OTHER_FILE__ Other file 1 __FILE__ File 3 here Note that these two examples completely ignore the lines: __OTHER_FILE__ Other file 1 which would be accessed via the "OTHER_FILE" filehandle. Unlike "<ARGV>"/@ARGV/$ARGV, Inline::Files also makes use of the hash associated with an inline file's symbol. That is, when you create an inline file with a marker "__WHATEVER__", the hash %WHATEVER will contain information about that file. That information is: $WHATEVER{file} The name of the disk file in which the inlined "__WHATEVER__" files were defined; $WHATEVER{line} The line (starting from 1) at which the current inline "__WHATEVER__" file being accessed by "<WHATEVER>" started. $WHATEVER{offset} The byte offset (starting from 0) at which the current inline "__WHATEVER__" file being accessed by "<WHATEVER>" started. $WHATEVER{writable} Whether the the current inline file being accessed by "<WHATEVER>" is opened for output. The hash and its elements are read-only and the entry values are only meaningful when the corresponding filehandle is open. Writable virtual files If the source file that uses Inline::Files is itself writable, then the virtual files it contains may also be opened for write access. For example, here is a very simple persistence mechanism: use Inline::Files; use Data::Dumper; open CACHE or die $!; # read access (uses $CACHE to locate file) eval join "", <CACHE>; close CACHE or die $!; print "$var was '$var' "; while (<>) { chomp; $var = $_; print "$var now '$var' "; } open CACHE, ">$CACHE" or die $!; # write access print CACHE Data::Dumper->Dump([$var],['var']); close CACHE or die $!; __CACHE__ $var = 'Original value'; Unlike "ARGV", if a virtual file is part of a writable file and is automagically opened, it is opened for full read/write access. So the above example, could be even simpler: use Inline::Files; use Data::Dumper; eval join "", <CACHE>; # Automagically opened print "$var was '$var' "; while (<>) { chomp; $var = $_; print "$var now '$var' "; } seek CACHE, 0, 0; print CACHE Data::Dumper->Dump([$var],['var']); __CACHE__ $var = 'Original value'; In either case, the original file is updated only at the end of execution, on an explicit "close" of the virtual file's handle, or when "Inline::Files::Virtual::vf_save" is explicitly called. Creating new Inline files on the fly. You can also open up new Inline output files at run time. Simply use the open function with a valid new Inline file handle name and no file name. Like this: use Inline::Files; open IFILE, '>'; print IFILE "This line will be placed into a new Inline file "; print IFILE "which is marked by '__IFILE__' "; Safety first Because Inline::Files handles are often read-write, it's possible to accidentally nuke your hard-won data. But Inline::Files can save you from yourself. If Inline::Files is loaded with the "-backup" option: use Inline::Files -backup; then the source file that uses it is backed up before the inline files are extracted. The backup file is the name of the source file with the suffix ".bak" appended. You can also specify a different name for the backup file, by associating that name with the "-backup" flag: use Inline::Files -backup => '/tmp/sauve_qui_peut'; SEE ALSO
The Inline::Files::Virtual module The Filter::Util::Call module BUGS ADDED BY Alberto Simoes (ambs@cpan.org) UNWITTING PAWN OF AN AUTHOR
Damian Conway (damian@conway.org) EVIL MASTERMIND BEHIND IT ALL
Brian Ingerson (INGY@cpan.org) COPYRIGHT
Copyright (c) 2001-2009. Damian Conway. All rights reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See http://www.perl.com/perl/misc/Artistic.html perl v5.16.3 2011-07-23 Inline::Files(3)
All times are GMT -4. The time now is 07:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy