Checking the Column values in a file. Post: 302543982

Sponsored Content

Top Forums Shell Programming and Scripting Checking the Column values in a file. Post 302543982 by filter on Tuesday 2nd of August 2011 03:23:19 PM

08-02-2011

Registered User

Checking the Column values in a file.

Hi All,

I have a file that has ~2.9Millions lines with 32 columns respectively.
The columns numbers 23,27 are the primary Keys for the file.
The fields are delimited by TAB.

I need to check the condition If
Column number: 20 is NOT NULL
Column number: 21 is not 0
Column number: 22 is not 0

If the above is TRUE (Satisfied) Then Write the entire line to an OUTFILE..
Else
Ignore the line and proceed further.

In order to check the above, I thought the following:

Since I have the primary keys for the file i.e.23,27 , I would load the file into a hash and then check the conditions i.e.
Column number: 20 is NOT NULL
Column number: 21 is not 0
Column number: 22 is not 0

i.e.

Code:

$FIELD_SEPARATOR = '    ';
$FIELD_NUMBER_LIST = $[                                   -- (23,27)
my @primary_set= split(/,/, $field_number_list);
my $line = '';
my %hash;

open(FILE, "<$old_file") or die "Can't open file $old_file";
my @file = <FILE>;
close(FILE);

my @vals;
foreach (@file) {
$line = $_;
@vals = split(/$field_separator/, $line);

   my $key;
   my $primary = "";
   foreach $key (@primary_set) {
      $primary = $primary.$vals[$key];
   }

   $hash{$primary} = $line;
}

while (<FILE>) {
$line = $_;
@vals = split(/$field_separator/, $line);
    my $key;
    my $primary = "";
    foreach $key (@primary_set) {
        $primary = $primary.$vals[$key];
    }
 
<<<< Here I need to check the condition for the Columns i.e.
Column number: 20 is NOT NULL
 Column number: 21 is not 0
 Column number: 22 is not 0
>>>>>>>>>>>>>

Could someone please help me out how to check the condition for a particular column. and how do I check for NOT NULL.

Really appreciate your thoughts.

Last edited by pludi; 08-02-2011 at 04:47 PM..

filter

View Public Profile for filter

Find all posts by filter

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a...

2. UNIX for Dummies Questions & Answers

Create file with column values

Hi, I have a data file looks like the following ID STARTDATE ENDDATE 101 20090520 20090521 102 20090521 20090522 103 20090522 20090523 104 20090523 20090524 105 20090524 20090525 106 20090525 20090526 107 ...

3. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the...

4. UNIX for Dummies Questions & Answers

Replace values in a specified column of a file

Hello, I have a file with four columns and I would like to replace values in the second column only. An arbitrary example is: 100 A 105 B 200 B 205 C 300 C 305 D 400 D 405 E 500 E 505 F I need to replace the second column as shown below: ...

5. UNIX for Dummies Questions & Answers

replace a column with values from another file

Dear all, I have a file1.pdb in pdb format and a dat file2 containing values, corresponding to the atoms in the pdb file. these values (file2.dat) need to be in the column instead of the 0.00 (file1) values for each atom in file1.pdb .(the red values must be replaced by the blue ones,in order)...

6. UNIX for Dummies Questions & Answers

shift values in one column as header for values in another column

Hi Gurus, I have a tab separated text file with two columns. I would like to make the first column values as headings for the second column values. Ex. >value1 subjects >value2 priorities >value3 requirements ...etc and I want to have a file >value1 subjects >value2 priorities...

7. Shell Programming and Scripting

Adding column values in a file

Hi, I am having a file in the following format. for aaaa 1111 1234 2222 3434 for bbbb 1111 3434.343 2222 2343 for cccc 3333 2343.343 4444 89000 for dddd 1111 5678.343 2222 890.3 aaaa 2343.343 bbbb 34343.343

8. Shell Programming and Scripting

Remove the values from a certain column without deleting the Column name in a .CSV file

9. Shell Programming and Scripting

Replace column values from other file

I have one file as it has the following format File1 S No Site IP Address 1 Australia 192.168.0.1/26 2 Australia 192.168.0.2/26 3 Australia 192.168.0.3/26 I need awk/sed command to replace the column2 value ( under Site) with some other...

LEARN ABOUT HPUX

quot_vxfs

quot_vxfs(1M)															     quot_vxfs(1M)

NAME

       quot_vxfs: quot - summarize ownership on a VxFS file system

SYNOPSIS

       filesystem ...

DESCRIPTION

       displays  the  number of 1024-byte blocks in the specified VxFS filesystem that are currently owned by each user.  filesystem is either the
       name of the directory on which the file system is mounted or the name of the device containing the file system.

   Options
       recognizes the following options:

	      Specifies file system type.

	      Generate a report for all mounted file systems.

	      Report file system size instead of user statistics.
			The option generates histogram statistics in 3-column format:

			Column 1:   File size in blocks.  Sizes are listed in ascending order up to 499 blocks per file.  Files occupying  499	or
				    more  blocks  are counted together on a single line as 499-block files, but column 3 is based on actual number
				    of blocks occupied.

			Column 2:   Number of files of size indicated in column 1.

			Column 3:   Cumulative total blocks occupied by files counted in current plus all preceding lines.

			The option overrides the and options.

	      Display the number of files and space occupied by each user.

	      Calculate the number of blocks in the file based on file size
			rather than actual blocks allocated.  The option does not account for sparse files (files with holes in them).

	      Accept	ncheck(1M) data as input.  The following pipeline will produce a list of all files and their owners:

	      Display three columns containing the number of blocks
			not accessed in the last 30, 60, and 90 days.

	      Validate the command line options, but do not execute the command.
			If the options specified are valid, echoes the complete command line.  If the options specified are not valid,	it  prints
			an error message.

   Operands
       recognizes the following operand:

	      filesystem
			Name of the directory on which the VxFS file system is mounted or the name of the device containing the file system..

EXAMPLES

       The following examples show the output of using various options.

AUTHOR

       Disk Quotas were developed by the University of California, Berkeley, Sun Microsystems, Inc., and HP.

FILES

       Mounted file system table
       Password file (contains user names).

SEE ALSO

       du(1), find(1), ls(1), fstyp(1M), mount(1M), ncheck(1M), quot(1M), repquota(1M), quota(5).

																     quot_vxfs(1M)