Getting the lines with nth column non-null


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Getting the lines with nth column non-null
# 8  
Old 02-09-2016
Hi Don and Ravinder,

Indeed the command:
Code:
gzip -dc *.gz | /usr/xpg4/bin/awk -F';' '$252!=""'>results.txt

works just fine!

Thank you very much!
Nenad.
# 9  
Old 02-09-2016
Hi.

On some instances of Solaris, one might find gawk:
Code:
OS, ker|rel, machine: SunOS, 5.11, i86pc
Distribution        : Solaris 11.2 X86
gawk GNU Awk 3.1.8

Best wishes ... cheers, drl
# 10  
Old 02-09-2016
Hi drl,

Unfortunately gawk is not available on my side.
Code:
bash-3.2$ gawk
bash: gawk: command not found
bash-3.2$

Cheers,
nenad.
# 11  
Old 02-10-2016
Hi.

There is a collection of perl text utilities at The Missing Textutils . This is a demonstration of one of them, colgrep, search columns (fields) for a regular expression match. The listing of the perl code is included in the output.
Code:
#!/usr/bin/env bash

# @(#) s1       Demonstrate searching fields (columns), colgrep.
# Missing textutils, verified Wed Feb 10 16:04:12 CST 2016
# http://www1.cuni.cz/~obo/textutils/

program=./colgrep

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C align $program

FILE=${1-data1}

pl " Input data file human readable $FILE:"
align -s t $FILE

pl " Results:"
$program 4 --re='.' $FILE

pl " Code for colgrep:"
cat $program

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.3 (jessie) 
bash GNU bash 4.3.30
align 1.7.0
./colgrep - ( local: RepRev 1.1, ./colgrep, 2016-02-10 )

-----
 Input data file human readable data1:
AtZv    qChk    QjxK    nXYM    MdLp    SMar
One     aqeP    Wpkl            cLJO    oeba
gHld    hUFv    dKcr    PKAi    xPnz    txAk
Two     zrHl    mpcg            ZAia    wMLN
hwey    ejyu    qzKA    Pcom    Jewb    mrGW
YfMp    PBUs    YUdo    itUy    vwHL    JVih

-----
 Results:
Searching cols 4 for ..
AtZv    qChk    QjxK    nXYM    MdLp    SMar
gHld    hUFv    dKcr    PKAi    xPnz    txAk
hwey    ejyu    qzKA    Pcom    Jewb    mrGW
YfMp    PBUs    YUdo    itUy    vwHL    JVih

-----
 Code for colgrep:
#!/usr/bin/env perl

# @(#) colgrep  Match specific columns with perl expression.
# $Id: colgrep,v 1.1 2011/11/23 04:01:03 drl Exp drl $

# The Missing Textutils, Ondrej Bojar, obo@cuni.cz
# http://www.cuni.cz/~obo/textutils
#
# 'colgrep' is just like grep (but using Perl regular expressions, not grep
# ones), but it checks only specified tab-delimited columns of the file.
#
# $Id: colgrep,v 1.1 2011/11/23 04:01:03 drl Exp drl $
#

use Getopt::Long;
use strict;

my $inverse = 0;
my $extrare = 0;
my $skip = 0;
my $help = 0;
GetOptions(
  "inverse" => \$inverse,
  "help" => \$help,
  "skip=i" => \$skip,
  're=s'=>\$extrare,
) or exit 1;

my $cols=shift;
my $re;
if ($extrare || $extrare eq "0") {
  $re = $extrare;
} else {
  $re=shift;
}

if ($help || !$cols || !($re || $re eq "0")) {
  print STDERR "usage: colgrep 1,2,3 --re=\"re_to_lookup\"
Options:
  --inverse   ... print lines where the RE was not found
  --skip=N    ... skip first N lines
";
  exit 1;
}

my @cols = split /,/, $cols;
my %cols;
foreach my $col (@cols) {
  $cols{$col-1} = 1;
}

print STDERR "Searching cols $cols for $re.\n";

while ($skip > 0) {
  my $l = <>;
  print $l;
  $skip --;
}


while (<>) {
  my @line = split /\t/;
  my $pattern = "";
  foreach my $col (sort keys %cols) {
    $pattern .= "$line[$col]\t";
  }
  chop $pattern;
  my $hit = ($pattern =~ /$re/);
  print if ($hit && !$inverse) || (!$hit && $inverse);
}

The data file separator is TAB, but that can be changed in the code for colgrep if necessary.

So in this output, we do not expect to see the lines beginning with One and Two because field 4 is null there, and we are searching for any character, ".", in that field.

Best wishes ... cheers, drl
# 12  
Old 02-13-2016
Thank you drl!

Nenad.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Taking nth column and putting its value in n+1 column using awk

Hello Members, Need your expert opinion how to tackle below. I have an input file that looks like below: USS|AWCC|AFGAW|93|70 USSAA|Roshan TDCA|AFGTD|93|72,79 ALB|Vodafone|ALBVF|355|69 ALGEE|Wataniya (Nedjma)|DZAWT|213|50,550 I like output file in below format: ... (7 Replies)
Discussion started by: umarsatti
7 Replies

2. Shell Programming and Scripting

Break Column nth in a CSV file into two

Hi Guys, Need help with logic to break Column nth in a CSV file into two for e.g Refer below the second column as the nth column "abcd","","type/beta-version" need output in a following format "abcd","/place/asia/india/mumbai","/product/sw/tomcat","type/beta-version" ... (5 Replies)
Discussion started by: awk-admirer
5 Replies

3. Shell Programming and Scripting

How to remove mth and nth column from a file?

Hi, i need to remove mth and nth column from a csv file. here m and n is not a specific number. it is a variable ex. m=2 n=5 now i need to remove the 2nd and 5th line.. Please help how to do that. Thanks!!! (18 Replies)
Discussion started by: zaq1xsw2
18 Replies

4. Shell Programming and Scripting

Find for line with not null values at nth place in pipe delimited file

Hi, I am trying to find the lines in a pipe delimited file where 11th column has not null values. Any help is appreciated. Need help asap please. thanks in advance. (3 Replies)
Discussion started by: manikms
3 Replies

5. Shell Programming and Scripting

Calculating average for every Nth line in the Nth column

Is there an awk script that can easily perform the following operation? I have a data file that is in the format of 1944-12,5.6 1945-01,9.8 1945-02,6.7 1945-03,9.3 1945-04,5.9 1945-05,0.7 1945-06,0.0 1945-07,0.0 1945-08,0.0 1945-09,0.0 1945-10,0.2 1945-11,10.5 1945-12,22.3... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

6. Shell Programming and Scripting

Using AWK to find top Nth values in Nth column

I have an awk script to find the maximum value of the 2nd column of a 2 column datafile, but I need to find the top 5 maximum values of the 2nd column. Here is the script that works for the maximum value. awk 'BEGIN { subjectmax=$1 ; max=0} $2 >= max {subjectmax=$1 ; max=$2} END {print... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

7. Shell Programming and Scripting

Finding Nth Column

Please help me how can I display every nth field present in a "|" delimited file. Ex: If a have a file with data as a|b|c|d|e|f|g|h|k|l|m|n I want to display every 3rd feild which means the output should be c f k n Please help me. (1 Reply)
Discussion started by: ngkumar
1 Replies

8. Shell Programming and Scripting

get 3rd column of nth line

hi; i have a file.txt and its 9th, 10th and 11th line lines are: RbsLocalCell=S2C1 maxPortIP 4 (this is 9th line) RbsLocalCell=S3C1 maxPortIP 4 (this is 10th line) RbsLocalCell=S1C1 ... (11 Replies)
Discussion started by: gc_sw
11 Replies

9. Shell Programming and Scripting

Editing 1st or nth column

Hi, I have a file whick is pipe delimited : 100| alpha| tabgo|watch| |||| 444444 | alpha| tabgo|watch| |||| 444444 | sweden |tabgo|watch| |||| 444444 | US| tabgo|watch| |||| 444444 100| factory| tabgo|watch| |||| 444444 | ABC| tabgo|watch| |||| 444444 | launch| tabgo|watch| ||||... (4 Replies)
Discussion started by: darshanw
4 Replies

10. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies
Login or Register to Ask a Question