Removing Headers and a Column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing Headers and a Column
# 22  
Old 01-30-2008
No that didn't work. I am not really sure what is causing it to cut off that 4th line on column 5. Very strange.
# 23  
Old 01-30-2008
Me either, substr($_,85) should slurp in everything after the 85 character until the end of the line.
# 24  
Old 01-30-2008
Quote:
Originally Posted by DerangedNick
I also attached the totals section, hopefully it makes since. If you have any questions let me know.
Ah, I see, that's not what I had thought you were after really (thus it removing column 4 from the headers - I thought that was a requirement as I'd assumed the headers where column headers Smilie).

It's not matching the "Organization Totals" string because it's looking for an exact line (the ^ and $ symbols mean start and end of line).

So, I believe you are looking for the following:

Remove characters 59-85 from each line,
EXCEPT:
Where the line is part of a repeating header block (6 lines of text every 52 lines)
Any line between the first instance of a line starting with the text "Organization Totals" and the end of the file

Do you still want the headers removed? (Assuming no for now)

So, the resulting code would then become:
Code:
#!/usr/bin/perl -w
$PAGESIZE=52;
$HEADERSIZE=6;
$linenumber=0;
$intotals=0;
while (<>) {
  $linenumber++;
  if (/^Organisation Totals/) {
    $intotals=1;
  }
  if ($intotals) {
    print $_;
  } elsif ($linenumber % $PAGESIZE > $HEADERSIZE) {
    # Not within the header block so cut column 4 out
    #  keep up to char 58, drop 27 chars, keep everything else
    if (/^(.{58}).{27}(.*)$/) {
      print "$1$2\n";
    } elsif (/^(.{58}).{1-27}$/) {
      print "$1\n";
    } else {
      print $_;
    }
  } else {
    # It's within the header so print it all
    #  if you want headers omitted, comment out the following line
    print $_;
  }
}

# 25  
Old 01-31-2008
That script works for the most part, however the same problem happens as before. It is leaving column 4 in tact if it is the only thing on the line.

Not really to sure what else to say to give you a idea as to why it may be doing that.
# 26  
Old 01-31-2008
change this line in Smiling Dragons code:

Code:
    if (/^(.{58}).{27}(.*)$/) {

to:

Code:
    if (/^(.{58}).{27}(.+)$/) {

and see if that helps. The problem is that .* means to match zero or more, so the elsif condition just below it might never get evaluated.
# 27  
Old 01-31-2008
Adding the + just cuts off pieces of the header, while before the header was still in tact. It does not resolve the other problem with leaving column 4 around though.

Thanks again for all the help you two.
# 28  
Old 01-31-2008
Long shot, a hybrid of my code and SM's code:

Code:
#!/usr/bin/perl
use strict;
use warnings;

while (<>) {
   # header lines
   if ($. < 7) {
      print; # comment out this line to delete header lines
      next;
   }
   # print the totals, assumes they are at the end
   if (/^Organisation Totals/) {
      print;
      print <>;
      exit;
   }
   # remove 4th column
   if (/^(.{58}).{27}(.+)$/) {
      print "$1$2\n";
   }
   # remove 4th column when its the only thing
   elsif (/^(.{58}).{1-27}$/) {
      print "$1\n";
   }
   else {
      # not sure what this might print if anything.
      # prefixed lines with '>>>' just to mark them for now	
      # to see if this condition is ever evaluated.
      print ">>>$_";
   }
}


Last edited by KevinADC; 01-31-2008 at 06:14 PM.. Reason: fixed a typo
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Row bind multiple csv files having different column headers

All, I guess by this time someone asked this kind of question, but sorry I am unable to find after a deep search. Here is my request I have many files out of which 2 sample files provided below. File-1 (with A,B as column headers) A,B 1,2 File-2 (with C, D as column headers) C,D 4,5 I... (7 Replies)
Discussion started by: ks_reddy
7 Replies

2. Shell Programming and Scripting

Capturing column headers in an array

Hello, I am processing a tab delimited text file and need to grab all of the column headers in an array. The input looks like, num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 ... (5 Replies)
Discussion started by: LMHmedchem
5 Replies

3. Shell Programming and Scripting

Merge csvs with column headers

hello gurus, Somebody must have done this before, I couldn't find anything. Please redirect me if this was solved before, and if not please help. To the problem now, I have multiple csv files (about 1000) which I need to concatenate by column header. The final file should have a superset... (4 Replies)
Discussion started by: abh.kumar
4 Replies

4. Shell Programming and Scripting

Sar -u generates multiple column headers in csv file

Hi All, The below sar -u command generates multiple column headers in csv file Expected output should print column headers only once in the csv file shell script: $cat sar_cpu_EBS.sh #!/bin/bash while ; do sar -u 15 1 | awk '/^/ {print $1,$2,$4,$6,$7}' | tr -s ' ' ',' >>... (6 Replies)
Discussion started by: a1_win
6 Replies

5. Shell Programming and Scripting

Merge column headers and transpose

Hello Everyone! I am new on this forum and this is my first post. I wish to apologize for my, not canonical, English. I would like to solve this problem but I have no clue of how do it!I will be grateful if someone could help me! I have a table like this: gene TF1 TF2 TF3 TF4 gene1 1 2 3 4... (5 Replies)
Discussion started by: giuliangiuseppe
5 Replies

6. Shell Programming and Scripting

Transpose field names from column headers to values in one column

Hi All, I'm looking for a script which can transpose field names from column headers to values in one column. for example, the input is: IDa;IDb;IDc;PARAM1;PARAM2;PARAM3; a;b;c;p1val;p2val;p3val; d;e;f;p4val;p5val;p6val; g;h;i;p7val;p8val;p9val; into the output like this: ... (6 Replies)
Discussion started by: popesk
6 Replies

7. Shell Programming and Scripting

Matching words based on column headers

Hi , Pls help on this. Input file: NAME1 BSC1 TEXT ID 1 MAINSFAIL TEXT ID 2 DGON TEXT ID 3 lOADONDG NAME2 BSC2 TEXT ID 1 DGON TEXT ID 3 lOADONG (1 Reply)
Discussion started by: bha148
1 Replies

8. UNIX for Dummies Questions & Answers

Sort by Column Headers

Hi All, I am new to UNIX can you please help me to sort a file with different columns my file looks like this $ cat gaut.txt UID PID PPID PGID SID C STIME TTY TIME CMD liveuser 3008 2892 3008 3008 0 11:58 ? 00:00:00 gnome-session liveuser 3019 1 ... (8 Replies)
Discussion started by: cgk1983
8 Replies

9. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

10. Shell Programming and Scripting

Excel Column Headers

cat ABC.log | egrep "Error 500" >> /tmp/Logs.log egrep "<Mango>.*<.Mango>" Logs.log | sed -e "s/^.*<Mango/<Mango/" | cut -f2 -d">"| cut -f1 -d"<" >> /tmp/temp1.xls egrep "<Apple>.*<.Apple>" Logs.log | sed -e "s/^.*<Apple/<Apple/" | cut -f2 -d">"| cut -f1 -d"<" >> /tmp/temp2.xls print Heading1,... (1 Reply)
Discussion started by: pk_eee
1 Replies
Login or Register to Ask a Question