Removing Headers and a Column

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing Headers and a Column
# 8  
Old 01-29-2008
It should now remove 6 lines of header at a time ($HEADERSIZE)
If you need to keep the first 6 lines, it's just a matter of changing the header calculation line to:
  } elsif (($linenumber > $HEADERSIZE) && ($linenumber % $PAGESIZE > $HEADERSIZE)) {

It will still remove column 4 from that first header though.
# 9  
Old 01-29-2008
It seems like it is removing 6 lines from the top, then counting down 51 lines and removing 6, but as if it shifted the lines up first. I can't really tell but as it goes down the report it is slowly removing data and leaving pieces of the header. I double checked the page size and it is still correct. I am not sure.

The only other problem besides that is when there is nothing else in a line but column 4, that is still showing in what was column 5.

Perhaps it'd be easier if we excluded the first 6 lines from each page? I will see what I can come up with. Thanks alot

It seems like my page size was off, it is 52 not 51. I changed this and it worked for about half of the report. For some reason about half way down it just stopped taking off the headers all together, i am not really sure why. The rest of the report from about half way down is intact and it just does not remove anymore headers. Odd

Any advise welcome. Thank you

As for the header, if it removes the header I am fine with that I will just make a template file to at least put a header at the top of it before it goes to printing, unfortunately I think without keeping the header it may just print and not keep any page breaks (since this will end up on a line printer). So keeping all the headers intact would be nice but regardless removing data from lines may cause the same problem. So it may not be a big deal. Thanks

Last edited by DerangedNick; 01-29-2008 at 09:11 PM..
# 10  
Old 01-29-2008
Fixed the problem when dealing with short lines (edited original script again)
If it stopped cutting columns out halfway down, it probably encountered the text that suggests you're into the totals section.

The headersize calculation should work better with >= in it instead of >

This will also allow you to turn off the header removal part by setting HEADERSZE to 0 at the top of the script.
# 11  
Old 01-29-2008
Search pattern not terminated at ./testscript line 16.

I will try to find it.

I have tried to find it but i am still not able to find it. I will keep looking for the time being but any help would be appreciated.

Ok i found the error, there appears to have been two. / missing on line 16 and a " on line 17. It runs however it is about the same. I do not see much of a difference. Starting about half way in it does the same thing and it is still not grabbing the lines that do not have anything in the other columns.

It continues pulling out column 4 the entire file after it stops removing headers however and it even pulls column 4 out of the totals section. Any help appreicated.

Thank you again for all the help

Ok this is what I have left that needs to be resolved.

1. syntax errors have been resolved to the best of my abilities.
2. Headers while still being cut into, I can replace the text that is being taken out, so I can deal with that.

Not resolved
1. Totals are still being cut into, it is printing but Column 4 is being taken out of the totals.
2. Column 4 where Columns 1/2/3/5/6 have something is still being printed. If I could get some type of script that could just remove a line if it contains X that would work. Each of the 4 lines under each record that is still printing in column 4 will always start with the same word. Maybe that would be easier?

Thanks again

Last edited by DerangedNick; 01-29-2008 at 11:13 PM.. Reason: Unable to locate error
# 12  
Old 01-29-2008
I've tested out the code, (found the same bugs you found and fixed them):
#!/usr/bin/perl -w
while (<>) {
  if (/^Totals$/) {
  if ($intotals) {
    print $_;
  } elsif ($linenumber % $PAGESIZE > $HEADERSIZE) {
    if (/^(.{58}).{27}(.*)$/) {
      print "$1$2\n";
    } elsif (/^(.{58}).{1-27}$/) {
      print "$1\n";
    } else {
      print $_;

I don't see the same behaviour you report.
What does the line that indicates you are entering the totals section look like?
What expression have you used on line 8 to search for this?

Do you want the headers at the top of each page removed still?
If not, change the headersize to 0.
# 13  
Old 01-30-2008
I changed the Header Size to 0 - however it still removes Column 4, but I did find a work around using the sed command to fix it. (may not be the easiest but it works).

On line 8 i used "Organization Totals" "Organization" and just "Organ" trying to get it to pick it up (each one by itself). The totals start 13 lines up from the bottom, but it still cuts out column 4 for some reason. The line Organization does contain alot more information besides that, could that be the problem? I just put what the line starts with.

Also it leaving the 4 lines under each record is still the only other thing that it is doing.

I am still not sure why.


Also if it was possible to just add a command in there that said remove all lines that contain "Random Word1", "Random Word2", "Random Word3", and "Random Word 4" None of the sayings in the 4 lines being left behind should ever appear in any other part of this document, that would resolve that issue.

Would just leave the totals area needing to be fixed.

It seems that the 4 lines being left behind do not actually go to the end of the line, it is a columb 4 by itself but they are not the full length of columb 4 because there is not a 5. So on these 4 lines, it starts at 59 but if it is only 10 characters the line would end at 69 instead of 85. Perhaps this is causing the problem?

Last edited by DerangedNick; 01-30-2008 at 12:27 AM..
# 14  
Old 01-30-2008
An example speaks more than a thousand words. you can provide some input sample and then describe how you want the output to look like.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Row bind multiple csv files having different column headers

All, I guess by this time someone asked this kind of question, but sorry I am unable to find after a deep search. Here is my request I have many files out of which 2 sample files provided below. File-1 (with A,B as column headers) A,B 1,2 File-2 (with C, D as column headers) C,D 4,5 I... (7 Replies)
Discussion started by: ks_reddy
7 Replies

2. Shell Programming and Scripting

Capturing column headers in an array

Hello, I am processing a tab delimited text file and need to grab all of the column headers in an array. The input looks like, num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 ... (5 Replies)
Discussion started by: LMHmedchem
5 Replies

3. Shell Programming and Scripting

Merge csvs with column headers

hello gurus, Somebody must have done this before, I couldn't find anything. Please redirect me if this was solved before, and if not please help. To the problem now, I have multiple csv files (about 1000) which I need to concatenate by column header. The final file should have a superset... (4 Replies)
Discussion started by: abh.kumar
4 Replies

4. Shell Programming and Scripting

Sar -u generates multiple column headers in csv file

Hi All, The below sar -u command generates multiple column headers in csv file Expected output should print column headers only once in the csv file shell script: $cat #!/bin/bash while ; do sar -u 15 1 | awk '/^/ {print $1,$2,$4,$6,$7}' | tr -s ' ' ',' >>... (6 Replies)
Discussion started by: a1_win
6 Replies

5. Shell Programming and Scripting

Merge column headers and transpose

Hello Everyone! I am new on this forum and this is my first post. I wish to apologize for my, not canonical, English. I would like to solve this problem but I have no clue of how do it!I will be grateful if someone could help me! I have a table like this: gene TF1 TF2 TF3 TF4 gene1 1 2 3 4... (5 Replies)
Discussion started by: giuliangiuseppe
5 Replies

6. Shell Programming and Scripting

Transpose field names from column headers to values in one column

Hi All, I'm looking for a script which can transpose field names from column headers to values in one column. for example, the input is: IDa;IDb;IDc;PARAM1;PARAM2;PARAM3; a;b;c;p1val;p2val;p3val; d;e;f;p4val;p5val;p6val; g;h;i;p7val;p8val;p9val; into the output like this: ... (6 Replies)
Discussion started by: popesk
6 Replies

7. Shell Programming and Scripting

Matching words based on column headers

Discussion started by: bha148
1 Replies

8. UNIX for Dummies Questions & Answers

Sort by Column Headers

Hi All, I am new to UNIX can you please help me to sort a file with different columns my file looks like this $ cat gaut.txt UID PID PPID PGID SID C STIME TTY TIME CMD liveuser 3008 2892 3008 3008 0 11:58 ? 00:00:00 gnome-session liveuser 3019 1 ... (8 Replies)
Discussion started by: cgk1983
8 Replies

9. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

10. Shell Programming and Scripting

Excel Column Headers

cat ABC.log | egrep "Error 500" >> /tmp/Logs.log egrep "<Mango>.*<.Mango>" Logs.log | sed -e "s/^.*<Mango/<Mango/" | cut -f2 -d">"| cut -f1 -d"<" >> /tmp/temp1.xls egrep "<Apple>.*<.Apple>" Logs.log | sed -e "s/^.*<Apple/<Apple/" | cut -f2 -d">"| cut -f1 -d"<" >> /tmp/temp2.xls print Heading1,... (1 Reply)
Discussion started by: pk_eee
1 Replies
Login or Register to Ask a Question