12-27-2008
Remove text between headers while leaving headers intact
Hi,
I'm trying to strip all lines between two headers in a file:
### BEGIN ###
[Header 1]
Text to remove, contains all kinds of characters
...
Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c
om/updates/
Antispyware-Downloadserver.com #2 (Germany)=http://www.antispyware-downloadserve
r.com/updates/
ComputerFixerTools.com (USA)=http://67.202.99.48/spybot/
Evertje Networks (Europe)=http://sbsd.mirror.evertjenetworks.nl/files/updates/
FastSpeedTest.com (USA)=http://fastspeedtest.net/mirrors/spybot/
...
[Header 2]
More stuff, more headers
### END ###
I've tried several different sed commands, and the results have been either nothing is removed, or the line [Header 2] is removed as well.
Sorry, I haven't saved a list of what I've tried; here is my most recent failed attempt though:
sed -e '/\[Header\ 1\]/,/\[Header\ 2\]/ { /^\[Header\ 2\]$/!d }
Any help would be appreciated, and thank you in advance =)
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi
I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines.
But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
Discussion started by: raj_thota
1 Replies
2. Shell Programming and Scripting
I have a data file with over 500,000 records/lines that has the header throughout the file.
SEQ_ID Name Start_Date Ins_date Add1 Add2
1 Harris 04/02/08 03/02/08 333 Main Suite 101
2 Smith 02/03/08 01/23/08 287 Jenkins
SEQ_ID Name ... (3 Replies)
Discussion started by: psmall
3 Replies
3. UNIX for Dummies Questions & Answers
Hello,
So i want to send mails in any way from a solaris 5.8 system, perhaps using mailx or sendmail. My purpose is to stay clear of systems name in head data. So i want to strip at least the "Message-Id" and the "Recieved" headers of the mail. Yet this seems to be a bit of a problem.
Now i... (2 Replies)
Discussion started by: congo
2 Replies
4. Shell Programming and Scripting
Hi ,
I have a typical situation. I have 4 files and with different headers (number of headers is varible ).
I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only).
For example -
File 1
H1|H2|H3|H4
11|12|13|14
21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies
5. UNIX for Dummies Questions & Answers
Hi there,
I have two questions. First, I was wondering how to use sed to remove two header lines or two tail lines. Here I just do the same operation twice...I'm sure there is a better way. Second, and more importantly, is there a better way to have these operations use files other than... (5 Replies)
Discussion started by: mikey11415
5 Replies
6. Shell Programming and Scripting
Heya there,
A small selection of my data is shown below.
DATE TIME FRAC_DAYS_SINCE_JAN1
2011-06-25 08:03:20.000 175.33564815
2011-06-25 08:03:25.000 175.33570602
2011-06-25 ... (4 Replies)
Discussion started by: gd9629
4 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies
8. Shell Programming and Scripting
Good evening
Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000
lines in average.
i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies
9. Shell Programming and Scripting
Good evening
I need your help please, im new at Unix and i wanted to remove the first 5 headers for 100000 records files and then create a control file .ctl that contains the number of records and all seem to work out but when i tested at production it didnt wotk.
Here is the code:
#!... (6 Replies)
Discussion started by: alexcol
6 Replies
10. Shell Programming and Scripting
I have a file called "dsout" with empty rows and duplicate headers.
DATE TIME TOTAL_GB USED_GB %USED
--------- -------- ---------- ---------- ----------
03/05/013 12:34 PM 3151.24316 2331.56653 73.988785 ... (3 Replies)
Discussion started by: Daniel Gate
3 Replies
LEARN ABOUT DEBIAN
text::header
Header(3) User Contributed Perl Documentation Header(3)
NAME
Text::Header - RFC 822/2068 "header" and "unheader" functions
SYNOPSIS
use Text::Header; # header and unheader exported
# Construct headers similar to CGI.pm and HTTP::Headers
@HEADERS = header(content_type => 'text/html',
author => 'Nathan Wiger',
last_modified => $date,
accept => [qw(text/html text/plain)]);
# The above produces the array:
@HEADERS = ("Content-Type: text/html
",
"Author: Nathan Wiger
",
"Last-Modified: Wed Sep 27 13:31:06 PDT 2000
",
"Accept: text/html, text/plain
");
# Can also construct SMTP headers to format mail
@mail_headers = header(from => 'Nathan Wiger <nate@sun.com>',
to => 'perl5-porters@perl.org');
print $MAIL @mail_headers, "
Keep up the great work!
";
# The above would print this to the $MAIL handle:
From: Nathan Wiger <nate@sun.com>
To: perl5-porters@perl.org
Keep up the great work!
DESCRIPTION
This module provides two new functions, "header" and "unheader", which provide general-purpose RFC 822 header construction and parsing.
They do not provide any intelligent defaults of HTTP-specific methods. They are simply aimed at providing an easy means to address the
mechanics of header parsing.
The output style is designed to mimic "CGI.pm" and "HTTP::Headers", so that users familiar with these interfaces will feel at home with
these functions. As shown above, the "headers" function automatically does the following:
1. uc's the first letter of each tag token and lc's the
rest, also converting _'s to -'s automatically
2. Adds a colon separating each tag and its value, and
exactly one newline after each one
3. Combines list elements into a comma-delimited
string
Note that a list is always joined into a comma-delimited string. To insert multiple separate headers, simply call "header" with multiple
args:
push @out, header(accept => 'text/html',
accept => 'text/plain');
This would create multiple "Accept:" lines.
Note that unlike "CGI.pm", the "header" function provided here does not provide any intelligent defaults. If called as:
@out_headers = header;
It will return an empty list. This allows "header" to be more general pupose, so it can provide SMTP and other headers as well. You can
also use it as a generic text formatting tool, hence the reason it's under the "Text::" hierarchy.
The "unheader" function works in exactly the opposite direction from "header", pulling apart headers and returning a list. "unheader":
1. lc's the entire tag name, converting -'s to _'s
2. Separates each tag based on the colon delimiter,
chomping newlines.
3. Returns a list of tag/value pairs for easy assignment
to a hash
So, assuming the @HEADERS array shown up top:
%myheaders = unheader(@HEADERS);
The hash %myheaders would have the following values:
%myheaders = (
content_type => 'text/html',
author => 'Nathan Wiger',
last_modified => 'Wed Sep 27 13:31:06 PDT 2000',
accept => 'text/html, text/plain'
);
Note that all keys are converted to lowercase, and their values have their newlines stripped. However, note that comma-separated fields are
not split up on input. This cannot be done reliably because some fields, such as the HTTP "Date:" header, can contain commas even though
they are not lists. Inferring this type of structure would require knowledge of content, and these functions are specifically designed to
be content-independent.
The "unheader" function will respect line wrapping, as seen in SMTP headers. It will simply join the lines and return the value, so that:
%mail = unheader("To: Nathan Wiger <nate@sun.com>,
perl5-porters@perl.org");
Would return:
$mail{to} = "Nathan Wiger <nate@sun.com>, perl5-porters@perl.org"
Notice that multiple spaces between the comma separator have been condensed to a single space. Since the "header" and "unheader" functions
are direct inverses, this call:
@out = header unheader @in;
Will result in @out being exactly equivalent to @in.
REFERENCES
This is designed as both a Perl 5 module and also a Perl 6 prototype. Please see the Perl 6 proposal at http://dev.perl.org/rfc/333.html
This module is designed to be fully compliant with the internet standards RFC 822 (SMTP Headers) and RFC 2068 (HTTP Headers).
AUTHOR
Copyright (c) 2000 Nathan Wiger <nate@sun.com>. All Rights Reserved.
This module is free software; you may copy this under the terms of the GNU General Public License, or the Artistic License, copies of which
should have accompanied your Perl kit.
perl v5.8.2 2003-12-23 Header(3)