Sponsored Content
Top Forums Shell Programming and Scripting Remove text between headers while leaving headers intact Post 302271737 by Trones on Saturday 27th of December 2008 05:55:02 PM
Old 12-27-2008
Remove text between headers while leaving headers intact

Hi,
I'm trying to strip all lines between two headers in a file:

### BEGIN ###

[Header 1]
Text to remove, contains all kinds of characters
...
Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c
om/updates/
Antispyware-Downloadserver.com #2 (Germany)=http://www.antispyware-downloadserve
r.com/updates/
ComputerFixerTools.com (USA)=http://67.202.99.48/spybot/
Evertje Networks (Europe)=http://sbsd.mirror.evertjenetworks.nl/files/updates/
FastSpeedTest.com (USA)=http://fastspeedtest.net/mirrors/spybot/
...

[Header 2]
More stuff, more headers

### END ###

I've tried several different sed commands, and the results have been either nothing is removed, or the line [Header 2] is removed as well.

Sorry, I haven't saved a list of what I've tried; here is my most recent failed attempt though:

sed -e '/\[Header\ 1\]/,/\[Header\ 2\]/ { /^\[Header\ 2\]$/!d }

Any help would be appreciated, and thank you in advance =)
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

help:how to remove headers in output file

Hi I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines. But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
Discussion started by: raj_thota
1 Replies

2. Shell Programming and Scripting

Remove Headers throughout a data file

I have a data file with over 500,000 records/lines that has the header throughout the file. SEQ_ID Name Start_Date Ins_date Add1 Add2 1 Harris 04/02/08 03/02/08 333 Main Suite 101 2 Smith 02/03/08 01/23/08 287 Jenkins SEQ_ID Name ... (3 Replies)
Discussion started by: psmall
3 Replies

3. UNIX for Dummies Questions & Answers

Remove certain headers using mailx or sendmail

Hello, So i want to send mails in any way from a solaris 5.8 system, perhaps using mailx or sendmail. My purpose is to stay clear of systems name in head data. So i want to strip at least the "Message-Id" and the "Recieved" headers of the mail. Yet this seems to be a bit of a problem. Now i... (2 Replies)
Discussion started by: congo
2 Replies

4. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

5. UNIX for Dummies Questions & Answers

sed remove two headers; writing more elegant code

Hi there, I have two questions. First, I was wondering how to use sed to remove two header lines or two tail lines. Here I just do the same operation twice...I'm sure there is a better way. Second, and more importantly, is there a better way to have these operations use files other than... (5 Replies)
Discussion started by: mikey11415
5 Replies

6. Shell Programming and Scripting

Remove interspersed headers in .dat file with AWK

Heya there, A small selection of my data is shown below. DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602 2011-06-25 ... (4 Replies)
Discussion started by: gd9629
4 Replies

7. UNIX for Dummies Questions & Answers

Using sed command to remove multiple instances of repeating headers in one file?

Hi, I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies

8. Shell Programming and Scripting

Faster command to remove headers for files in a directory

Good evening Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000 lines in average. i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies

9. Shell Programming and Scripting

Remove headers thar dont match

Good evening I need your help please, im new at Unix and i wanted to remove the first 5 headers for 100000 records files and then create a control file .ctl that contains the number of records and all seem to work out but when i tested at production it didnt wotk. Here is the code: #!... (6 Replies)
Discussion started by: alexcol
6 Replies

10. Shell Programming and Scripting

Remove white space and duplicate headers

I have a file called "dsout" with empty rows and duplicate headers. DATE TIME TOTAL_GB USED_GB %USED --------- -------- ---------- ---------- ---------- 03/05/013 12:34 PM 3151.24316 2331.56653 73.988785 ... (3 Replies)
Discussion started by: Daniel Gate
3 Replies
Header(3)						User Contributed Perl Documentation						 Header(3)

NAME
Text::Header - RFC 822/2068 "header" and "unheader" functions SYNOPSIS
use Text::Header; # header and unheader exported # Construct headers similar to CGI.pm and HTTP::Headers @HEADERS = header(content_type => 'text/html', author => 'Nathan Wiger', last_modified => $date, accept => [qw(text/html text/plain)]); # The above produces the array: @HEADERS = ("Content-Type: text/html ", "Author: Nathan Wiger ", "Last-Modified: Wed Sep 27 13:31:06 PDT 2000 ", "Accept: text/html, text/plain "); # Can also construct SMTP headers to format mail @mail_headers = header(from => 'Nathan Wiger <nate@sun.com>', to => 'perl5-porters@perl.org'); print $MAIL @mail_headers, " Keep up the great work! "; # The above would print this to the $MAIL handle: From: Nathan Wiger <nate@sun.com> To: perl5-porters@perl.org Keep up the great work! DESCRIPTION
This module provides two new functions, "header" and "unheader", which provide general-purpose RFC 822 header construction and parsing. They do not provide any intelligent defaults of HTTP-specific methods. They are simply aimed at providing an easy means to address the mechanics of header parsing. The output style is designed to mimic "CGI.pm" and "HTTP::Headers", so that users familiar with these interfaces will feel at home with these functions. As shown above, the "headers" function automatically does the following: 1. uc's the first letter of each tag token and lc's the rest, also converting _'s to -'s automatically 2. Adds a colon separating each tag and its value, and exactly one newline after each one 3. Combines list elements into a comma-delimited string Note that a list is always joined into a comma-delimited string. To insert multiple separate headers, simply call "header" with multiple args: push @out, header(accept => 'text/html', accept => 'text/plain'); This would create multiple "Accept:" lines. Note that unlike "CGI.pm", the "header" function provided here does not provide any intelligent defaults. If called as: @out_headers = header; It will return an empty list. This allows "header" to be more general pupose, so it can provide SMTP and other headers as well. You can also use it as a generic text formatting tool, hence the reason it's under the "Text::" hierarchy. The "unheader" function works in exactly the opposite direction from "header", pulling apart headers and returning a list. "unheader": 1. lc's the entire tag name, converting -'s to _'s 2. Separates each tag based on the colon delimiter, chomping newlines. 3. Returns a list of tag/value pairs for easy assignment to a hash So, assuming the @HEADERS array shown up top: %myheaders = unheader(@HEADERS); The hash %myheaders would have the following values: %myheaders = ( content_type => 'text/html', author => 'Nathan Wiger', last_modified => 'Wed Sep 27 13:31:06 PDT 2000', accept => 'text/html, text/plain' ); Note that all keys are converted to lowercase, and their values have their newlines stripped. However, note that comma-separated fields are not split up on input. This cannot be done reliably because some fields, such as the HTTP "Date:" header, can contain commas even though they are not lists. Inferring this type of structure would require knowledge of content, and these functions are specifically designed to be content-independent. The "unheader" function will respect line wrapping, as seen in SMTP headers. It will simply join the lines and return the value, so that: %mail = unheader("To: Nathan Wiger <nate@sun.com>, perl5-porters@perl.org"); Would return: $mail{to} = "Nathan Wiger <nate@sun.com>, perl5-porters@perl.org" Notice that multiple spaces between the comma separator have been condensed to a single space. Since the "header" and "unheader" functions are direct inverses, this call: @out = header unheader @in; Will result in @out being exactly equivalent to @in. REFERENCES
This is designed as both a Perl 5 module and also a Perl 6 prototype. Please see the Perl 6 proposal at http://dev.perl.org/rfc/333.html This module is designed to be fully compliant with the internet standards RFC 822 (SMTP Headers) and RFC 2068 (HTTP Headers). AUTHOR
Copyright (c) 2000 Nathan Wiger <nate@sun.com>. All Rights Reserved. This module is free software; you may copy this under the terms of the GNU General Public License, or the Artistic License, copies of which should have accompanied your Perl kit. perl v5.8.2 2003-12-23 Header(3)
All times are GMT -4. The time now is 09:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy