Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Using sed command to remove multiple instances of repeating headers in one file? Post 302560358 by jim mcnamara on Thursday 29th of September 2011 01:55:42 PM
Old 09-29-2011
Code:
grep -v  'HEADER'  bigfile > newfile
# or
sed '/HEADER/d'  bigfile > newfile

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

help:how to remove headers in output file

Hi I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines. But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
Discussion started by: raj_thota
1 Replies

2. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

3. Shell Programming and Scripting

Remove text between headers while leaving headers intact

Hi, I'm trying to strip all lines between two headers in a file: ### BEGIN ### Text to remove, contains all kinds of characters ... Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c om/updates/ Antispyware-Downloadserver.com #2... (3 Replies)
Discussion started by: Trones
3 Replies

4. Shell Programming and Scripting

Remove repeating pattern from beginning of file names.

I want a shell script that will traverse a file system starting at specific path. And look at all file names for repeating sequences of and remove them from the file name. The portion of the name that gets removed has to be a repeating sequence of the same characters. So the script would... (3 Replies)
Discussion started by: z399y
3 Replies

5. UNIX for Dummies Questions & Answers

sed remove two headers; writing more elegant code

Hi there, I have two questions. First, I was wondering how to use sed to remove two header lines or two tail lines. Here I just do the same operation twice...I'm sure there is a better way. Second, and more importantly, is there a better way to have these operations use files other than... (5 Replies)
Discussion started by: mikey11415
5 Replies

6. Shell Programming and Scripting

sed remove multiple set of lines in one command

is there a way with sed to removed more than one set of lines in one line? so i mean sed ${firstElem},${lastIndex}d web.xml > web1.xml this will delete lines between ${firstElem},${lastIndex} i want in the same line to do somethinkg like this (doesn't work so far) sed... (3 Replies)
Discussion started by: Poki
3 Replies

7. Shell Programming and Scripting

How to remove a semi-repeating character in position 1 of a file

My file is in a good column format but several lines in the file begin with a zero. I'm in KSH and looking for a command to remove this zero and keep the text next to it. I don't want any of the zeros in the other columns removed. Below is a snip from the file... all I need to do is remove that... (2 Replies)
Discussion started by: right_coaster
2 Replies

8. Shell Programming and Scripting

Sed replace using same pattern repeating multiple times in a line

Sed replace using same pattern repeating multiple times in a line I have text like below in a file: I am trying to replace the above line to following How can I acheive this? I am able to do it if the occurrence is for 1 time: But If I try like below I am getting like this: I have to... (4 Replies)
Discussion started by: sol_nov
4 Replies

9. Shell Programming and Scripting

Faster command to remove headers for files in a directory

Good evening Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000 lines in average. i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies

10. Shell Programming and Scripting

awk and or sed command to sum the value in repeating tags in a XML

I have a XML in which <Amt Ccy="EUR">3.1</Amt> tag repeats. This is under another tag <Main>. I need to sum all the values of <Amt Ccy=""> (Ccy may vary) coming under <Main> using awk and or sed command. can some help? Sample looks like below <root> <Main> ... (6 Replies)
Discussion started by: bk_12345
6 Replies
REFORMAIL(1)						      Double Precision, Inc.						      REFORMAIL(1)

NAME
reformail - E-mail reformatting tool SYNOPSIS
reformail -s command [option...] reformail -D len filename reformail -x header:... reformail -X header:... reformail [options...] See below for additional options DESCRIPTION
The reformail program reads a message on standard input, reformats it in some way, and writes the message to standard output: Splitting mailboxes into individual messages The -s option splits the mbox-formatted mailbox file on standard input into individual messages. An external program is executed for each message. The contents of each individual message will be provided to the external program on standard input. If the FILENO environment variable is set to a number, reformail will consecutively increment FILENO each time the program is executed for each individual message. If FILENO is not set, it's initial value will be "000". If FILENO is set to a non-numeric value, FILENO will remain unchanged. Detecting duplicate messages The -D option implements a simple way to delete duplicate messages in incoming mail. filename is a file that will be approximately 'len' bytes long. This file will be used by reformail to save message IDs seen in recent mail. reformail reads the message on standard input. If the message has a Message-ID: header that's already in the cache file, reformail terminates with the exit code set to 0. Otherwise, reformail terminates with the exit code set to 1. Extracting headers The -x and -X options extract the indicated headers from the message, and print them to standard output. Multiple -x and -X options can be specified at the same time, and may be intermixed. The -x option extracts and prints the contents of the header. The -X option prints the name of the header as well. In all other situations, reformail copies the message on its standard input to its standard output, reformatting the message as follows: OPTIONS
-a'header: value' Append a custom header to the message if this header does not already exist. If the header is either Message-ID: or Resent-Message-ID: and the value is empty, reformail generates a (hopefully) unique message ID for you. -A'header: value' Append a custom header to the message even if this header already exists. If the header is either Message-ID: or Resent-Message-ID: and the value is empty, reformail generates a (hopefully) unique message ID for you. -c Concatenate multi-line headers. Headers split on multiple lines are combined into a single line. -dn If n is 1, each line will be terminated with CRLF. If n is 0 (default), each line will be terminated with LF. reformail reads a message with either line terminator, and will force the message to have the specified line termination. -f0 Remove the "From_" line from the message, if it's there. -f1 Add the "From_" line to the message, if it's not there. reformail will attempt to generate the "From_" line from any Errors-To:, Return-Path:, or From: headers in the message. "root" will be used if reformail is unable to determine the return address. -i'header: value' Appends a custom header to the message. If this header already exists it is renamed by prepending "Old-" to the name of the header. -I'header: value' Append a custom header to the message. If this header already exists in the message, the old header is completely removed. If the value is empty, any existing header is completely removed, and nothing gets appended. -R oldheader: newheader: Rename the indicated header. -u'header:' If this header occurs multiple times in the message, remove all occurrences except the first one. -U'header:' If this header occurs multiple times in the message, remove all occurrences except the last one. AUTOREPLIES
The autoreply options from earlier versions of mailbot have been moved into mailbot(1)[1]. BUGS
For the -a, -A, and -I options, a space after the header name and the colon is considered to be a non-empty field. Do not provide the same header to more than one family of header-modifying options, such as -u/-U and -a/-A. Doing so yields unpredictable results. It's better to run reformail several times (use a pipe, perhaps). SEE ALSO
courier(8)[2], sendmail(8), mailbot(1)[1], maildrop(1)[3]. AUTHOR
Sam Varshavchik Author NOTES
1. mailbot(1) [set $man.base.url.for.relative.links]/mailbot.html 2. courier(8) [set $man.base.url.for.relative.links]/courier.html 3. maildrop(1) [set $man.base.url.for.relative.links]/maildrop.html Courier Mail Server 08/30/2011 REFORMAIL(1)
All times are GMT -4. The time now is 02:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy