Sponsored Content
Top Forums Shell Programming and Scripting Removing matching text from multiple files with a shell script Post 302525761 by ctsgnb on Saturday 28th of May 2011 05:37:07 AM
Old 05-28-2011
Depending on what you want to filter out, take care about this :
if you use # THISMATCHESINEVERYENTRY as the pattern triggering the start/stop deletion, you should make sure that this pattern appear an even number of time.

Consider the following example and see what append with a odd number of that pattern :

Code:
$ cat tst
blabla
# THISMATCHESINEVERYENTRY
filtered out
filtered out
# THISMATCHESINEVERYENTRY
blabla
# THISMATCHESINEVERYENTRY
filtered out
filtered out
filtered out
tutu
# THISMATCHESINEVERYENTRY
toto
# THISMATCHESINEVERYENTRY
filtered out
titi
filtered out

Code:
$ sed '/# THISMATCHESINEVERYENTRY/,/# THISMATCHESINEVERYENTRY/ d' tst
blabla
blabla
toto
$

As you see, the end of the file will be filtered out.

---------- Post updated at 11:37 AM ---------- Previous update was at 11:33 AM ----------

If you choose the same /startRE/ and /endRE/ pattern, every odd occurence of the pattern will start the deletion, make sure that the file you want to parse fit the format you need (so you will avoid bad surprise of unexpected deletion)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell script for matching 2 files

Hi, Would anyone be able to suggest on this shell script query? =) From the contents of the 2 sample files below, if both files' 1st fields matched (ex: XYZ1234=XYZ1234), I want to append the 4th field (semi-colon delimited) from File2 to the end of that line in File1. If no match found, then... (8 Replies)
Discussion started by: gholdbhurg
8 Replies

2. Shell Programming and Scripting

Removing matching text

Hello Everyone! Of course I am rather new to these forums, but I have been browsing the forums for awhile, always has a lot of useful information :) This time however, I can not find the information I need, well, not completely. So here is the problem, I am a linux admin (dealing mostly with... (1 Reply)
Discussion started by: Rhije
1 Replies

3. Homework & Coursework Questions

shell script that can create, monitor the log files and report the issues for matching pattern

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Write an automated shell program(s) that can create, monitor the log files and report the issues for matching... (0 Replies)
Discussion started by: itian2010
0 Replies

4. UNIX for Dummies Questions & Answers

Using Shell Script To Loop Program Through Multiple Text Files

Hello, So I have approximately 300 files of raw data (.txt) files that I am using to perform statistical analysis. I have been able to construct a Fortran program that is able to perform my statistical analysis on a file by file basis. However, I now want to be able to loop program through... (19 Replies)
Discussion started by: Jimmyd24
19 Replies

5. Shell Programming and Scripting

Help me to find files in a shell script with any matching pattern

Hi friends.. I have many dirs in my working directory. Every dir have thousands of files (.jsp, .java, .xml..., etc). So I am working with an script to find every file recursively within those directories and subdirectories ending with .jsp or .java which contains inside of it, the the pattern... (3 Replies)
Discussion started by: hnux
3 Replies

6. Shell Programming and Scripting

Removing files matching a pattern

I am on ubuntu 11.10 using bash scripts I want to remove all files matching a string pattern and I am using the following code find . -name "*$pattern*" -exec rm -f {} \;I have encountered a problem when $pattern is empty. In this case all my files in my current directory were deleted. This... (3 Replies)
Discussion started by: kristinu
3 Replies

7. Shell Programming and Scripting

awk script issue redirecting to multiple files after matching pattern

Hi All I am having one awk and sed requirement for the below problem. I tried multiple options in my sed or awk and right output is not coming out. Problem Description ############################################################### I am having a big file say file having repeated... (4 Replies)
Discussion started by: kshitij
4 Replies

8. Shell Programming and Scripting

Shell Script Help..Renaming Quoted files removing the timestamp

Hi all, i am new to this forum, unix and shell scripting. I would really appreciate if you all can help me here.. I have files coming in the below format 'filename20513'13May06:03:45 filename are characters.. like 'ABDDUT20513'13May06:03:45 i need it to be renamed as... (17 Replies)
Discussion started by: khman
17 Replies

9. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

10. UNIX for Beginners Questions & Answers

Find and removing the old files and zipping the files using shell script

Hi, I am trying to removing the old files which were older than 10 days and same g zipping the files using the shell script. script was return as follows. find /jboss7_homes/JBOSS7/SKYLIV??/SKYLIV??_CRM/jboss-eap-7.0/standalone/log -mtime +10 -type f | xargs rm -f find /cer_skyliv??/log... (6 Replies)
Discussion started by: venkat918
6 Replies
FBB::OFilterStreambuf(3bobcat)					 ostream filtering				    FBB::OFilterStreambuf(3bobcat)

NAME
FBB::OFilterStreambuf - Base class for std::ostream filtering SYNOPSIS
#include <bobcat/ofilterstreambuf> Linking option: -lbobcat DESCRIPTION
The FBB::OFilterStreambuf class is a specialization of the std::streambuf class and can be used as a base class for classes implementing ostream-filtering. Ostream filtering is defined here as the process by which inserted characters are subject to processing before they are passed on to another (filtered) ostream object (or they may be rejected). The filtering may also result in inserting additional information into the filtered ostream. Chaining of filters is also possible: the filtered ostream may itself use an OFilterStreambuf to filter its received information before passing it on to yet another ostream. As OFilterStreambuf inherits from std::streambuf an OFilterStreambuf object can be used to provide an ostream object with a std::streambuf. Information inserted into such a stream travels the following route: o The information is converted to characters using the standard conversion facilities implemented by std::ostream objects. E.g., when inserting the value 123 this value is converted to the characters '1', '2' and '3', respectively. o Each of the characters is then offered (in turn) to the std::streambuf that is associated with the ostream object. In particular, the std::streambuf's overflow() member is called. o OFstreamBuf's default overflow() function ignores characters, but specializations can override overflow() to process the received characters ad lib. o A overriding overflow() function has access to the member OFstreambuf::out() which is a reference to the std::ostream receiving the filtered information. To implement a simple copy-filter (i.e., all characters are accepted as-is) a class must be derived from OFilterStreambuf providing an overriding implementation of overflow(), e.g., as follows: int DerivedClass::overflow(int ch) { out().put(ch); } Next this std::streambuf specialization can be associated with an ostream into which information to be `copy filtered' can be inserted (cf. the EXAMPLE section below). NAMESPACE
FBB All constructors, members, operators and manipulators, mentioned in this man-page, are defined in the namespace FBB. INHERITS FROM
std::streambuf CONSTRUCTORS
o OFilterStreambuf(): This constructor creates a OFilterStreambuf object without associating it with a destination (filtered) ostream. o OFilterStreambuf(char const *fname, openmode mode = std::ios::out): This constructor creates a OFilterStreambuf object and opens a private std::ofstream object whose filename is provided and that should receive the filtered information. o OFilterStreambuf(std::ostream &out): This constructor creates a OFilterStreambuf object and will insert any filtered information into the provided ostream object. The class's destructor closes the destination (filtered) stream (cf. the description of close() below). MEMBER FUNCTIONS
All members of std::ostreambuf are available, as FBB::OFilterStreambuf inherits from that class. In particular, derived classes should pro- vide their own implementation of int underflow(int ch) to implement any non-trivial filtering. o void close(): This member calls the streambuf::sync() member to flush any pending information to the destination (filtered) stream and then closes the destination stream. Note that the default sync() member performs no special actions but it can be overridden by derived classes to flush the destination stream just prior to its closing. o void open(char const *fname, openmode mode = std::ios::out): This member closes the current destination (filtered) std::ostream object and associates the OFilterStreambuf object with a private std::ofstream object whose filename is provided and that should receive subsequently filtered information. o void open(std::ostream &out): This member closes the current destination (filtered) std::ostream object and associates the OFilterStreambuf object with the pro- vided ostream object. PROTECTED MEMBER FUNCTION
o std::ostream &out() const: This member is available to derived classes to insert information into the destination (filtered) stream. EXAMPLE
#include <iostream> #include <cctype> #include <bobcat/ofilterstreambuf> class NoDigits: public FBB::OFilterStreambuf { public: int overflow(int ch) { if (not isdigit(ch)) out().put(ch); return ch; } int sync() { out() << flush; return 0; } }; using namespace FBB; using namespace std; int main() { NoDigits nod(cout); // no digits to cout ostream out(&nod); out << in.rdbuf(); // rm digits from cin return 0; } FILES
bobcat/ofilterstreambuf - defines the class interface SEE ALSO
bobcat(7) BUGS
None Reported. DISTRIBUTION FILES
o bobcat_3.01.00-x.dsc: detached signature; o bobcat_3.01.00-x.tar.gz: source archive; o bobcat_3.01.00-x_i386.changes: change log; o libbobcat1_3.01.00-x_*.deb: debian package holding the libraries; o libbobcat1-dev_3.01.00-x_*.deb: debian package holding the libraries, headers and manual pages; o http://sourceforge.net/projects/bobcat: public archive location; BOBCAT
Bobcat is an acronym of `Brokken's Own Base Classes And Templates'. COPYRIGHT
This is free software, distributed under the terms of the GNU General Public License (GPL). AUTHOR
Frank B. Brokken (f.b.brokken@rug.nl). libbobcat1-dev_3.01.00-x.tar.gz 2005-2012 FBB::OFilterStreambuf(3bobcat)
All times are GMT -4. The time now is 11:11 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy