Sponsored Content
Top Forums Shell Programming and Scripting Removing matching text from multiple files with a shell script Post 302525746 by boxx on Saturday 28th of May 2011 01:54:55 AM
Old 05-28-2011
Error Removing matching text from multiple files with a shell script

Hello all,

I am in need of assistance in creating a script that will remove a specified block of text from multiple .htaccess files. (roughly 1000 files)

I am attempting to help with a project to clean up a linux server that has a series of unwanted url rewrites in place, as well as some rules that need to stay in place.

The series of rewrite rules that needs to be removed is the same in each file, and the htaccess files are spread throughout 100's of directories within the server.

I am not all that great with command line scripts but I know enough to navigate the server via shell and implement fairly simple scripts. Unfortunately no on else within the company has any experience within maintaining a linux server, so I was delegated to take the lead on this project.

I believe this task can be accomplished using a combination of find, grep, and sed (or awk) but I could use some help in combining these functions to complete the task.

I was able to output a list of all the files that contain the matching rules to a text file to see how extensive the changes that need to be made are going to be using the following:

find /var/www/html/vhosts -name ".htaccess" -exec grep -l "Text To Match" > output.txt {} \;

This seemed to spit out the requested information in a timely matter, now I need to add sed into the equation to actually modify the files that I found, using either the text file containing the path's to each of these files, or piping the output into sed instead of to the text file.

This is where it gets a little bit complicated and the confusion for me sets in. I figured rather than spend countless hours searching google, I would seek the assistance of some others.. that hopefully are a little more knowledgeable than me at the command line.

Can anyone chime in and explain to me how i can incorporate sed or awk into the equation to modify all of these htaccess files to only remove the block of text I specify, while leaving the remaining text in place.

Any help or direction would be greatly appreciated.

Thanks,
BoxX
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell script for matching 2 files

Hi, Would anyone be able to suggest on this shell script query? =) From the contents of the 2 sample files below, if both files' 1st fields matched (ex: XYZ1234=XYZ1234), I want to append the 4th field (semi-colon delimited) from File2 to the end of that line in File1. If no match found, then... (8 Replies)
Discussion started by: gholdbhurg
8 Replies

2. Shell Programming and Scripting

Removing matching text

Hello Everyone! Of course I am rather new to these forums, but I have been browsing the forums for awhile, always has a lot of useful information :) This time however, I can not find the information I need, well, not completely. So here is the problem, I am a linux admin (dealing mostly with... (1 Reply)
Discussion started by: Rhije
1 Replies

3. Homework & Coursework Questions

shell script that can create, monitor the log files and report the issues for matching pattern

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Write an automated shell program(s) that can create, monitor the log files and report the issues for matching... (0 Replies)
Discussion started by: itian2010
0 Replies

4. UNIX for Dummies Questions & Answers

Using Shell Script To Loop Program Through Multiple Text Files

Hello, So I have approximately 300 files of raw data (.txt) files that I am using to perform statistical analysis. I have been able to construct a Fortran program that is able to perform my statistical analysis on a file by file basis. However, I now want to be able to loop program through... (19 Replies)
Discussion started by: Jimmyd24
19 Replies

5. Shell Programming and Scripting

Help me to find files in a shell script with any matching pattern

Hi friends.. I have many dirs in my working directory. Every dir have thousands of files (.jsp, .java, .xml..., etc). So I am working with an script to find every file recursively within those directories and subdirectories ending with .jsp or .java which contains inside of it, the the pattern... (3 Replies)
Discussion started by: hnux
3 Replies

6. Shell Programming and Scripting

Removing files matching a pattern

I am on ubuntu 11.10 using bash scripts I want to remove all files matching a string pattern and I am using the following code find . -name "*$pattern*" -exec rm -f {} \;I have encountered a problem when $pattern is empty. In this case all my files in my current directory were deleted. This... (3 Replies)
Discussion started by: kristinu
3 Replies

7. Shell Programming and Scripting

awk script issue redirecting to multiple files after matching pattern

Hi All I am having one awk and sed requirement for the below problem. I tried multiple options in my sed or awk and right output is not coming out. Problem Description ############################################################### I am having a big file say file having repeated... (4 Replies)
Discussion started by: kshitij
4 Replies

8. Shell Programming and Scripting

Shell Script Help..Renaming Quoted files removing the timestamp

Hi all, i am new to this forum, unix and shell scripting. I would really appreciate if you all can help me here.. I have files coming in the below format 'filename20513'13May06:03:45 filename are characters.. like 'ABDDUT20513'13May06:03:45 i need it to be renamed as... (17 Replies)
Discussion started by: khman
17 Replies

9. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

10. UNIX for Beginners Questions & Answers

Find and removing the old files and zipping the files using shell script

Hi, I am trying to removing the old files which were older than 10 days and same g zipping the files using the shell script. script was return as follows. find /jboss7_homes/JBOSS7/SKYLIV??/SKYLIV??_CRM/jboss-eap-7.0/standalone/log -mtime +10 -type f | xargs rm -f find /cer_skyliv??/log... (6 Replies)
Discussion started by: venkat918
6 Replies
DH_COMPRESS(1)							     Debhelper							    DH_COMPRESS(1)

NAME
dh_compress - compress files and fix symlinks in package build directories SYNOPSIS
dh_compress [debhelperoptions] [-Xitem] [-A] [file...] DESCRIPTION
dh_compress is a debhelper program that is responsible for compressing the files in package build directories, and makes sure that any symlinks that pointed to the files before they were compressed are updated to point to the new files. By default, dh_compress compresses files that Debian policy mandates should be compressed, namely all files in usr/share/info, usr/share/man, files in usr/share/doc that are larger than 4k in size, (except the copyright file, .html and other web files, image files, and files that appear to be already compressed based on their extensions), and all changelog files. Plus PCF fonts underneath usr/share/fonts/X11/ FILES
debian/package.compress These files are deprecated. If this file exists, the default files are not compressed. Instead, the file is ran as a shell script, and all filenames that the shell script outputs will be compressed. The shell script will be run from inside the package build directory. Note though that using -X is a much better idea in general; you should only use a debian/package.compress file if you really need to. OPTIONS
-Xitem, --exclude=item Exclude files that contain item anywhere in their filename from being compressed. For example, -X.tiff will exclude TIFF files from compression. You may use this option multiple times to build up a list of things to exclude. -A, --all Compress all files specified by command line parameters in ALL packages acted on. file ... Add these files to the list of files to compress. CONFORMS TO
Debian policy, version 3.0 SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Joey Hess <joeyh@debian.org> 8.9.0ubuntu2.1 2012-06-12 DH_COMPRESS(1)
All times are GMT -4. The time now is 05:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy