delete all lines with string, process all files in directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting delete all lines with string, process all files in directory
# 1  
Old 04-25-2011
delete all lines with string, process all files in directory

Simply, I have a directory of text files and I need to delete every line in every file containing a specific string. I want to write the modified files to an empty sub directory.

I can't seem to get the sed command to delete the lines containing the string, and not just the string, in other words, not sed '/*string*/d', which doesn't seem to work. I am also not sure how to accumulate the list of input file names and create the output file names. I usually use sed on specific files, so I'm not sure how to process an entire directory. I seem to remember that you can use grep to create a list of filenames, and then process the list, os something like that.

LMHmedchem
# 2  
Old 04-25-2011
Code:
string='text pattern'
cd /path/to/files
grep -l  "$string" * |
while read fname 
do
    sed "/$string/d"  $fname > tmp.tmp
    mv tmp.tmp $fname
    mv $fname ./subdirectory/$fname 
done

Start with that.
This User Gave Thanks to jim mcnamara For This Post:
# 3  
Old 04-25-2011
Quote:
Originally Posted by jim mcnamara
Code:
string='text pattern'
cd /path/to/files
grep -l  "$string" * |
while read fname 
do
    sed "/$string/d"  $fname > tmp.tmp
    mv tmp.tmp $fname
    mv $fname ./subdirectory/$fname 
done

Start with that.
Thanks, the second move command seems redundant. It seems as if the first mv will overwrite the original file, and then the second will move it. I probably want to keep the original file, so it looks like I could just do,
sed "/$string/d" $fname > tmp.tmp
mv tmp.tmp ./subdirectory/$fname

Does that look right?

This one of the lines I am trying to delete,
_D: "trichlormethiazide" "V" 0.683893
where the string is trichlormethiazide

LMHmedchem

---------- Post updated at 04:34 PM ---------- Previous update was at 04:12 PM ----------

That works great, thanks.

Just for clarification, grep is returning only files that contain the value of $string, correct?
# 4  
Old 04-25-2011
Quote:
Originally Posted by LMHmedchem
Just for clarification, grep is returning only files that contain the value of $string, correct?
Yes. It's returning the names of the files that contain "$string".

No biggie, perhaps, but I would put quotes around $fname (i.e. "$fname"), should your $string contain spaces..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete all lines from file matching a string

I wish to search and delete all lines in /app/Jenkins/deploy.txt having this filename string /app/Jenkins/file2.mrt as entry: I'm using : colon as delimiter in sed command as I'm dealing with file paths. Below is the command I was expecting to work. sed -i ":/app/Jenkins/file2.mrt:d"... (1 Reply)
Discussion started by: mohtashims
1 Replies

2. Shell Programming and Scripting

Delete all lines except a line starting with string

Shell : bash OS : RHEL 6.8 I have a file like below. $ cat pattern.txt hello txt1 txt2 txt3 some other text txt4 I want to remove all lines in this file except the ones starting with txt . How can I do this ? (4 Replies)
Discussion started by: omega3
4 Replies

3. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

4. Shell Programming and Scripting

Delete all files if another files in the same directory has a matching occurrence of a specific word

he following are the files available in my directory RSK_123_20141113_031500.txt RSK_123_20141113_081500.txt RSK_126_20141113_041500.txt RSK_126_20141113_081800.txt RSK_128_20141113_091600.txt Here, "RSK" is file prefix and 123 is a code name and rest is just timestamp of the file when its... (7 Replies)
Discussion started by: kridhick
7 Replies

5. Shell Programming and Scripting

How to delete lines starting with specific string?

Dear all, I would like to delete even lines starting with "N" together with their respective titles which are actually odd lines. Below is the example of input file. I would like to remove line 8 and 12 together with its title line, i.e., line 7 and 11, respectively.... (2 Replies)
Discussion started by: huiyee1
2 Replies

6. Shell Programming and Scripting

Grep a string from input file and delete next three lines including the line contains string in xml

Hi, 1_strings file contains $ cat 1_strings /home/$USER/Src /home/Valid /home/Review$ cat myxml <projected value="some string" path="/home/$USER/Src"> <input 1/> <estimate value/> <somestring/> </projected> <few more lines > <projected value="some string" path="/home/$USER/check">... (4 Replies)
Discussion started by: greet_sed
4 Replies

7. Shell Programming and Scripting

need to delete all lines from a group of files except the 1st 2 lines

Hello, I have a group of text files with many lines in each file. I need to delete all the lines in each and only leave 2 lines in each file. (3 Replies)
Discussion started by: script_op2a
3 Replies

8. Shell Programming and Scripting

Delete the lines if it matchs a selected string

HI, I need one help to generate the file based on few condition, I need to print only those line which has N and delete the line which as all Y's. ANd I need to compare only the fields which are marked as Y and N . And one more thing is in the below file 1 and file 2 and file2 abc, dbc can... (6 Replies)
Discussion started by: rashmisb
6 Replies

9. Shell Programming and Scripting

Delete all files if another files in the same directory has a matching occurence of a specific word

Hello, I have several files in a specific directory. A specific string in one file can occur in another files. If this string is in other files. Then all the files in which this string occured should be deleted and only 1 file should remain with the string. Example. file1 ShortName "Blue... (2 Replies)
Discussion started by: premier_de
2 Replies

10. Shell Programming and Scripting

How to delete first 5 lines and last five lines in all text files

Hi I want to delete first five and last five lines in text files without opening the file and also i want to keep the same file name for all the files. Thanks in advance!!! Ragav (10 Replies)
Discussion started by: ragavendran31
10 Replies
Login or Register to Ask a Question