Shell Programming and Scripting

I need to create a shell script to delete multiple items (Strings) at a time from a file.

I need to iterate through a list of strings.

My plan is to create an array and then iterate through the array.
My code is not working

Code:

#!/bin/bash -x


declare -a array=(one, two, three, four)

for element in "${array[@]}"
do
    sed "s/"$element"//g" input
    #awk "{gsub("$element", "");print}" input
done

the file input contains
one
two
three
four

Why do you want an array?

Code:

for element in one two three four
do
...

input file contains
one
two
three
four

I moved away from the array to an input file. The script hangs as shown here.

Code:

#!/bin/bash -x


for i in `cat input`
#while true
do
    sed -e 's/'"$i"'/'" "'/g'
#    echo $i
done > output.remove

shell$ ./remove-these.sh

++ cat input
+ for i in '`cat input`'
+ sed -e 's/one/ /g'

Code:

#!/bin/bash -x

for i in `cat input`
do
    sed -e 's/'"$i"'/'" "'/g' input > output.remove
#done < input > outfile1
done

Where are the items to delete,
and what is the file you want to delete the items from?

If your items are from the file "input" then it does not make sense to delete the items from the file "input". It will become empty. Or do you want to prove just that?

The items to delete are in the input file. This script is just to prove the loop works. input is the file I want to delete from. I could use sed edit in place or not. Some internet posts said the sed -i option was not working in a loop. I did not verify.

I have an 18MB data file with a lot of redundant repeated lines that need to be deleted. Having an outside file containing a list of those lines to be deleted is best for this script. I want to read into the loop the source file and output to a second file. The source file has about 15 MB of repeated useless data to remove so looping over the file seems the best choice. I dont really care what construct I use but I am not having luck with those I have tested.

You say you want to remove useless repeated data, but the code you are using replaces every copy of the data (not just repeated data) with a <space> character (even if the data you want to "remove" is at the start, at the end, or in the middle of a longer string.

Please be very clear about:
If you want to completely remove complete lines of matched text (as in your example), consider using something more like:

Code:

grep -Fvx -f file_of_lines_to_remove file_to_be_updated > updated_file

to do the entire job in one pass instead of one pass per word "removed". If you aren't removing complete lines, consider using awk to process each input file once instead of using sed to process each input file once per word to be "removed".