Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help with how to search a file for a variable string and delete that line
Top Forums UNIX for Beginners Questions & Answers Need help with how to search a file for a variable string and delete that line Post 303031186 by newbie_01 on Saturday 23rd of February 2019 12:21:56 AM
Old 02-23-2019
Need help with how to search a file for a variable string and delete that line
Hi,

I have a working script.

It does what I am intending it to but a bit confused whether the sed part is supposed to be working or not. Further down is the script with the sed part that should have been working but not and the grep -v part which is the workaround that I am using at the moment.

I am posting this to get some help on how to use sed to get it to work. I've run out of things to try. OS is Solaris 5.9. Not sure how to check the sed version. I can't use sed -i, it gives error.

list.all is the input file that we want to search and delete line. Below is just an example. The original file is 10K lines long, it contains the absolute file path plus some other commands of sorts in between.

Code:
  
  '/one/one/one/one',
  '/two/two/two/two',
  '/three/three/three/three',
  '/four/four/four/four',
  '/five/five/five/five',
  '/six/six/six/six',
  '/seven/seven/seven/seven',
  '/eight/eight/eight/eight',
  '/nine/nine/nine/nine',
  '/ten/ten/ten/ten',

list.err contain the string that we want to search and delete line from list.all. We want to search for the string that is the first field and delete line containing that string.

Code:
/one/one/one/one: No such file or directory
/three/three/three/three: No such file or directory
/five/five/five/five: No such file or directory
/seven/seven/seven/seven: No such file or directory
/nine/nine/nine/nine: No such file or directory

Below is the script that I am using at the moment:

Code:
$: cat fix.ksh
#!/bin/ksh
#

source="list.all"
cp -p ${source}.save ${source}

echo
echo "------------------------------------------"
echo
echo "- This is the list that we want to cleanse"
echo
cat ${source}
echo
echo "------------------------------------------"
echo

echo ""
echo "============================"
echo "- sed not working :("
echo "============================"
echo ""
count=0
while read line
do
   let count=$count+1
   datafile=`echo $line | awk -F":" '{ print $1 }'`
   echo "- $count // Checking $datafile ... "
   grep -i "${datafile}" ${source}
   sed "#${datafile}#d" ${source} > ${source}.tmp
   cp -p ${source}.tmp ${source}
   echo ""
done < list.err
echo
echo "==> Not working :("
echo "Contents of ${source} is as below:"
echo
cat ${source}

echo
echo "============================"
echo "- Using grep -v "
echo "============================"
echo
cp -p ${source}.save ${source}
count=0
while read line
do
   let count=$count+1
   datafile=`echo $line | awk -F":" '{ print $1 }'`
   echo "- $count // Checking $datafile ... "
   grep -v "${datafile}" ${source} > ${source}.tmp
   cp -p ${source}.tmp ${source}
   echo ""
done < list.err
echo
echo "==> Required Working Output:"
echo "Contents of ${source} is as below:"
echo
cat ${source}

echo
echo "============================"
echo "- Using grep and sed"
echo "============================"
echo
cp -p ${source}.save ${source}
count=0
while read line
do
   let count=$count+1
   datafile=`echo $line | awk -F":" '{ print $1 }'`
   echo "- $count // Checking $datafile ... "
   n=`grep -n "${datafile}" ${source} | awk -F":" '{ print $1 }'`
   sed "${n}d" ${source} > ${source}.tmp
   cp -p ${source}.tmp ${source}
   echo ""
done < list.err
echo
echo "==> Required Working Output:"
echo "Contents of ${source} is as below:"
echo
cat ${source}

Below is an example run of the script:

Code:
$: ./fix.ksh

------------------------------------------

- This is the list that we want to cleanse

  '/one/one/one/one',
  '/two/two/two/two',
  '/three/three/three/three',
  '/four/four/four/four',
  '/five/five/five/five',
  '/six/six/six/six',
  '/seven/seven/seven/seven',
  '/eight/eight/eight/eight',
  '/nine/nine/nine/nine',
  '/ten/ten/ten/ten',

------------------------------------------


============================
- sed not working :(
============================

- 1 // Checking /one/one/one/one ...
  '/one/one/one/one',

- 2 // Checking /three/three/three/three ...
  '/three/three/three/three',

- 3 // Checking /five/five/five/five ...
  '/five/five/five/five',

- 4 // Checking /seven/seven/seven/seven ...
  '/seven/seven/seven/seven',

- 5 // Checking /nine/nine/nine/nine ...
  '/nine/nine/nine/nine',


==> Not working :(
Contents of list.all is as below:

  '/one/one/one/one',
  '/two/two/two/two',
  '/three/three/three/three',
  '/four/four/four/four',
  '/five/five/five/five',
  '/six/six/six/six',
  '/seven/seven/seven/seven',
  '/eight/eight/eight/eight',
  '/nine/nine/nine/nine',
  '/ten/ten/ten/ten',

============================
- Using grep -v
============================

- 1 // Checking /one/one/one/one ...

- 2 // Checking /three/three/three/three ...

- 3 // Checking /five/five/five/five ...

- 4 // Checking /seven/seven/seven/seven ...

- 5 // Checking /nine/nine/nine/nine ...


==> Required Working Output:
Contents of list.all is as below:

  '/two/two/two/two',
  '/four/four/four/four',
  '/six/six/six/six',
  '/eight/eight/eight/eight',
  '/ten/ten/ten/ten',

============================
- Using grep and sed
============================

- 1 // Checking /one/one/one/one ...

- 2 // Checking /three/three/three/three ...

- 3 // Checking /five/five/five/five ...

- 4 // Checking /seven/seven/seven/seven ...

- 5 // Checking /nine/nine/nine/nine ...


==> Required Working Output:
Contents of list.all is as below:

  '/two/two/two/two',
  '/four/four/four/four',
  '/six/six/six/six',
  '/eight/eight/eight/eight',
  '/ten/ten/ten/ten',

I used
Code:
sed "#${datafile}#d" ${source}

instead of
Code:
sed "/${datafile}/d" ${source}

because the latter gives the error
Code:
First RE may not be null

For the moment, I stick with the grep -v option because the grep -n and sed "${n}d" option doesn't work if there is multiple match of the variable string. I've tried some of the ones posted to the forum that has a similar question and they doesn't work for me. The sed that I have doesn't allow the sed -i option.

Any advise much appreciated. Thanks in advance.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search for string in a file and extract another string to a variable

Hi, guys. I have one question: I need to search for a string in a file, and then extract another string from the file and assign it to a variable. For example: the contents of the file (group) is below: ... ftp:x:23: mail:x:34 ... testing:x:2001 sales:x:2002 development:x:2003 ...... (6 Replies)
Discussion started by: daikeyang
6 Replies

2. Shell Programming and Scripting

search string and delete the line

Hi All, I have a file from Mainframe which has one of the lines with so many words... i tried to fold, format to 80 chararcter.. stil did not work. So i have decided to search for a string in that line Ex.FLIGHT PLAN and once if it is found i want to delete the entire line. Please help... (2 Replies)
Discussion started by: digitalrg
2 Replies

3. Shell Programming and Scripting

search a string in a line and save it in a variable

Hi I want to read a file line by line and search for a particular string in each line(say for example string containing @ )and save that string into a variable. Can someone suggest me the way to implement it.I am using K- shell Thanks Ishita (5 Replies)
Discussion started by: Ishita
5 Replies

4. Shell Programming and Scripting

Search for a line, delete a string in it

let me start out by saying i have ZERO exp with any kind of scripting, so sorry if this is really basic stuff..... For example, I need to have a script that will search a file and find this line in the file: *.cat;dog;kennel;house;barn;horse;hay;coat hat and remove the "coat" from the... (12 Replies)
Discussion started by: skunky
12 Replies

5. Shell Programming and Scripting

Grep a string from input file and delete next three lines including the line contains string in xml

Hi, 1_strings file contains $ cat 1_strings /home/$USER/Src /home/Valid /home/Review$ cat myxml <projected value="some string" path="/home/$USER/Src"> <input 1/> <estimate value/> <somestring/> </projected> <few more lines > <projected value="some string" path="/home/$USER/check">... (4 Replies)
Discussion started by: greet_sed
4 Replies

6. UNIX for Dummies Questions & Answers

search for a string and delete it from the file

Hi , I am breaking my head from past one day ...to delete lines from a file which match to the string pattern.:wall: I am storing the search string in a variable and search if the file exists in the folder,if not delete that entry from the file. I am having problem to delete that line from... (2 Replies)
Discussion started by: rashmisb
2 Replies

7. Shell Programming and Scripting

Sed find exact string and delete line with variable

All, I am trying to read in a variable and search a file then delete based on that string, but i want to match exact word. This works but it matches all, i don't want to match anthing that contains the string, just the exact string. sed -i "/$feedname/d" file I tried sed... (1 Reply)
Discussion started by: markdjones82
1 Replies

8. Shell Programming and Scripting

Search a string in a text file and add another string at the end of line

Dear All I am having a text file which is having more than 200 lines. EX: 001010122 12000 BIB 12000 11200 1200003 001010122 2000 AND 12000 11200 1200003 001010122 12000 KVB 12000 11200 1200003 In the above file i want to search for string KVB... (5 Replies)
Discussion started by: suryanarayana
5 Replies

9. Shell Programming and Scripting

Search string within a file and list common words from the line having the search string

Hi, Need your help for this scripting issue I have. I am not really good at this, so seeking your help. I have a file looking similar to this: Hello, i am human and name=ABCD. How are you? Hello, i am human and name=PQRS. I am good. Hello, i am human and name=ABCD. Good bye. Hello, i... (12 Replies)
Discussion started by: royzlife
12 Replies

10. UNIX for Dummies Questions & Answers

Search for a string,delete the line and replace with new string in a file

Hi Everyone, I have a requirement in ksh where i have a set of files in a directory. I need to search each and every file if a particular string is present in the file, delete that line and replace that line with another string expression in the same file. I am very new to unix. Kindly help... (10 Replies)
Discussion started by: Pradhikshan
10 Replies
All times are GMT -4. The time now is 08:11 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy