SED help (remove line::parse again::add line)

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting SED help (remove line::parse again::add line)
# 1  
Old 10-23-2007
Question SED help (remove line::parse again::add line)

Aloha! I have just over 1k of users that have permissions that they shouldn't under our system. I need to parse a provided list of usernames, check their permissions file, and strip the permissions that they are not allowed to have. If upon the permissions strip they are left with no permissions, I need have a basic permissions line added to their permission file. I am creating this script so our helpdesk guys can use it later because I can see something similar happening again.

Everything on the following script works correctly unless the user has no permissions left. The sed statement will display the correct insertion but will not overwrite the file. I have also tried piping it to cat and then redirect to the file but that just appends....Please Help!! Smilie

Oh! BTW this is in Korn shell under AIX 5.3
Also FYI permission lines are always first in the file and they are the only lines that start with numerals.

echo "Where is the user Listing?"

read usrList



while [ 1 ]


  read usrName || break

      sudo sed -e '/APCLERK/d' /home/$usrName/.facet | cat >> /home/testdir/$usrName
      fChar=`head -c 1 /home/testdir/$usrName`

          if [[ $fChar != [0-9] ]]


            sudo sed -f /home/testdir/sedInqIns.sed /home/testdir/$usrName > /home/testdir/$usrName



done < $usrList

Here is the sed insert file::

1       /PRISM/SD-START "INQUIRY~      ; ; ; ;Y;   ;    ;0000;      ;    ;Y;      ; ;      ;     ~~"

On execution of this script the users that have no permissions are left with a completely blank file (there should be other settings in the file besides permissions). The users that have other permissions are fine.

Any help will be greatly appreciated.
Mahalo!! Smilie

Last edited by Malumake; 10-23-2007 at 09:17 PM.. Reason: Additional Information
# 2  
Old 10-24-2007
I suspect the problem to be the following:

After changing the file /home/$usrName/.facet and writing the results to /home/testdir/$usrName (so far ok) you try to modify this file writing back to it immediately. This is FORBITTEN! Something like

sed 's/this/that' file >

works, but

sed 's/this/that' file > file

won't. The reason is: sed will read the first line of the input file (which is "file"), apply any rules that might be applicable and then write the result (the changed or not changed line) to <stdout>. This <stdout> is at this moment pointing to "file" and this is why after the first line the file will contain nothing more than exactly that first line - so sed encounters the EOF and thinks it is finished. In the first example it is not writing back to its input file but some other file "", which is why there is no problem.

There is another (perhaps minor) detail i found: You read from keyboard a variable "usrList" and do nothing with it. Why?

# 3  
Old 10-24-2007
To Bakunin

just a minor bit of pedantry - usrList is being read via the '< usrList' statement at the end of the 'read.....' clause - or so it seems to me......???!

# 4  
Old 10-24-2007

Sorry - I take that back - it doesn't seem to be being used at all. I should have kept my mouth shut!
# 5  
Old 10-24-2007
Originally Posted by bakunin
The reason is: sed will read the first line of the input file (which is "file"), apply any rules that might be applicable and then write the result (the changed or not changed line) to <stdout>. This <stdout> is at this moment pointing to "file" and this is why after the first line the file will contain nothing more than exactly that first line - so sed encounters the EOF and thinks it is finished.
I would have explained this behaviour somewhat differently: when the shell begins to process a command line, it will literally remove the characters that involve re-direction. In dealing with re-direction, the shell will "zero" a file that is being re-directed to. If there was data on the file, it will have been erased before the command is executed. The result is that the command -- in this case sed -- will never see any input, because it already has been destroyed. The first read will encounter EOF.

Most of us have fallen into this operational trap. For example, we often want to sort a file onto itself -- just re-ordering data, neither gaining or losing data. So sort has an option to write the output to the input file, but one must use "-o file", and not "> file". The GNU sed allows a similar construction with "in-place" editing, but what actually happens is that sed writes to a temporary file and then renames it at the end. You can see this if you display the inode number (ls -li file) before and after the operation. You'll see that they are different.

The flow-chart of shell operations is displayed in some books, for example, O'Reilly's Learning the bash shell, 2nd, page 177 ff ... cheers, drl
# 6  
Old 10-24-2007

Believe it or not usrList actually feeds line by line into the do loop.

So, what I'm getting, is that in order for this to work correctly, I need to feed the intial sed statement to a temp file and then have the second sed use the temp back into the original? Smilie
# 7  
Old 10-24-2007
It Worked!

I changed the script to push to a temp file in the first sed, it works fine now.

Mahalo for all the help!!SmilieSmilie
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sed/grep: check if line exists, if not add line?

Hello, I'm trying to figure out how to speed up the following as I want to use multiple commands to search thousands of files. is there a way to speed things up? Example I want to search a bunch of files for a specific line, if this line already exists do nothing, if it doesn't exist add it... (4 Replies)
Discussion started by: f77hack
4 Replies

2. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

3. UNIX for Dummies Questions & Answers

Sed to remove only first line erroneously removes last line too

Hello everyone, This is my first posting. I have read the rules of this forum. I have searched many various threads and haven't found one that applies to my situation or suggestions to fix the issue. I do appreciate the help. I am trying to execute a basic UNIX script in a Solaris... (4 Replies)
Discussion started by: dqrgk0
4 Replies

4. Shell Programming and Scripting

Parse configuration file & add line in particular section

Greetings, I recently built a replicated DRBD, Heartbeat, & iSCSI Target Initiator storage server on Ubuntu 10.04 to offer shared storage to server Vmware ESX and Microsoft Clusters. Everything works flawlessly, however I wanted to make a script to create, remove, grow volumes to offer ESX... (6 Replies)
Discussion started by: Aeudian
6 Replies

5. Shell Programming and Scripting

Help to Add and Remove Records only from first line/last line

Hi, I need help with a maybe total simple issue but somehow I am not getting it. I am not able to etablish a sed or awk command which is adding to the first line in a text and removing only from the last line the ",". The file is looking like follow: TABLE1, TABLE2, . . . TABLE99,... (4 Replies)
Discussion started by: enjoy
4 Replies

6. Shell Programming and Scripting

SED remove line feed and add to certain area

Hi All, I have a xml file and requirement is to remove the line feed and add line feed after some element. <?xml version="1.0" ?> <AUDITRECORDS> <CARF> <HED> <VN1>20090616010622</VN1> <VN2>0</VN2> <VN3>1090</VN3> <VN4>CONFIG_DATA</VN4> ... (8 Replies)
Discussion started by: sreejitnair123
8 Replies

7. UNIX for Advanced & Expert Users

how do you parse 1 line at a time of file1 ie. line(n) each line into new file

File 1 <html>ta da....unique file name I want to give file=>343...</html> <html>da ta 234 </html> <html>pa da 542 </html> and so on... File 2 343 234 542 and so on, each line in File 1 one also corresponds with each line in File 2 I have tried several grep, sed, while .. read, do,... (4 Replies)
Discussion started by: web_developer
4 Replies

8. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

9. Shell Programming and Scripting

Parse Line Using Sed

Hello All, I am new to using sed, and I need to extract from the string data after : delimeter. Can you help me please with the sed command? Here's the input: ipAddress: ioIpAddressNodeB: ioIpAddressNodeA: ipAddress: Expected Output:... (7 Replies)
Discussion started by: racbern
7 Replies
Login or Register to Ask a Question