use argument file with sed command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting use argument file with sed command
# 1  
Old 05-12-2009
use argument file with sed command

I am trying to delete lines from a file with the sed command.

I have two files. File a.txt 200,000 records. Here is a sample of the file.
File a.txt
2401ABC0212345678 plus 250 characters of other data
2401ABC0212345678 plus 250 characters of other data
2402ABC0212345678 plus 250 characters of other data
2403ABC0212345678 plus 250 characters of other data
2405ABC0298765432 plus 250 characters of other data
2407ABC0276543211 plus 250 characters of other data
2407ABC0265432198 plus 250 characters of other data
2401ABC0212345678 plus 250 characters of other data
2401ABC0245231448 plus 250 characters of other data
2402ABC0212345678 plus 250 characters of other data
2403ABC0222245678 plus 250 characters of other data
2405ABC0247895678 plus 250 characters of other data
2407ABC0212345678 plus 250 characters of other data
2407ABC0212345678 plus 250 characters of other data

File B has about 5,000 records.
2401ABC0212345678
2402ABC0212345678
2407ABC0265432198

If the first 18 characters of a line in file a.txt matches an entry from file b.txt I need to delete the line from file a.txt

Normally file B only contains one or two entires so I use
sed 's/2401ABC0212345678//' a.txt > a.txt

Since I don't want to run the command 5,000 time I am looking for a way to feed the entries from file b.txt into my
sed command
sed 's/next line from file b.txt //' a.txt > a.txt

If there is a better way to do this than with the sed command I am open to any other suggestions.
Thanks for your help.
# 2  
Old 05-12-2009
Code:
nawk 'FNR==NR {a[$1];next} !($1 in a)}' b.txt a.txt

# 3  
Old 05-14-2009
First let me say thank you this worked great.

I am new to awk and have been trying to dissect your answer so I can expand it for future uses. Here is what I have come up with. If you have time to provide any additional explanation it would be appreciated.

nawk
'
FNR==NR
I don't understand what this is doing.

FNR is the current record number in the current file. FNR is
incremented each time a new record is read.

NR is the number of input records awk has processed since the
beginning of the program's execution.

{a[$1];next}
I think the "a "refers to the first input file, an the $1 is the first
column of data with in that file.

!($1 in a)}'
The ! is NOT. and $1 again refers to the first column of data

b.txt
a.txt
# 4  
Old 05-14-2009
Code:
nawk '
# FNR will equal NR for the FIRST file specified on the command line - read the 
# definition of FNR and NR in 'man nawk'
# The associated action '{...}' executed for the FIRST file.
# 'a' is the name of an array indexed by the FIRST field ($1) of the current
# record/line from the FIRST file.
# 'next' - skip the rest of the actions, and go on to the 'next' record of the
# current file.
FNR==NR {a[$1];next} 

# We get here when the FIRST file has already been completely processed 
# when the ABOVE condition (FNR== NR) is no longer 'true' - processing 
# SECOND file from the command line
# If the value in the FIRST column ($1) in the current record/line is NOT (!) 
# in arra 'a' (populated from the FIRST file), print the current record/line.
# The absence of the ACTION associated with the condition is the same
# as printing the ENTIRE record/line.
!($1 in a)}
# End of code - specifying the input files: b.txt is the FIRST, a.txt is the SECOND.
' b.txt a.txt

HTH
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Python script to grep fields and use values from file 1 as argument for another command

Hi Experts, I am working one one python script in version 3.x and 2.6. Need your support to complete it Basically for both commands i have telnet to device and run command and then receiving input File 1 and File 2 I have two commands, need to grep data and output in csv file. Next script/code... (0 Replies)
Discussion started by: as7951
0 Replies

2. UNIX for Beginners Questions & Answers

Creating file and passing argument to a command

Hi All, I am having command to run which will take argument as input file. Right now we are creating the input file by cat and executing the command ftptransfer -i input file cat >input file file1 file2 cntrl +d Is there a way I can do that in a single command like ... (1 Reply)
Discussion started by: arunkumar_mca
1 Replies

3. Shell Programming and Scripting

How to pass each line of a text file as an argument to a command?

I'm looking to write a script that takes a .txt filename as an argument, reads the file line by line, and passes each line to a command. For example, it runs command --option "LINE 1", then command --option "LINE 2", etc. I am fetching object files from a library file, I have all the object file... (2 Replies)
Discussion started by: Paul Martins
2 Replies

4. Shell Programming and Scripting

Renice command with input argument as a file

Hi all could you help me how to give pids in a file as an argument to renice command. Thanks in Advance.. :) Pradeep (4 Replies)
Discussion started by: nanz143
4 Replies

5. Shell Programming and Scripting

Specify an entire UNIX command as a command line argument

I'm trying to write a bash script called YN that looks like the following YN "Specify a question" "doThis" "doThat" where "doThis" will be executed if the answer is "y", otherwise "doThat". For example YN "Do you want to list the file dog?" "ls -al dog" "" Here's my attempt... (3 Replies)
Discussion started by: LeoKSimon
3 Replies

6. Shell Programming and Scripting

Make script that run with argument if not run from configuration file argument

Hello, Is there any method thorugh which script can take argument if pass otherwise if argument doesn't pass then it takes the argument from the configuration file i.e I am workiing on a script which will run through crontab and the script will chekout the code ,zip and copy to the... (3 Replies)
Discussion started by: rohit22hamirpur
3 Replies

7. UNIX for Dummies Questions & Answers

Argument list too long for Sed command

Hi guys Following command results in sed -i 's/#/\\#/g' /home/test/sqlstents* -bash: /bin/sed: Argument list too long Please help me solve it.. is there any other way i can do this?.. thanks (4 Replies)
Discussion started by: depakjan
4 Replies

8. Shell Programming and Scripting

Cannot compare argument in if statement in csh/grep command if argument starts with “-“

If ($argv == “-debug”) then Echo “in loop” Endif But this is not working. If I modify this code and remove “-“, then it works. Similarly I am getting problem using grep command also Grep “-debug” Filename Can someone please help me on how to resolve these... (1 Reply)
Discussion started by: sarbjit
1 Replies

9. Shell Programming and Scripting

about an argument with variable in sed

hi all let say I run the ./xxx.bash x x x x or ./xxx.bash x x x or ./xxx.bash x x x x x the last argument always a filename the last arugment filename format is 5-10-22.txt my question is how can I put this arugment into variable and I can use it in sed or any other way that i can use I... (1 Reply)
Discussion started by: yuesko
1 Replies

10. Shell Programming and Scripting

assign a command line argument and a unix command to awk variables

Hi , I have a piece of code ...wherein I need to assign the following ... 1) A command line argument to a variable e.g origCount=ARGV 2) A unix command to a variable e.g result=`wc -l testFile.txt` in my awk shell script When I do this : print "origCount" origCount --> I get the... (0 Replies)
Discussion started by: sweta_doshi
0 Replies
Login or Register to Ask a Question