Multi Line 'While Read' command issue when using sh -c


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multi Line 'While Read' command issue when using sh -c
# 1  
Old 07-14-2011
Multi Line 'While Read' command issue when using sh -c

Hi,
I'm trying to run the following command using sh -c

ie
Code:
sh -c "while read EachLine 
do  
rm -f $EachLine ; 
done < file_list.lst;"

It doesn't seem to do anything.

When I run this at the command line, it does remove the files contained in the list so i know the command works
ie
Code:
while read EachLine 
do  
rm -f $EachLine ; 
done < file_list.lst;

What am i doing wrong when trying to run it with the sh -c command?

FYI. This is scripting for a etl tool that whenever you want to run a command, it always wraps the command in 'sh -c "..command"'

Any help would be great.

Last edited by Franklin52; 07-14-2011 at 05:07 PM.. Reason: Please use code tags for data and code samples, thank you
# 2  
Old 07-14-2011
EachLine is expanded too early.

You can use:
Code:
sh -c "while read EachLine 
do  
rm -f \$EachLine ; 
done < file_list.lst;"

or
Code:
sh -c 'while read EachLine 
do  
rm -f $EachLine ; 
done < file_list.lst;'

or simply:
Code:
sh -c 'rm -f `cat file_list.lst`'

or even, depending on your sh implementation:
Code:
sh -c 'rm -f $(<file_list.lst)'

# 3  
Old 07-14-2011
If the file is too large to fit in a shell variable some of it might get silently ignored when you use `` or $().

Code:
sh -c 'xargs -d "\n" rm -f < ile_list.lst'

# 4  
Old 07-15-2011
Quote:
Originally Posted by Corona688
If the file is too large to fit in a shell variable
I guess you mean "If the file list is too large ..."
Quote:
some of it might get silently ignored when you use `` or $().
I would expect the whole command to fail with an "Argument list too long" or similar error message.
Quote:
Code:
sh -c 'xargs -d "\n" rm -f < ile_list.lst'

-d "\n" might not be supported by xargs, being a Gnu extension. Beware there is a small typo in the file name too.
# 5  
Old 07-15-2011
Quote:
Originally Posted by jlliagre
EachLine is expanded too early.

You can use:
Code:
sh -c "while read EachLine 
do  
rm -f \$EachLine ; 
done < file_list.lst;"

or
Code:
sh -c 'while read EachLine 
do  
rm -f $EachLine ; 
done < file_list.lst;'

or simply:
Code:
sh -c 'rm -f `cat file_list.lst`'

or even, depending on your sh implementation:
Code:
sh -c 'rm -f $(<file_list.lst)'

I tried those different ideas. The one that works the best for me is:

sh -c "rm -f `cat $PMSourceFileDir/dqa_file_list.lst`"

(FYI. The etl tool expands $PMSourceFileDir to the full path before it issues the command).

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with reformat single-line multi-fasta into multi-line multi-fasta

Input File: >Seq1 ASDADAFASFASFADGSDGFSDFSDFSDFSDFSDFSDFSDFSDFSDFSDFSD >Seq2 SDASDAQEQWEQeqAdfaasd >Seq3 ASDSALGHIUDFJANCAGPATHLACJHPAUTYNJKG ...... Desired Output File >Seq1 ASDADAFASF ASFADGSDGF SDFSDFSDFS DFSDFSDFSD FSDFSDFSDF SD >Seq2 (4 Replies)
Discussion started by: patrick87
4 Replies

2. Shell Programming and Scripting

Performance issue - to read line by line

All- We have a performance issue in reading a file line by line. Please find attached scripts for the same. Currently it is taking some 45 min to parse "512444" lines. Could you please have a look at it and provide any suggestions to improve the performance. Thanks, Balu ... (12 Replies)
Discussion started by: balu1729
12 Replies

3. UNIX for Beginners Questions & Answers

Performance issue to read line by line

Hi All- we have performance issue in unix to read line by line. I am looking at processing all the records. description: Our script will read data from a flat file, it will pickup first four character and based on the value it will set up variables accordingly and appended the final output to... (11 Replies)
Discussion started by: balu1729
11 Replies

4. Shell Programming and Scripting

awk - 2 files comparison without for loop - multi-line issue

Greetings Experts, I need to handle the views created over monthly retention tables for which every new table in YYYYMMDD format, there is equivalent view created and the older table which might be dropped, the view over it has to be re-created over a dummy table so that it doesn't fail.... (2 Replies)
Discussion started by: chill3chee
2 Replies

5. Shell Programming and Scripting

Sql multi line comment /* shell interpretation issue

Greetings Experts, I do have some basic knowledge of Unix. The task I am trying to do through shell script is to generate the view script for all of the tables which is in YYYYMMDD format (I assume I am on Ksh). I have certain tables that ends in YYYYMMDD format (eg: tbl_20150630) For each... (1 Reply)
Discussion started by: chill3chee
1 Replies

6. Shell Programming and Scripting

[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter). The typewriter-part is a 'bonus' but still has its own flaws, but thats for later. So in some way, i'm trying to rewrite cat using bash and other commands. But sadly it fails on... (2 Replies)
Discussion started by: sea
2 Replies

7. Shell Programming and Scripting

Read line, issue with leading - and {}'s

Heyas With my forum search term 'issue with leading dash' i found 2 closed threads which sadly didnt help me. Also me was to eager to add the script, that i didnt properly test, and just now figured this issue. So i have this code: if ] then while read line do line="${line/-/'\-'}"... (7 Replies)
Discussion started by: sea
7 Replies

8. Shell Programming and Scripting

Multi-line filtering based on multi-line pattern in a file

I have a file with data records separated by multiple equals signs, as below. ========== RECORD 1 ========== RECORD 2 DATA LINE ========== RECORD 3 ========== RECORD 4 DATA LINE ========== RECORD 5 DATA LINE ========== I need to filter out all data from this file where the... (2 Replies)
Discussion started by: Finja
2 Replies

9. Shell Programming and Scripting

how to read multi line characters in a single column

Hi, I have file in which fields are pipe (|) separated and the record separator is new line (\n). But sometime I am getting a field value which is spread across multiple line. Basically I am getting a file from another system in the below format and needs to process the file and load data into a... (3 Replies)
Discussion started by: satyaatcgi
3 Replies

10. Shell Programming and Scripting

while read LINE issue

Hi, This is the script and the error I am receiving Can anyone please suggest ? For the exmaple below assume we are using vg01 #!/bin/ksh echo "##### Max Mount Count Fixer #####" echo "Please insert Volume Group name to check" read VG lvs |grep $VG | awk {'print $1'} > /tmp/audit.log ... (2 Replies)
Discussion started by: galuzan
2 Replies
Login or Register to Ask a Question