Login or Register to Ask a Question and Join Our Community


Linux

Ambiguous redirect error and syntax error when using on multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Linux Ambiguous redirect error and syntax error when using on multiple files
# 8  
Old 08-19-2012
Quote:
Originally Posted by Madhusudan Das
Hi Don, Yes, that true I need to delete all lines from the file in current directory not having word "ASPRODUCTSUMMARY" on 15th position.
No. It will delete all lines no matter what is on them; not just lines not having the text you're looking for.
The use of any utility, for example x, will get rid of all data in file when you issue the command:
Code:
x < file > file

This will be done by the shell before xeven starts running.
Quote:
Originally Posted by Madhusadan
Also the thing you mentioned of the code line :
Code:
echo $files $(mv -v $files 'UNPROCESSED/ASPRODUCTSUMMARY/'${files%}.CSV)

is missing the pattern to be removed, I corrected it to be
Code:
echo $files $(mv -v $files 'UNPROCESSED/ASPRODUCTSUMMARY/'${files%.MCESOLUTION}.CSV)

main problem is for loop syntax having while loop, which is giving syntax error:
MCEASPRODUCTSUMMARY.sh: line 26: syntax error near unexpected token `done'
'CEASPRODUCTSUMMARY.sh: line 26: `echo "${prefix}$line"; done< "$file" > temp.CSV;

Now I am not getting ambiguous redirect, but snytax error.
Line 20 in your script:
Code:
for file in '$(ls -l $FOLDERDIR)'; do

sets $file to be a line of output from ls -l when it seems highly unlikely that you want an expansion of $file to be anything other than the file's name. Also, since line 11 of your script is:
Code:
cd $FOLDERDIR;

and you haven't changed directory between these two lines, I think you just want line 20 to be:
Code:
for file in '$(ls)'; do

You probably got lots of syntax errors from using $file instead of "$file" on lines 21 and 22. Even when $(ls) instead of $(ls -l), if any of your filenames contain characters that have meaning to the shell you're using, you need to use double-qoutes around $file. You should get into the habit of surrounding the expansion of any variable that may contain characters that the script itself did not produce. You never know when a user will try to hack your script by including characters that are special to the shell!
This User Gave Thanks to Don Cragun For This Post:
Madhusudan Das
# 9  
Old 08-19-2012
syntax Error on Multiple files
Hi Don, I truly appreciate your help.
Here is the script I changed and have placed "$file" everywhere it was.
Script is working fine upto line 13 (where grep command is redirecting to newtemp.csv file (with a bug that it will do it for the first file and second file remains untouched after it was moved to the ASPRODUCTSUMMARY subfolder.
Also giving syntax error at line 15 where there is
Code:
for file in $(ls -l)
do

.
I changed everything as you suggested. I am attaching some sample files that this script is processing.

Code:
#!/bin/bash
# MCE Repricing ASPRODUCTSUMMARY

cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS;
for files in $(ls *.MCESOLUTION);do $(cp -v $files 'UNPROCESSED/ASPRODUCTSUMMARY/'${files%.MCESOLUTION}.CSV); done;
cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY;
####ls -1 /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY > ListofMCECSVFilesASPRODUCTSUMMARY.txt
FOLDERDIR='/u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY';
cd $FOLDERDIR;

for i in $(ls -l | grep '^-' | awk '{print $9}')
do  
$(grep -i "^..............ASPRODUCTSUMMARY"< "$i" > "newtemp.csv");$(mv -v newtemp.csv "$i"); done;
for file in $(ls -l)
do
caseid=$(echo "$file" | cut -c1-10)
studytype=$(echo "$file" | cut -c12-15)
prefix="$caseid,$studytype,"
while read -r line
do echo "${prefix}$line" ;done< "$file" > "temp.CSV"; $(mv -v temp.CSV "$i");
done;
exit 0;

below is the latest script output error :
Quote:
[madhu@devinfaav POPS]$ bash MCEASPRODUCTSUMMARY.sh
: command not found.sh: line 3:
: command not found.sh: line 4:
MCEASPRODUCTSUMMARY.sh: line 5: `1400009823_RTBM_20120614_1715_0_0_0_00.MCESOLUTION': command not found
MCEASPRODUCTSUMMARY.sh: line 5: `1400009823_RTBM_20120614_1715_0_0_0_02.MCESOLUTION': command not found
: command not found.sh: line 5:
: command not found.sh: line 6:
: command not found.sh: line 8:
: command not found.sh: line 9:
: command not found.sh: line 10:
: command not found.sh: line 12:
MCEASPRODUCTSUMMARY.sh: line 13: `newtemp.csv': command not found
: command not found.sh: line 12:
MCEASPRODUCTSUMMARY.sh: line 13: `newtemp.csv': command not found
: command not found.sh: line 12:
MCEASPRODUCTSUMMARY.sh: line 13: `newtemp.csv': command not found
: command not found.sh: line 12:
: No such file or directorye 13: 1400009823_RTBM_20120614_1715_0_0_0_02.CSV
mv: cannot stat `newtemp.csv': No such file or directory
: command not found.sh: line 13:
'CEASPRODUCTSUMMARY.sh: line 15: syntax error near unexpected token `do
'CEASPRODUCTSUMMARY.sh: line 15: `do
[madhu@devinfaav POPS]$
Error on line 12 newtemp.csv is rather processing correctly for 1 out of 2 files, and didn't process the second file and left it as it is. then gives error
Quote:
: command not found.sh: line 12:
: No such file or directorye 13: 1400009823_RTBM_20120614_1715_0_0_0_02.CSV
And final error on line 15 :
Quote:
'CEASPRODUCTSUMMARY.sh: line 15: syntax error near unexpected token `do
'CEASPRODUCTSUMMARY.sh: line 15: `do
.

Thank you very much
# 10  
Old 08-19-2012
Please provide the basic information I requested in Post #5.

Bottom line: Your script is a disaster. It needs a total rewrite. Only certain syntax issues are stopping the script destroying all files in the directory (as others have advised).
Please stop running trial-and-error on live data, and publish the specification for the process.
# 11  
Old 08-19-2012
syntax Error on Multiple files
Hi Methyle,

Thank you for your concern and help man. I agree the script must be a disaster for sure.
The Specifications are :

1) Will copy the data to a subfolder and rename them as .CSV files. Which now successfully being achieved by the following code:
Code:
#!/bin/bash
# copying MCE Repricing ASPRODUCTSUMMARY

cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS;
for files in $(ls *.MCESOLUTION);do $(cp -v $files 'UNPROCESSED/ASPRODUCTSUMMARY/'${files%.MCESOLUTION}.CSV); done;
exit 0;

2) Delete all the lines from each file which are not having "ASPRODUCTSUMMARY" word in the file in the subdirectory files were copied(that's why the original files provided were copied so that we don't destroy the entire data but have our own version of file copied and deleting all unwanted lines). This is also being successfully achieved by the following code:
Code:
#!/bin/bash
# MCE Repricing ASPRODUCTSUMMARY ONLY DATA

cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY;
####ls -1 /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY > ListofMCECSVFilesASPRODUCTSUMMARY.txt
FOLDERDIR='/u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY';
cd $FOLDERDIR;

for i in $(ls -l | grep '^-' | awk '{print $9}')
do  
$(grep -i "^..............ASPRODUCTSUMMARY"< "$i" > newfile);
$(mv -v "newfile" "$i"); done;
exit 0;

3) In each file modified with the lines required. Want append each line of each file with CASEID and STUDYTYPE which are extracted from the filename itself. For example if the filename is 1400009823_RTBM_20120614_1715_0_0_0_002.CSV so the caseid ='1400009823' and studytype='RTBM'. So this should repeat for each file and append in front of each line. This is the challenge (Smilie) right now.
The code for this which is giving syntax error :
Quote:
line 10:syntax error near unexpected token `do
When running while loop for individual file one by one it runs perfectly fine, but gives syntax error when trying to run for multiply files using "for loop".
Code:
#!/bin/bash
# MCE Repricing ASPRODUCTSUMMARY Append Casid n studytype in each line

cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY;
####ls -1 /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY > ListofMCECSVFilesASPRODUCTSUMMARY.txt
FOLDERDIR='/u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY';
cd $FOLDERDIR;

for i in '$(ls -l)'
do
caseid=$(echo "$i" | cut -c1-10)
studytype=$(echo "$i" | cut -c12-15)
prefix="$caseid,$studytype,"
while read -r line
do echo "${prefix}$line" ;done< "$i" > "temp.CSV"; $(mv -v "temp.CSV" "$i");
done;
exit 0;

I did try putting
Code:
for i in "$(ls -l)"

on line 10 in single quote, double quote. ; before do, nothing helps resolving the syntax error.
Thank you
Last edited by Madhusudan Das; 08-19-2012 at 10:35 PM..
# 12  
Old 08-19-2012
Please stop tinkering with syntax in a fundamentally faulty script and provide the basic specification for the script.
# 13  
Old 08-19-2012
Methyle BASIC Specifications are :
1) Will copy the data to a subfolder and rename them as .CSV files.
2) Delete all the lines from each file which are not having "ASPRODUCTSUMMARY" word in the file in the subdirectory files were copied
3) append each line of each file with CASEID and STUDYTYPE which are extracted from the filename itself. For example if the filename is 1400009823_RTBM_20120614_1715_0_0_0_002.CSV so the caseid ='1400009823' and studytype='RTBM'. So this should repeat for each file and append in front of each line
# 14  
Old 08-19-2012
Quote:
Originally Posted by Madhusudan Das
Hi Don, I truly appreciate your help.
Here is the script I changed and have placed "$file" everywhere it was.
Script is working fine upto line 13 (where grep command is redirecting to newtemp.csv file (with a bug that it will do it for the first file and second file remains untouched after it was moved to the ASPRODUCTSUMMARY subfolder.
Also giving syntax error at line 15 where there is
Code:
for file in $(ls -l)
do

.
I changed everything as you suggested. I am attaching some sample files that this script is processing.

Code:
#!/bin/bash
# MCE Repricing ASPRODUCTSUMMARY

cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS;
for files in $(ls *.MCESOLUTION);do $(cp -v $files 'UNPROCESSED/ASPRODUCTSUMMARY/'${files%.MCESOLUTION}.CSV); done;
cd /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY;
####ls -1 /u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY > ListofMCECSVFilesASPRODUCTSUMMARY.txt
FOLDERDIR='/u02/sppdw/dw/infa_shared/SrcFiles/POPS/UNPROCESSED/ASPRODUCTSUMMARY';
cd $FOLDERDIR;

for i in $(ls -l | grep '^-' | awk '{print $9}')
do  
$(grep -i "^..............ASPRODUCTSUMMARY"< "$i" > "newtemp.csv");$(mv -v newtemp.csv "$i"); done;
for file in $(ls -l)
do
caseid=$(echo "$file" | cut -c1-10)
studytype=$(echo "$file" | cut -c12-15)
prefix="$caseid,$studytype,"
while read -r line
do echo "${prefix}$line" ;done< "$file" > "temp.CSV"; $(mv -v temp.CSV "$i");
done;
exit 0;

below is the latest script output error :

Error on line 12 newtemp.csv is rather processing correctly for 1 out of 2 files, and didn't process the second file and left it as it is. then gives error
And final error on line 15 :
.

Thank you very much
You didn't make all of the changes I suggested and you made some changes I didn't suggest. Let's analyze the commands being run inside the for loop. (I've removed the semicolons and split the commands onto separate lines to make it easier to read.) With the reformatting, you have:
Code:
$(grep -i "^..............ASPRODUCTSUMMARY"< "$i" > "newtemp.csv")
$(mv -v newtemp.csv "$i")

Since you have $(...) around both of these commands, the shell executes the grep (which succeeded the first time) and then tried to execute the output produced by the grep. I assume this gave you the first command not found error.

Then the mv command executes producing output that looks something like
Quote:
newtemp.csv -> some_other_file_name
and then tries to execute that string (which I assume gave you a
Quote:
newtemp.csv not found
error message.) Remove the $( and the matching ) from both of these commands and you should make some progress.

Then you have a for loop starting with:
Code:
for file in $(ls -l)

this executes the loop nine times for every file found in the directory. The first time through the loop, $file will be something like -rw-r--r--; the second time through the loop, $file will be the number of links to the file; ...; and the ninth time through the loop for each file, $file will be set to the name of a file. I had suggested that you change your original '$(ls -l $FOLDERDIR)' to '$(ls)'. I made a mistake leaving in the single quotes (and you corrected that. Thank you. But leaving $(ls -l) instead of $(ls) gives you a big problem.

Then in the final while loop you have another mv -v command inside $(...) which I assume is giving you several more command not found errors. Get rid of the $( and the matching ) again here.

Please also reformat this. It has been noted (and you agreed) that this code is a disaster. It looks like you are trying to minimize the number of lines of code in your script instead of trying to make it easy to understand. This may make sense when you have a simple program that can be written as a single line; but in a case like this it just makes your script unreadable. Don't you agree that:
Code:
for i in $(ls -l | grep '^-' | awk '{print $9}')
do 
        grep -i "^..............ASPRODUCTSUMMARY" < "$i" > "newtemp.csv"        mv -v newtemp.csv "$i"
done
for file in $(ls)
do
        caseid=$(echo "$file" | cut -c1-10)
        studytype=$(echo "$file" | cut -c12-15)
        prefix="$caseid,$studytype,"
        while read -r line
        do      echo "$prefix$line"
        done < "$file" > temp.CSV
        mv -v temp.CSV "$i"
done
exit 0

is a lot easier to read and understand than:
Code:
for i in $(ls -l | grep '^-' | awk '{print $9}');do  
grep -i "^..............ASPRODUCTSUMMARY"< "$i" > "newtemp.csv"
mv -v newtemp.csv "$i";done;for file in $(ls);do
caseid=$(echo "$file" | cut -c1-10);studytype=$(echo "$file" | cut -c12-15)
prefix="$caseid,$studytype,";while read -r line;do;echo "$prefix$line"
done < "$file" > temp.CSV;mv -v temp.CSV "$i";done;exit 0

The code above is an exaggeration of the formatting you're using, but not by much.

There are some problems that will arise even with this if any of the file names encountered while running ls contain any spaces, tabs, or newlines. I will assume for now that these directories are always being updated programmatically and that other users don't have permission to create any files in these directories.
This User Gave Thanks to Don Cragun For This Post:
methyl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

$1”: ambiguous redirect

New to the site, please let me know I'm not meeting the post guidelines. I'm creating a bash script to generate a report with output from a grep command. The goal is to direct the output to a different log file by using a 'logger file'. But I get this error during the run: $1: ambiguous... (5 Replies)
Discussion started by: dallas88
5 Replies

2. Shell Programming and Scripting

Ambiguous output redirect in xterm

Hi all, I've been working on a bash script to help with backups that I have to do at work. One of the lines in the script is supposed to launch an xterm, log into a specific server node and launch a tar backup to tape. This part works ok, but I've been trying to get stdout and stderr to... (2 Replies)
Discussion started by: Exitalterego
2 Replies

3. Shell Programming and Scripting

Ambiguous error

Hello everybody, I just took over this job from someone else and in the past this script they built worked but i recently upgraded from openSuSe 11.4 to 12.1 Now when i run the script i get an ambiguous error at line 25 (the first line after add() ) I have edited out the webpath and any... (1 Reply)
Discussion started by: gumbicus
1 Replies

4. Shell Programming and Scripting

Receiving 'ambiguous redirect' when trying to run command against multiple files

I came across the command string on https://www.unix.com/shell-programming-scripting/141885-awk-removing-data-before-after-pattern.html which was what I was looking for to be able to remove data before a certain pattern. However, outputting the result to a file seems to work on an individual basis... (4 Replies)
Discussion started by: HLee1981
4 Replies

5. Shell Programming and Scripting

ambiguous redirect error

This script has ambiguous redirect error. ... cd $HOME cd folder/work # search all subfolders in work directory find -mindepth 1 -maxdepth 1 -type d | while read directory do CUR_FOLDER="${directory#"./"}" cd $CUR_FOLDER chmod 644 * for ff in *; do if ; then ... (5 Replies)
Discussion started by: candyme
5 Replies

6. Shell Programming and Scripting

Ambiguous redirect

Hello there, I'm totally new in bash programming and ran into my first problem. My script should generate 3 textfiles where the content of the first and the third row are the same in each file. Only the second row is different. This is what I did in a very simplified explanation: ... (6 Replies)
Discussion started by: johndoe
6 Replies

7. UNIX for Dummies Questions & Answers

ambiguous redirect issue

I am trying to run the following script and I am getting an "ambiguous redirect" error. I have checked to make sure that the files are all where I have specified and are read/write as needed. Any ideas? Note: I have removed the actual path info for privacy sake. I have triple checked to make... (1 Reply)
Discussion started by: malantha
1 Replies

8. Shell Programming and Scripting

> to empty files, but ambiguous redirect

Hi Everyone, # ll total 0 -rw-r--r-- 1 root root 0 2010-05-13 11:29 a1.log -rw-r--r-- 1 root root 0 2010-05-13 11:29 a2.log -rw-r--r-- 1 root root 0 2010-05-13 11:29 a3.log # rm a.log above rm no problem, but when i use "> a.log", it says "-bash: a.log: ambiguous redirect". ... (3 Replies)
Discussion started by: jimmy_y
3 Replies

9. Shell Programming and Scripting

Ambiguous output redirect error

Hi everyone, While I was trying to do DATE=`date +"%Y%m%d_%H%M%S"` STARTLOG=$TUXSTDDIR/start_$DATE.log tmboot -y > $STARTLOG 2>&1 I got an error i.e. Ambiguous output redirect error. Here the first part is to boot the account so there is nothing wrong with that.... (6 Replies)
Discussion started by: pareshan
6 Replies

10. UNIX for Dummies Questions & Answers

ambiguous redirect

i have following statement in the script echo -e "$str_XML_col_name:$str_field_type;" >> $i_DC_Key_$i_Tgt_DC_key_Schema here $i_DC_Key is DC key and $i_Tgt_DC_key are the variables............... when i ran the script i am getting error rec_merge.sh: $i_DC_Key_$i_Tgt_DC_key_Schema:... (1 Reply)
Discussion started by: mahabunta
1 Replies
Login or Register to Ask a Question