I posted on here a while ago about processing multiple files within a script.
See original post below:
I have a program cfxfrwb which is designed to remove headers from reports files.
The cfxfrwb is located in the following directory /u01/efin/v40/live/bin
I run the program against a single report file in the temp directory and it does it's job.
I'd like to run the ./cfxfrwb against several report files.
I modified a script I found on the web but it doesn't seem to work
Code:
#!/bin/bash
FILES=/home/efin/efintran/temp/*
for f in $FILES
do cfxfrwb
echo "Processing $f file..."
# take action on each file. $f store current file name
cat $f
done
I'd be grateful if someone could offer some assistance.
Many thanks
The problem was solved by using the following:
Code:
#!/bin/bash
FILES=/home/efin/efintran/temp/*
for f in $FILES
do
/u01/efin/v40/live/bin/cfxfrwb $f
echo "Processing $f file..."
# take action on each file. $f store current file name
We have recently upgraded our systems and my script has stopped working again. We have different file paths.
The cfxfrwb program will work if it runs against files individually
for example /u01/efin/v42/live/bin/cfxfrwb act_r09r3817.prt ...this processes the file act_r09r3817.prt as expected and removes it's header.
when it is executed within the script it returns an error:
Code:
Removing headers from report files \n
./efdsx_trans: line 209: 18443 Segmentation Fault (core dumped) /u01/efin/v42/live/bin/cfxfrwb $f
Processing /export/home/efin/efintran/temp/act_r09r3814.prt file...
We have discovered that the cfxfrwb program will only work when the environment has been set. I have looked at the settings within the environment and included them in my script but it still returns core dump errors.
Anyone have any ideas?
Many thanks
Moderator's Comments:
Please use code tags for your code and data, thanks
Last edited by vbe; 02-18-2016 at 11:44 AM..
Reason: code tags
My crystal ball is in the repair shop today. Without seeing the code that is dropping core, seeing how the environment being used by the code that is dropping core is established, and the context in which that code is being run; it is going to be difficult for most of us to guess at what is wrong.
# Script to copy latest efin Reports V403 to a new file for today
and then transfer them to Datastore
#
# Mar 2013 BM
#
DSDIR=/app/MSS/PROD/print_outs/ds export DSDIR
TODAY=`date '+%b%d'` export TODAY
MONTH=`date '+%b%y'` export MONTH
HOST='192.168.1.22'
USER='xxxxxx'
PASSWD='xxxxx'
#
clear
#
echo "Moving previous EFIN file to /u01/efin/v40/live/report/archive/bon.\n"
#
cd /export/home/efin/efintran/temp
#
mv * /u01/efin/v42/live/report/archive/bon
#
echo ""
#
echo "Done.\n"
#
# Change to home directory /jrie
#
echo "Moving to efin user reports folder \n"
#
cd /u01/efin/v42/live/report/jrie
#
echo "Transfering e-fin reports to temporary folder \n"
#
cp cli_r02*.prt /export/home/efin/efintran/temp
#
cp cli_r04*.prt /export/home/efin/efintran/temp
#
cp act_r07r*.prt /export/home/efin/efintran/temp
#
cp act_r09r*.prt /export/home/efin/efintran/temp
#
cp act_r90r*.prt /export/home/efin/efintran/temp
#
cp act_r92r*.prt /export/home/efin/efintran/temp
#
cp act_r93r*.prt /export/home/efin/efintran/temp
#
cp act_r38r*.prt /export/home/efin/efintran/temp
#
echo ""
#
echo "Done.\n"
#
cd /u01/efin/v42/live/bin
#
echo ""
#
echo "Removing headers from report files \n"
#
#!/bin/bash
FILES=/export/home/efin/efintran/temp/*
for f in $FILES
do
/u01/efin/v42/live/bin/cfxfrwb $f
echo "Processing $f file..."
# take action on each file. $f store current file name
done
#
echo "Sticking files together \n"
#
cd /export/home/efin/efintran/temp
#
cat cli_r02*.prt > /export/home/efin/efintran/temp/efinr02$TODAY
#
cat cli_r04*.prt > /export/home/efin/efintran/temp/efinr04$TODAY
#
cat act_r07r*.prt > /export/home/efin/efintran/temp/efin$TODAY
#
cat act_r09r*.prt >> /export/home/efin/efintran/temp/efin$TODAY
#
cat act_r90r*.prt >> /export/home/efin/efintran/temp/efin$TODAY
#
cat act_r92r*.prt >> /export/home/efin/efintran/temp/efin$TODAY
#
cat act_r93r*.prt >> /export/home/efin/efintran/temp/efin$TODAY
#
cat act_r38r*.prt >> /export/home/efin/efintran/temp/efin$TODAY
#
echo "Removing empty spaces from efin files \n"
#
sed '/^$/d' efinr02$TODAY > efinr02a$TODAY
#
sed '/^$/d' efinr04$TODAY > efinr04a$TODAY
#
sed '/^$/d' efin$TODAY > efinall$TODAY
#
rm efinr02$TODAY
#
rm efinr04$TODAY
#
rm efin$TODAY
#
echo "*** Connecting to the DSX Server for file transfer *** \n"
#
ftp -n -v $HOST <<EOF
type ascii
user $USER $PASSWD
cd ..
cd efin
put efinr02*
put efinr04*
put efinall*
bye
/
EOF
#
Basically this script:
goes into a user's directory,
copies some report files to a temp directory,
attempts to process the files using cfxfrwb (which strips out the headers of the report files),
Cats the files together,
FTPs the files over to a document archive device.
As mentioned before , setting the enviroment above will allow cfxfrwb to process an individual report file but it errors when processing a directory full.
Someone mentioned that the #!/bin/bash statement will override the environment settings (I tried removing it and it still errored). I have also tried just setting the enviroment and manually executing
Code:
FILES=/export/home/efin/efintran/temp/*
for f in $FILES
do
/u01/efin/v42/live/bin/cfxfrwb $f
done
still errors.
Many thanks for having a look.
Maybe I'm missing something obvious here..I don't know.
---------- Post updated at 01:33 PM ---------- Previous update was at 01:30 PM ----------
Sorry...I used the wrong code tag...Could a moderator please amend my tags if possible?
Thanks
Last edited by Don Cragun; 02-18-2016 at 06:47 PM..
Reason: Change most ICODE tags to CODE tags.
First: You don't need a moderator to edit your post for you. (Although I was happy to edit your script for you this time.) There is an "Edit" button in the lower right corner of every post you submit. You can click on that button and edit your own post.
Second: You have now shown us the script that is calling the script that is dropping core. But you have not shown us the script that is calling something on line 209 that is dropping core. We need to see the script named efdsx_trans )which is the script that the diagnostic messages you showed us in post #1 in this thread came from.
It's a bit misleading as I edited the script before posting (it goes into about 5 different user's accounts to retrieve files, I trimmed out 4 of the users).
I've edited the script now so that it is the same as the one in the previous post.
It's a bit misleading as I edited the script before posting (it goes into about 5 different user's accounts to retrieve files, I trimmed out 4 of the users).
I've edited the script now so that it is the same as the one in the previous post.
OK. So, now that the output from the script shown above comes from the script you showed us, we can now get a better idea of what is going on. And specifically, the diagnostics of the form:
Tells us that Process ID 488 dropped core due to a segmentation fault and that it was invoked from line # 63 in the script efdsx_trans which is:
Code:
/u01/efin/v42/live/bin/cfxfrwb $f
and, from the echo statement on line #64 in that script, we know that $f expanded to /export/home/efin/efintran/temp/act_r09r3814.prt this time through the loop.
So, now you need to look at cfxfrwb and find out why it is dropping core as it processes that file. So, if you can't track down the problem with this new information, show us the source for that utility and the stack trace from the core dump it produced when it died.
Hello
I have a program cfxfrwb which is designed to remove headers from reports files.
The cfxfrwb is located in the following directory /u01/efin/v40/live/bin
I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Hi,
I'm using awk command in bash script. I'm able to pass multiple files to awk for processing.The code i can use is as below(sample code)
#!/bin/bash
awk -F "," 'BEGIN {
...
...
...
}' file1 file2 file3
In the above code i'm passing the file names manually and it is fine till my... (7 Replies)
hai i need my single awk script to act on 4 trace files of ns2 and to calculate througput and it should print result from each trace file in a single trace file. i tried with the following code but it doesnt work awk -f awkscript inputfile1 inputfile2 inputfile3 inputfile4>outputfile ... (4 Replies)
I have five directories, dir1 to dir5
for each directory, I have all same number-named folders. There are four types of folders, {1..10}, {20..30}, { 40..50}, {60..70}
Now for each types of folder, I will do the same thing, here is the code
for i in {1..5}
do
cd dir$i
mkdir temp1
for... (5 Replies)
Hi
I am new to Solaris and was just given my id and need to setup my environment, what do i need to do to run certain commands without putting in the complete path.
How do I create my .profile, I do not see under my login?
Any help would be greatly appreciated. (5 Replies)
Helllo UNIX Forum :)
Since I am posting on this board, yes, I am new to UNIX!
I read a copy of "UNIX made easy" from 1990, which felt like a making a "computer-science time jump" backwards ;)
So, basically I have some sort of understanding what the basic concept is.
Problem Description:... (6 Replies)
Hi all,
I am faced with a rather unusual problem regarding interaction between NT and UNIX. I am using an ETL (Extract-Transform-Load) tool on unix that has the capability to read .xls files. So, when I FTP an excel (.xls) file from a windows server to unix and attempt to read it with this... (3 Replies)
Hello Everyone,
I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this.
I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this
177.994 847.9
178.996 ... (1 Reply)
02-18-2016
