Anyone have better way to remove ls from this function?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Anyone have better way to remove ls from this function?
# 1  
Old 02-19-2017
Anyone have better way to remove ls from this function?

all,

i have this script which grabs the number of files to process and displays the percent done as the loop works. i'm never sure how many files will need to be processed.

Code:
let k=1
tot=$( ls -l *.${src_ext} | wc -l)
TIMEFORMAT='%3lR'
for i in `ls *.${src_ext} | sort -k2 -t_ -n`
     do
    //lots of stuff that takes a while here,
   // example 1. variable file size could be anywhere from 10mb to 1gb, *.mp4
   // example 2 fairly consistent filesizes +- few percent with a huge list of image files *.tiff or *.dng
SECONDS=0
     // initialize with ffprobe $i, dump results into array
    //  ffmpeg -i $i  $command[0] \
   //  $command[1] \
TIME2FINISH=$(($SECONDS * ($tot - $i )))
percentDone=$(echo "scale=3; $i/$tot*100.0" | bc -l)
echo -e "percentDone $percentDone")
echo -e "Time to finish $(($TIME2FINISH / 3600))hrs $((($TIME2FINISH / 60) % 60))min $(($TIME2FINISH % 60))sec" 
let k=$((k+1))
done

i'm fairly certain using 'ls' is not a good way to do things, can anyone suggest better way?

anyone have any functions that could estimate amount of time remaining based on the wallclock?

for the timing is this a better way to do it

time_1=echo `time ffprobe -i $i command1 command2`
time_2=echo `time ffmpeg -i $i commandA command B`


thanks

Last edited by f77hack; 02-19-2017 at 07:22 PM.. Reason: added timing
# 2  
Old 02-19-2017
ls -l is overdone here - you don't need the extra fields in a long listing.
Code:
tot=$(echo *.${src_ext} | wc -w)

may save you a few CPU cycles.

Unless you tell people if and how "lots of stuff that takes a while here" depend on file count or sizes or even other parameters, I'm afraid no one will be able to lend you a hand.
This User Gave Thanks to RudiC For This Post:
# 3  
Old 02-19-2017
If you happy with Percent complete being accurate to within 1% you can also save spawning bc like replace this:

Code:
tot=$(printf "%s\n" *.${src_ext} | wc -l)
for i in `ls *.${src_ext} | sort -k2 -t_ -n`
do

    //lots of stuff that takes a while here

    printf "percentDone %d\n" $((100*++k / tot))
done

or, using an array when can also get rid of wc -l:

Code:
F=( *.${src_ext} )
tot=${#F[@]}
for i in "${F[@]}"
do

    //lots of stuff that takes a while here

    printf "percentDone %d\n" $((100*++k / tot))
done


Last edited by Chubler_XL; 02-19-2017 at 03:42 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 4  
Old 02-19-2017
Quote:
Originally Posted by RudiC
ls -l is overdone here - you don't need the extra fields in a long listing.
Code:
tot=$(echo *.${src_ext} | wc -w)

may save you a few CPU cycles.

Unless you tell people if and how "lots of stuff that takes a while here" depend on file count or sizes or even other parameters, I'm afraid no one will be able to lend you a hand.
Thanks for the help.

Ok, if it's an video file *.mp4, the video gets shoved into a long ffmpeg command line to do some heavy duty processing, crop, noise reduction, pixel conversion, etc. I probe the file intially, get some parameters and let it crunch.
# 5  
Old 02-20-2017
If those actions are somehow file size dependent, and you supply the accurate dependency algorithm, people might be able to help you develop a formula to calculate a "time percentage" or "time to finish" value.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Function - Make your function return an exit status

Hi All, Good Day, seeking for your assistance on how to not perform my 2nd, 3rd,4th etc.. function if my 1st function is in else condition. #Body function1() { if then echo "exist" else echo "not exist" } #if not exist in function1 my all other function will not proceed.... (4 Replies)
Discussion started by: meister29
4 Replies

2. UNIX for Beginners Questions & Answers

DB2 Query modification to remove duplicate values using LISTAGG function

I am using DB2 v9 and trying to get country values in comma seperated format using below query SELECT distinct LISTAGG(COUNTRIES, ',') WITHIN GROUP(ORDER BY EMPLOYEE) FROM LOCATION ; Output Achieved MEXICO,UNITED STATES,INDIA,JAPAN,UNITED KINGDOM,MEXICO,UNITED STATES The table... (4 Replies)
Discussion started by: Perlbaby
4 Replies

3. Shell Programming and Scripting

Need help on awk for printing the function name inside each function

Hi, I am having script which contains many functions. Need to print each function name at the starting of the function. Like below, functionname() { echo "functionname" commands.... } I've tried like below, func=`grep "()" scriptname | cut -d "(" -f1` for i in $func do nawk -v... (4 Replies)
Discussion started by: Sumanthsv
4 Replies

4. UNIX for Beginners Questions & Answers

awk function to remove lines that contain contents of another file

Hi, I'd be grateful for your help with the following. I have a file (file.txt) with 10 columns and about half a million lines, which in simplified form looks like this: ID Col1 Col2 Col3.... a 4 2 8 b 5 6 1 c 8 4 1 d... (4 Replies)
Discussion started by: aberg
4 Replies

5. Shell Programming and Scripting

Will files, creaetd in one function of the same script will be recognized in another function?

Dear All. I have a script, which process files one by one. In the script I have two functions. one sftp files to different server the other from existing file create file with different name. My question is: Will sftp function recognize files names , which are created in another... (1 Reply)
Discussion started by: digioleg54
1 Replies

6. Shell Programming and Scripting

Help to Modify File Name in each function before calling another function.

I have a script which does gunzip, zip and untar. Input to the script is file name and file directory (where file is located) I am reading the input parameters as follows: FILENAME=$1 FILEDIR=$2 I have created 3 functions that are as follows: 1) gunzip file 2) unzip file... (2 Replies)
Discussion started by: pinnacle
2 Replies

7. Shell Programming and Scripting

Function to remove the directories from PATH variable

Hello, From the URL https://www.unix.com/shell-programming-scripting/121303-remove-path-path-environment-variable-2.html I got a function to remove the directories from a path. but looks like this isnt quite working.. i am also not able to post the comments in the thread as it is closed. ... (6 Replies)
Discussion started by: satishkumar432
6 Replies

8. Shell Programming and Scripting

SHELL SCRIPT Function Calling Another Function Please Help...

This is my function which is creating three variables based on counter & writing these variable to database by calling another function writeRecord but only one record is getting wrote in DB.... Please advise ASAP...:confused: function InsertFtg { FTGSTR="" echo "Saurabh is GREAT $#" let... (2 Replies)
Discussion started by: omkar.sonawane
2 Replies

9. Shell Programming and Scripting

Return a value from called function to the calling function

I have two scripts. script1.sh looks -------------------------------- #!/bin/bash display() { echo "Welcome to Unix" } display ----------------------------- Script2.sh #!/bin/bash sh script1.sh //simply calling script1.sh ------------------------------ (1 Reply)
Discussion started by: mvictorvijayan
1 Replies

10. Shell Programming and Scripting

Passing global variable to a function which is called by another function

Hi , I have three funcions f1, f2 and f3 . f1 calls f2 and f2 calls f3 . I have a global variable "period" which i want to pass to f3 . Can i pass the variable directly in the definition of f3 ? Pls help . sars (4 Replies)
Discussion started by: sars
4 Replies
Login or Register to Ask a Question