I'm assuming the XML data for that looks like what you posted before?
Working on something.
Yes, but extracting the title from the MovieInfo.nfo file is pretty fast it's all those if / test statement and writing to the file that seem to take a long time. For example if my drive contain approximately 200 movies (not really that much considering some people have thousand) divided in 9 movies genres the loop would be run like 10 times using the same list but different MOVIESPATH. For example
first run = All the movies 200 index to create XML file
Second run = genre science-fiction maybe 30 movies indexes to create XML file
Third run = genre romance maybe 20 movies indexes to create XML file
etc.....
Obviously the total of movies for the Second run to the last run (10) will include a total of 200 movies.
I hope I am making sense and thanks again for your help/time.
Here's a version tested in busybox which uses almost pure shell builtins:
---------- Post updated at 02:02 PM ---------- Previous update was at 01:57 PM ----------
Quote:
Originally Posted by snappy46
Yes, but extracting the title from the MovieInfo.nfo file is pretty fast it's all those if / test statement and writing to the file that seem to take a long time.
What sort of disk are you writing to?
Shell builtins are hundreds of times faster than calling an external utility to operate on tiny amounts of data.
I think I've reduced the number of if/else's by using a case, too.
Also, you were reopening $RSS dozens of times, which probably didn't help.
Shell builtins are hundreds of times faster than calling an external utility to operate on tiny amounts of data.
I think I've reduced the number of if/else's by using a case, too.
Also, you were reopening $RSS dozens of times, which probably didn't help.
That is awesome I did not expect anyone to do all the dirty work for me. Hopefully this script will work fine on the old default busybox available on the media player.
I forgot to mentioned on my previous post that the whole indexing process All movies + all genres (200 movies) takes about 3to4 minutes. Hopefully your way of doing things will cut that down some.
The jukebox.xml file created by the loop is normally stored/written to the media player (internal drive) or externally connected USB drive; it all depends where the movies are located. The jukebox.xml is store in the genre root directory or the movie directory for the All movies.
I can wait to try this out. I will post the results once I did.
Again thank you.
---------- Post updated at 11:27 PM ---------- Previous update was at 04:55 PM ----------
Hi Corona,
Here's your script with some minor changes to make it work for me.
I could not get the function to extract the title from the MovieInfo.nfo file to work so I just inserted the one I already had in my script just to test the difference between the two script. The results were very surprising to me to say the least.
The original script as provided in my first post took 1 minutes and 49 second to process my 200 movies. The new source code you provided took 2 minutes and 46 seconds almost a whole minute longer ???? I could not believe the results so I tried it again with the same results. It would appear that using sed/cut/grep etc to get the work done is faster than using the built-in substitution command ????? I was quite shock, something in there seem to take a lot of time to accomplish.
I still think that I can use some of your code process to cut down further the process time ... well maybe. I would think that creating the file only once would be faster than appending the file for every movie element. I will try to introduce some of those step you have one at a time and see what makes thing go faster and what make things go slower.
I learned a lot from your inputs/code so again thank you.
I could not get the function to extract the title from the MovieInfo.nfo file to work so I just inserted the one I already had in my script just to test the difference between the two script.
Throwing away nearly all the speed gain in the process
If you could post the literal contents of one of those files, that'd be good. I built it to work with your test data. If it's actually any different, then I really need to see what it is.
Quote:
It would appear that using sed/cut/grep etc to get the work done is faster than using the built-in substitution command ????? I was quite shock, something in there seem to take a lot of time to accomplish.
The built-in substitution operator is hundreds of times faster than calling an external utility. At the very least the builtins instead of basename and basedir ought to be better. The rest of my code may have depended on certain assumptions about your data.
Are these folders full of irrelevant files? If so, the for FILE in "$MOVIEPATH"/* loop will waste a lot of time. Come to think of it, since we're only interested in .jpg, you can make it for FILE in "$MOVIEPATH"/*.jpg
---------- Post updated at 09:29 AM ---------- Previous update was at 09:20 AM ----------
Here's a version which works without trawling every file in the folder. You can replace the long if-else chain with two for's. It also helps make the lists longer without making your code longer (though testing for too many things will slow you down in any case).
---------- Post updated at 09:35 AM ---------- Previous update was at 09:29 AM ----------
And you can strip out this completely:
I just put it there in case your input data was radically different from what I assumed it was.
Last edited by Corona688; 10-05-2011 at 01:35 PM..
Throwing away nearly all the speed gain in the process
I know this is temporary I plan on figuring out why this does not work; I am getting "bad substitution" on the busybox version that I am using on those lines:
SHORT="${LINE/<title>}"
LINE="${LINE//<title>}" # Strip out <title>
etc....
Can not remember which ones exactely but more than one of those substitution caused a problem. I had the same issue with that line:
which I fixed by changing to this:
I did not have much time to play around with that last nights and was curious about the change in processing time so I jump right in with the old code. Apparently my wife's needs are more important than working on this script.
Quote:
Originally Posted by Corona688
If you could post the literal contents of one of those files, that'd be good. I built it to work with your test data. If it's actually any different, then I really need to see what it is. The built-in substitution operator is hundreds of times faster than calling an external utility. At the very least the builtins instead of basename and basedir ought to be better. The rest of my code may have depended on certain assumptions about your data.
I will provide sample of the input files (SORTEDTMP, MovieInfo.nfo) and output file (jukebox.xml) hopefully tonight time permitting; I am not home right now and do not have access to those file. Agree built-in substitution should be faster.
Quote:
Originally Posted by Corona688
Are these folders full of irrelevant files? If so, the for FILE in "$MOVIEPATH"/* loop will waste a lot of time. Come to think of it, since we're only interested in .jpg, you can make it for FILE in "$MOVIEPATH"/*.jpg
Yeah there is quite a few files on there some relevant some not so much. I think you just nailed it why it takes so much more time. I will use those new "for loop" you provided and I have a feeling that I will finally see some speed increase compare to the old script.
Thanks again from one fellow Canadian to another.
Cheers!!!
---------- Post updated 10-06-11 at 12:17 AM ---------- Previous update was 10-05-11 at 01:51 PM ----------
Ok when using the for loop and deleting the unnecessary if statement as indicated in your previous post I now can process my 200 movies in 1 minutes and 10 seconds .... wow!! That is an improvement of about 40 sec compare to the original script. That makes more sense; I guess the case statement + the number of files were definitely slowing things down.
Now trying to get the procedure to pull the title from the MovieInfo.nfo file to work but I am still stuck on one thing that does not want to work. In my version of busybox we must use "#" for deletion from the left to match; and "%" for deletion from the right to match. Once I figure that out you would think that thing would have when pretty smoothly but of course it did not. I can easily delete the <title> but I am unsuccessful in deleting the </title> ... I think that the "/" forward slash is creating a problem ???? Funny thing is that it works fine in the interactive mode but refuses to work in the script ????
This works fine in the shell:
export foo="<title>hello</title>
echo "${foo%</title>}"
or
echo "${foo%%</title>}"
or
echo "${foo%<?title>}"
or
echo "${foo%<*title>}"
all those returns "<title>hello" but none of those combination will remove the </title> when used in the script ???? This is driving me crazy any ideas??????
But this works fine in the script to remove the <title>
By the way I also removed that line since it is not really needed and just used SHORT for the Strip out </title>
Here's the procedure that works fine except for the strip put </title>.
# strips from the beginning of the string, % from the end. ## and %% are the same when you're removing a literal string, so might as well use % and #. See string operations. Not all shells have all of these. Some don't have any of them. Developers on embedded busybox are more fortunate than some people using 'real' shells, it has at least a few
---------- Post updated at 03:17 PM ---------- Previous update was at 03:08 PM ----------
YOu could also try putting quotes around "${VAR%"</title>"}"
---------- Post updated at 03:20 PM ---------- Previous update was at 03:17 PM ----------
globbing should also work inside it, so:
---------- Post updated at 03:21 PM ---------- Previous update was at 03:20 PM ----------
Also: If it works in the prompt and not in your script, then your data may not be what you really think it is. You feed it a nice "<title>stuf</title>" and it works but the data actually has stuff after <title> perhaps. PLEASE post a sample of your input data!!
Last edited by Corona688; 10-06-2011 at 07:16 PM..
I have prepared a shell script to find the duplicates based on the part of filename and retain latest.
#!/bin/bash
if ; then
mkdir -p dup
fi
NOW=$(date +"%F-%H:%M:%S")
LOGFILE="purge_duplicate_log-$NOW.log"
LOGTIME=`date "+%Y-%m-%d %H:%M:%S"`
echo... (6 Replies)
Hi,
I need to remove first column from a csv file and i can do this by using below command.
cut -f1 -d, --complement Mytest.csv
I need to implement this in shell scripting, Whenever i am using the above command alone in command line it is working fine.
I have 5 files in my directory and... (3 Replies)
Hi All,
I have a script that I wrote on a bash shell, I use it to sort files from a directory into various other directories. I have an variable set, which is an array of strings, I then check each file against the array and if it is in there the script sorts it into the correct folder.
But... (5 Replies)
Hi,
I have two files in the format listed below. I need to find out all values from field 12 to field 20 present in file 2 and list them in file3(format as file2)
File1 :
FEIN,CHRISTA... (2 Replies)
Hello -- I am trying to learn to do a little sed and awk scripting to search for text and numbers in text files (text processing/manipulation). My professor gave me a piece of uncommented code and I am very unfamiliar w/ the language. Can someone help me with comments so I can understand what is... (2 Replies)
Hi,
Does anybody know how to enable the shell sh while creating Ramdisk fs using BusyBox? while creating a configuration using the GUI, I see options only for the ash shell. Is there some option in the config file that gets created with which I can enable the sh shell also apart from the ash... (0 Replies)
hi fndz.
Can you please help me with the code if I call a stored procedure from my shell script and stored procedure returns a cursor,
cursor output should be saved to a file (3 Replies)
Hi All,
I am trying to understand and change some code written by some programmer a while ago. There are following three lines of code that I am unable to grasp. Could anybody please help me understand it?
1) cd - > /dev/null
2) fname=`basename "$1"` where $1 = /dirA/dirB/a.txt
... (3 Replies)
use "getopts" to get params from command. Need replace black with a specified string like "%20
DEFAULT_DELIM=%20
...
while getopts dek:f:t:vh OPTION
do
case $OPTION in
t)
DELIM=`tvar=/'"$OPTARG"'/ svar="$DEFAULT_DELIM" awk 'BEGIN{T=ENVIRON;S=ENVIRON; while(index(T,S)!=0){S=S"0"};print... (0 Replies)