Trying to use a convoluted for loop with VLC and Parallel or OpenMPI with no success. Help?
I have about 12,000,000 mod files I'm trying to turn into a test of "unlimited cloud storage" by running them all through VLC and blowing them into mp3 files. I can get this to work serially but when trying to use openMPI or Parallel, something in the syntax is tripping it up some. Here is an example of the command I'm using. Open to any help or suggestions.
Code:
for file in `find ./modarchive/modarchive_2007_official_snapshot_addendum1/ -name "*.mod" -type f`; do parallel -j4 --citation NEWFILE=`echo $file | sed 's/modarchive/mp3archive/'`;echo working on $file; vlc -I dummy -f $file --sout "#transcode{acodec=mp3,ab=128}:standard{mux=mp3,dst=$NEWFILE.mp3,acce~ss=file}" vlc://quit 2> /dev/null ; done
Moderator's Comments:
Please use CODE tags when displaying sample input, output, and code segments (as required by forum rules).
Last edited by Don Cragun; 06-03-2017 at 08:07 PM..
Reason: Add CODE and ICODE tags.
You define, but do not use, the variable NEWFILE in the command substitution that runs in a subshell environment and disappears before your for loop starts running. So, in that command $NEWFILE will expand to a constant (probably an empty string) value instead of a value based on the name of the file you're processing.
Does the vlc utility really want a sub-option of the form acce~ss=file, or is the tilde (~) in that string a typo?
Instead of throwing away the diagnostic messages produced by the code you're running (2>/dev/null), it might help to actually read those diagnostics and see if they provide any insight into what might be going wrong.
I would also imagine that invoking sed 12 million times makes your script take a long time before it ever gets to the point that it invokes vlc the first time. Why not use a couple of variable expansions inside the loop to avoid invoking sed at all? Note that this operation needs to be in a position where NEWFILE will be paired with the file value it has modified; not in someplace where the vlc command you invoke will be using unpaired $file and $NEWFILE values.
Hi,
I am using solaris 5.10 environment and need help on doing parallel increment of nested for loop.
Samples
#inside the code the values assigned to a variable by another awk command will be like
a=/xyz/pg/as
/xyz/pg/as2
/xyz/pg/as3
b=/xyz/sd/fd1
/xyz/sd/fd2
/xyz/sd/fd3
for q in... (1 Reply)
Hi
I am running a loop which actually runs same script for different argument value passed to it.
while read repID
do
echo "Starting for $repID";
date;
perl process_report.pl $repID
done<${FILE_TO_READ}
However this runs in sequence.
I want the loop to not to wait for perl to... (3 Replies)
I have the below code which runs on multiple databases , but this runs one-after-one. I will need this to run in parallel so that i could save a lot of time. Please help!!! Thanks in advance
for Db in `cat /var/opt/oracle/oratab |egrep -v "ASM" |grep -v \# |cut -d\: -f1`
do
{
export... (5 Replies)
Hello All,
I want to test how much parallel ssh connections can be done on a server.
I am thinking of reading username and hostname from a file and then using a loop (may be for) to do ssh on different host.
Could anyone suggest me how can i write the script for the above.
Thank you in... (0 Replies)
Hello,
My script shell is:
for i in $(seq $nb_lignes)
do
//command java
done
Please, how can i execute all iteration in parallel ?
Thank you so much. (9 Replies)
Hi everybody,
my problem is the following:
I'm trying to run an openmpi program on a cluster (atlasz.elte.hu, it's in hungarian, but you can try google translate),
but I always got this error:
Fatal error in MPI_Init: Other MPI error, error stack:
MPIR_Init_thread(394)...........:... (0 Replies)
Hi,
I have been thinking of how to script this but i have no clue at all..
Could someone please help me out or give me some idea on this?
I would like to group those lines with the same first variable in each line, joining the 2nd variables with commas.
Let's say i have the following input.
... (8 Replies)
OK, I hope you can help me here: I'm trying to crack a hash using John the Ripper. (BindShell.Net: John The Ripper MPI Patch)
I'm runnig it on a cluster system using OpenMPI. But it is not working properly. If i try to run it, i get the following error:
Any idea what can be wrong? Happen on... (3 Replies)
Hi,
I need to execute parallel with while loop.
Input File(source_file.csv) contains filenames the below
source_file.csv file contains
Customer1.txt
Product1.txt
Sales.txt
Emp.txt
Dept.txt
Based on the number of rows that file I want to run the script ‘n' times.
while... (2 Replies)
Hi,
please can someone point me in the right direction with a shell scripting problem.
I want to execute a command in a for loop and the command should be started not one-by-one meaning the for loop is waiting for the exit code , it should be started in parallel.
I have a plain text file... (3 Replies)
06-03-2017
