Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Move File Containing More Than two "-" at 3rd Line To New Directory

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Move File Containing More Than two "-" at 3rd Line To New Directory
# 36  
Old 05-16-2015
Got you. That was the only method that came to my mind; I thought your had another method.

---------- Post updated at 18:37 ---------- Previous update was at 18:29 ----------

You could do it also without sed, just shell parameter expansion...
This User Gave Thanks to RudiC For This Post:
Nexeu
# 37  
Old 05-16-2015
Quote:
Originally Posted by RudiC
You could do it also without sed, just shell parameter expansion...
Yes, but the original problem was: read a lot (~700k) files and extract only a certain part of line 3. Shell expansion can extract that part but it is not easy to interrupt the reading process after only 3 lines. Therefore i figured there must be a tradeoff between the preserved fork() of shell expansion and the lesser I/O the sed solution produces.

Which optimisation weighs heavier is probably different from system to system and depends on so many factors i didn't even try to take measurements. I could have, but the disks i have on all my systems all come from several EMC VMaxes (we even boot from LUNs via VIOS) and i doubt that thread-O/P has an I/O-subsystem capable of shoveling up to 700MB/s to/from the disks. This will, IMHO, have such a big impact on the tme it takes to read the 700k files that i could as well roll a dice.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
Nexeu
# 38  
Old 05-16-2015
Remember that we're processing a single directory containing 690,000 files. So, we have some constraints...

In theory for i in *.txt should work, but even though no exec is involved, we are still talking about a list of arguments that is probably well over 7.5Mb (and the shell will waste time sorting this list when the order in which the files are processed doesn't matter for this project).

I can't use:
Code:
find . -name '*.txt' ! -name '* *' | "figure out where file should go and move it"

because the behavior of find is undefined if the directory changes while find is reading it.

Invoking sed (or any other utility 690,000 times) to determine the directory to which a file should be moved will take forever. Similarly, invoking mv 690,000 times will take forever. We need to efficiently determine to which directory a list of files should move and move those files in large groups (not individually).

Once I have the list of files to go to a directory, I can use:
Code:
xargs -J 'Arg' mv 'Arg' target_directory

on OS X to minimize the number of needed invocations of mv.

Even if we do this in the shell using entirely shell built-ins to determine the directory to which a file should be moved, echo filename>> listN or printf filename>> listN will still be opening and closing the list files 690,000 times.

There will be more target directories than there are available open files in an awk script on OS X, but we don't know the maximum number of hyphens (nor the number of different values for the number of hyphens) in the 1st word of the 3rd line of these 690,000 files. (We do know that there can be at least 18 hyphens.) I think I can use a pipeline with:
Code:
ls -f | awk "select target directory & create up to 16 lists" | awk "create up to 17 lists"

and easily just read the first 3 lines of the files being processed and just open and close the list files once. The first stage of the above pipeline could also be replaced by the find command mentioned before and that would simplify the 1st awk script in the pipeline. (This could fairly easily be extended to let awk spawn more copies of itself to handle an unlimited number of open list files, but I don't think it will be needed for this project.)

I have a good start on this pipeline, but it will take me a while to finish the code and test it.
This User Gave Thanks to Don Cragun For This Post:
Nexeu
# 39  
Old 05-17-2015
What if we execute commands one by one, will it be easier? There was a time I was executing many different command lines by writing them all in a text editor and saved it as Unix Executable File . I don't know what you all call this process but I found it out myself so I don't what this process is called. Smilie

So there was a time I did this to organise file by using mv filename containing "x" into folder x with respect to the amount of x.

The script was like this [This script wasn't about this problem]

Code:
mkdir x
mkdir xx
mkdir xxx
mkdir xxxx
mkdir xxxxx
mkdir xxxxxx
mkdir xxxxxxx
mkdir xxxxxxxx
mkdir xxxxxxxxx
mkdir xxxxxxxxxx
mkdir xxxxxxxxxxx
mkdir xxxxxxxxxxxx
mkdir xxxxxxxxxxxxx
mkdir xxxxxxxxxxxxxx
mkdir xxxxxxxxxxxxxxx
mkdir xxxxxxxxxxxxxxxx
mkdir xxxxxxxxxxxxxxxxx
mkdir xxxxxxxxxxxxxxxxxx

find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxxx'
find . -maxdepth 1 -name '*x*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxxx'
find . -maxdepth 1 -name '*x*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxxx'
find . -maxdepth 1 -name '*x*x*x*' -type f | xargs -I '{}' mv '{}' './xxx'
find . -maxdepth 1 -name '*x*x*' -type f | xargs -I '{}' mv '{}' './xx'
find . -maxdepth 1 -name '*x*' -type f | xargs -I '{}' mv '{}' './x'

find . -empty -type d -delete

And I dragged that Unix Executable File into Terminal and hit Enter. Terminal executed all of them one by one.

My command is kind of amateur but I kind of managed to execute them in orderly manner. Smilie

At that time I had a million over files and these command could do the work.

I wasn't also sure how many x there could be in filename so I wrote the command as many as I thought it would be.

But of course, this is just executing filename so it was easy and I didn't expect executing file-content would so hard Smilie

Sorry guys.
# 40  
Old 05-17-2015
No, you can't look at the names of files and magically guess how many hyphens are in the 3rd lines of those files. And, as noted before using find | ... | mv ... may miss files depending on filesystem type when you have a directory with this many files in it...

If I have correctly understood what you want to do, the following script will move *.txt files with names that do not contain any space characters from /Users/Nexeu/Documents/Dict to subdirectories under /Users/Nexeu/Documents/Syllable. The target directory and subdirectories will be created if they do not already exist. This script will give errors if you try to move files with more than 33 different values for the number of hyphens contained on the 3rd line. If you save the output containing those errors, extract lines from that output that start and end with a ' character, and feed those lines into a modified script that runs in $SRCDIR and just runs the 2nd awk script, it will create list files for another 17 target subdirectories and the last part of the script will use those list files to move those files into the proper target directories. Or, you can just run the entire script again to process up to 33 more different hyphen counts (but that will take longer if there are still lots of files to process).

Code:
#!/bin/ksh
# USAGE: mvhyphen
# DESCRIPTION:
#	This script depends on having two variables defined:
#	SRCDIR:	Absolute pathname of directory containing files to be
#		processed.  Results are unspecified in there are any
#		subdirectories in this directory.
#	DESTDIR:Destination base directory.  This can be an absolute
#		pathname or a pathname relative to $SRCDIR.  (whichever
#		of these is shorter is preferred.)

#	This script moves to $SRCDIR and processes files with names
#	ending with ".txt".  Files with names containing a space are
#	ignored.  The 3rd line of each file is read.  If that line does
#	not match the pattern '^[[].*[]]', the file is also ignored.
#	Otherwise, the number of hyphens between the '[' and the first
#	comma or ']' after that are counted.  For each unique count
#	value, a list file is created in $DESTDIR named "listNH" where
#	"N" is the count value.  After the lists have been created,
#	files in each list will be moved from $SRCDIR to $DESTDIR/"N"H.
#	$DESTDIR and $DESTDIR/*H will be created if they are not already
#	present.

#	This script is OS X specific.  It is tuned to work within the
#	number of files awk can have open at once (stdin, stdout, and 17
#	more files) and uses the non-standard xargs -J option.  This
#	script whould be able to handle up to 33 different values for
#	the number of hyphens found in the 1st word in the 3rd line in
#	the files being processed.

# Initialize variables...
DESTDIR=../Syllable
SRCDIR=/Users/Nexeu/Documents/Dict

# Move to source directory and process the files found there...
cd "$SRCDIR" || exit 1
mkdir -p "$DESTDIR" || exit 2
find . -name '*.txt' ! -name '* *' |
awk -v sq="'" -v dest="$DESTDIR" '
{	# Open and read the 1st three lines of the file named on the input line.
	f = substr($0, 3)	# Discard the leading "./" from find.
	getline x < f
	getline x < f
	rc = getline x < f
	close(f)		# Close the file.
	if(rc != 1) {
		printf("Cannot read 3 lines from file: %s\n", f)
		next
	}
	if(x !~ /^[[].*[]]/) {
		printf("File line 3 bad format: %s\n", f)
		next
	}
	sub(/[],].*/, "", x)	# Discard all but 1st word...
	nh = gsub(/-/, "", x)	# count hyphens remaining on the line.
	if(!(nh in flist)) {
		# Add to the list of known counts.
		flist[nh] = sprintf("%s/list%dH", dest, nh)
		fd[nh] = ++nfd
	}
	if(fd[nh] <= 16) {
		# Write this filename directly to the appropriate list file.
		print sq f sq > flist[nh]
	} else {# Write the list file filename and this filename to stdout
		# to be processed by the 2nd awk in the pipeline...
		print sq flist[nh] sq f sq
	}
}' | \
# The following awk script interprets lines of the form:
#	'listfile_filename'file_to_be_move_filename'
# (without the double quotes) as a request to add the 2nd filename to
# the list of files in the 1st filename.  Other lines are copied
# directly to stdout assuming that they are diagnostics from the
# previous awk script.  If more than 17 different listfile pathnames are
# found in the input, lines for those listfiles will also be copied to
# stdout (so another copy of this script can be used to create upto 17
# more listfiles without running the find and the 1st awk again.
awk -F "'" -v sq="'" -v pat="^'[^']*'[^']*'$" '
$0 ~ pat { # Process listfile data lines...
	if(!($2 in flist) && nfd++ < 17) {
		# Add to known file list file array.
		flist[$2]
	}
	if($2 in flist) {
		print sq $3 sq > $2
		next
	}
	print "Too many list files to process..."
}
1'

# Now that all of the list files have been created, create the destination
# directories and move the files included in the list files into them.
for listpath in "$DESTDIR"/list*H
do	dirpath="${listpath%%list*}${listpath##*list}"
	printf 'Processing list file: "%s"\n' "$listpath"
	mkdir -p "$dirpath"
	xargs -J '#' mv '#' "$dirpath" < "$listpath" && rm "$listpath"
#	xargs -J '#' -t mv '#' "$dirpath" < "$listpath" && rm "$listpath"
done

When tested on a MacBook Pro running OS X Yosemite 10.10.3, it did what I expected with a couple of hundred files with 35 different hyphen counts. Obviously, it has not been tested in an environment with 690,000 files.

If you want it to provide a verbose list of the mv commands it uses while moving files from .../Dict to subdirectories of .../Syllable, uncomment the next to the last line in the script and comment out the line before that.

Good luck!
This User Gave Thanks to Don Cragun For This Post:
Nexeu
# 41  
Old 05-17-2015
OMG that's a lot of work. THANK YOU SO MUCH! I will give it a try now Smilie

---------- Post updated at 03:01 AM ---------- Previous update was at 02:36 AM ----------

I have this verbose list below :

Code:
File line 3 bad format: κ.txt
File line 3 bad format: μci.txt
File line 3 bad format: μg.txt
File line 3 bad format: μm.txt
Processing list file: "../Syllable/list0H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list10H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list11H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list12H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list13H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list14H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list15H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list18H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list1H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list2H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list3H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list4H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list5H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list6H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list7H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list8H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Processing list file: "../Syllable/list9H"
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
xargs: invalid option -- J
Usage: xargs [-0prtx] [--interactive] [--null] [-d|--delimiter=delim]
       [-E eof-str] [-e[eof-str]]  [--eof[=eof-str]]
       [-L max-lines] [-l[max-lines]] [--max-lines[=max-lines]]
       [-I replace-str] [-i[replace-str]] [--replace[=replace-str]]
       [-n max-args] [--max-args=max-args]
       [-s max-chars] [--max-chars=max-chars]
       [-P max-procs]  [--max-procs=max-procs] [--show-limits]
       [--verbose] [--exit] [--no-run-if-empty] [--arg-file=file]
       [--version] [--help] [command [initial-arguments]]

Report bugs to <bug-findutils@gnu.org>.
Untitleds-MacBook-Pro:Dict Nexeu$

It executed way faster than I ever expected. I began to have these files in Syllable directory :

Code:
/Users/Nexeu/Documents/Syllable/0H
/Users/Nexeu/Documents/Syllable/1H
/Users/Nexeu/Documents/Syllable/2H
/Users/Nexeu/Documents/Syllable/3H
/Users/Nexeu/Documents/Syllable/4H
/Users/Nexeu/Documents/Syllable/5H
/Users/Nexeu/Documents/Syllable/6H
/Users/Nexeu/Documents/Syllable/7H
/Users/Nexeu/Documents/Syllable/8H
/Users/Nexeu/Documents/Syllable/9H
/Users/Nexeu/Documents/Syllable/10H
/Users/Nexeu/Documents/Syllable/11H
/Users/Nexeu/Documents/Syllable/12H
/Users/Nexeu/Documents/Syllable/13H
/Users/Nexeu/Documents/Syllable/14H
/Users/Nexeu/Documents/Syllable/15H
/Users/Nexeu/Documents/Syllable/18H
/Users/Nexeu/Documents/Syllable/list0H
/Users/Nexeu/Documents/Syllable/list1H
/Users/Nexeu/Documents/Syllable/list2H
/Users/Nexeu/Documents/Syllable/list3H
/Users/Nexeu/Documents/Syllable/list4H
/Users/Nexeu/Documents/Syllable/list5H
/Users/Nexeu/Documents/Syllable/list6H
/Users/Nexeu/Documents/Syllable/list7H
/Users/Nexeu/Documents/Syllable/list8H
/Users/Nexeu/Documents/Syllable/list9H
/Users/Nexeu/Documents/Syllable/list10H
/Users/Nexeu/Documents/Syllable/list11H
/Users/Nexeu/Documents/Syllable/list12H
/Users/Nexeu/Documents/Syllable/list13H
/Users/Nexeu/Documents/Syllable/list14H
/Users/Nexeu/Documents/Syllable/list15H
/Users/Nexeu/Documents/Syllable/list18H

Files start with "list" are the text file and it contain quoted filenames
Like these for example :
At list0H
Code:
...
...
...
'rome.txt'
'romes.txt'
'romp.txt'
...
...


I guess the command successfully organised the files by writing out the list of the file names.

All I have to do is to move it into directories it created. It only require simple mv command to do it. I think I can do it Smilie

Thanks a lot guys! Especially Don Cragun, I owe you lots Smilie
# 42  
Old 05-17-2015
In post #5 in this thread you showed us that you were using the prompt:
Code:
Untitleds-MacBook-Pro:~ Nexeu$

I made the obviously bad assumption that that meant you were running OS X on a MacBook Pro (which has a BSD based xargs; not the GNU xargs you're using).

With the list files created by the current script, the following should finish the job for you:
Code:
#!/bin/ksh
DESTDIR=../Syllable
SRCDIR=/Users/Nexeu/Documents/Dict

cd "$SRCDIR" || exit 1

for listpath in "$DESTDIR"/list*H
do	dirpath="${listpath%%list*}${listpath##*list}"
	printf 'Processing list file: "%s"\n' "$listpath"
#	mkdir -p "$dirpath"
	xargs mv -t "$dirpath" < "$listpath" && rm "$listpath"
done

This is untested code (I don't have a mv utility that has a -t option), but it should come close to doing what you need if you're using GNU xargs and rm utilities.

The first four lines of the output you showed us say that the files κ.txt, μci.txt, μg.txt, and μm.txt do not have the expected format:
Code:
[text]

on line 3.
This User Gave Thanks to Don Cragun For This Post:
Nexeu
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

Apache 2.4 directory cannot display "Last modified" "Size" "Description"

Hi 2 all, i have had AIX 7.2 :/# /usr/IBMAHS/bin/apachectl -v Server version: Apache/2.4.12 (Unix) Server built: May 25 2015 04:58:27 :/#:/# /usr/IBMAHS/bin/apachectl -M Loaded Modules: core_module (static) so_module (static) http_module (static) mpm_worker_module (static) ... (3 Replies)
Discussion started by: penchev
3 Replies

2. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

3. Shell Programming and Scripting

Failure: if grep "$Var" "$line" inside while read line loop

Hi everybody, I am new at Unix/Bourne shell scripting and with my youngest experiences, I will not become very old with it :o My code: #!/bin/sh set -e set -u export IFS= optl="Optl" LOCSTORCLI="/opt/lsi/storcli/storcli" ($LOCSTORCLI /c0 /vall show | grep RAID | cut -d " "... (5 Replies)
Discussion started by: Subsonic66
5 Replies

4. Shell Programming and Scripting

Move a line containg "char" above line containing "xchar"

Okay, so I have a rather large text file and will have to process many more and this will save me hours of work. I'm not very good at scripting, so bear with me please. Working on Linux RHEL I've been able to filter and edit and clean up using sed, but I have a problem with moving lines. ... (9 Replies)
Discussion started by: rex007can
9 Replies

5. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

6. Shell Programming and Scripting

Find lines with "A" then change "E" to "X" same line

I have a bunch of random character lines like ABCEDFG. I want to find all lines with "A" and then change any "E" to "X" in the same line. ALL lines with "A" will have an "X" somewhere in it. I have tried sed awk and vi editor. I get close, not quite there. I know someone has already solved this... (10 Replies)
Discussion started by: nightwatchrenba
10 Replies

7. UNIX for Dummies Questions & Answers

look for file size greater than "0" of specific pattern and move those to another directory

Hi , i have some files of specific pattern ...i need to look for files which are having size greater than zero and move those files to another directory.. Ex... abc_0702, abc_0709, abc_782 abc_1234 ...etc need to find out which is having the size >0 and move those to target directory..... (7 Replies)
Discussion started by: dssyadav
7 Replies

8. UNIX for Dummies Questions & Answers

script works well but displays " line 6: =: No such file or directory"

strange :) can you tell why?:cool: #!/bin/bash echo " enter your age " read age if ; then echo " you do not have to pay tax " elif ]; then echo " you are eligible for income tax " else echo " you dont have to pay tax " fi (3 Replies)
Discussion started by: me.
3 Replies

9. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

10. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question