Running program and output files in specific directories


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Running program and output files in specific directories
# 8  
Old 02-18-2018
Have you considered using an absolute pathname on the find (i.e. find "$PWD" ... instead of find . ...) and moving to the directory in which you want the files to be created before invoking mseed2sac instead of moving all of the files mseed2sac creates after it is done?

Your script will run faster and use fewer system resources if you change:
Code:
  dir=$(dirname "$fn")  # Gets directory path
  fnm=$(basename "$fn") # Gets filename excl. path

to:
Code:
  dir=${fn%/*}  # Gets directory path
  fnm=${fn##*/} # Gets filename excl. path

And, I assume that you will also actually invoke mseed2sac and mkdir instead of just including them in comments and echo statements. (This also applies to mv if you decide to ignore my first suggestion above.)
This User Gave Thanks to Don Cragun For This Post:
# 9  
Old 02-18-2018
Calling the commands

That's correct. I am just doing echo to check the commands created are correct so
when I happy with how things are set up, I can then invoke them properly.

---------- Post updated at 05:12 PM ---------- Previous update was at 04:24 PM ----------

Should I do `cd` to the directory so that I can run `mseed2sac`? I am not following
very well how to use `find $pwd`

I give here some of the output. There are lot of stations

Code:
ls iv/
agst  cavt  cmpr  dgi   esln  gib   hlni  lado  mesg  mmgo  mpnc  mtgr   plac  raff  soi  alja  
cgl   corl  ecnv  esml  gmb   hmdc  lpdg  meu   mno   mrlc  noci   plln  resu  solun  vent
cafe  clta  crac  emsg  favr  haga  hpac  ltrz  mfnl  mpaz  msfr  nov    psb1  scte  ssy
cagr  cmdo  crja  epzf  galf  havl  hvzn  mct   milz  mpg   msru  petra  ptcc  sers

ls iv/favr/
bhe.d/ bhn.d/ bhz.d/ hhe.d/ hhn.d/ hhz.d/ lhe.d/ lhn.d/ lhz.d/ vhe.d/ vhn.d/ vhz.d/

Code:
fn: ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.141
+ dir: ./iv/ptcc/hhz.d
+ fnm: iv.ptcc..hhz.d.2016.141
+ mkdir -p ./iv.sac/ptcc/hhz.d
cd ./iv/ptcc/hhz.d
+ /home/hagbard/swadmin/swbuild/mseed2sac/mseed2sac ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.141
+ mv iv.ptcc..hhz.d.2016.141.* ./iv.sac/ptcc/hhz.d/
[*                                                           ]   1%

fn: ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.021
+ dir: ./iv/ptcc/hhz.d
+ fnm: iv.ptcc..hhz.d.2016.021
+ mkdir -p ./iv.sac/ptcc/hhz.d
cd ./iv/ptcc/hhz.d
+ /home/hagbard/swadmin/swbuild/mseed2sac/mseed2sac ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.021
+ mv iv.ptcc..hhz.d.2016.021.* ./iv.sac/ptcc/hhz.d/
[*                                                           ]   1%

fn: ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.350
+ dir: ./iv/ptcc/hhz.d
+ fnm: iv.ptcc..hhz.d.2016.350
+ mkdir -p ./iv.sac/ptcc/hhz.d
cd ./iv/ptcc/hhz.d
+ /home/hagbard/swadmin/swbuild/mseed2sac/mseed2sac ./iv/ptcc/hhz.d/iv.ptcc..hhz.d.2016.350
+ mv iv.ptcc..hhz.d.2016.350.* ./iv.sac/ptcc/hhz.d/
[*                                                           ]   1%

---------- Post updated at 05:55 PM ---------- Previous update was at 05:12 PM ----------

Quote:
Originally Posted by Don Cragun
Have you considered using an absolute pathname on the find (i.e. find "$PWD" ... instead of find . ...) and moving to the directory in which you want the files to be created before invoking mseed2sac instead of moving all of the files mseed2sac creates after it is done?

Your script will run faster and use fewer system resources if you change:
Code:
  dir=$(dirname "$fn")  # Gets directory path
  fnm=$(basename "$fn") # Gets filename excl. path

to:
Code:
  dir=${fn%/*}  # Gets directory path
  fnm=${fn##*/} # Gets filename excl. path

And, I assume that you will also actually invoke mseed2sac and mkdir instead of just including them in comments and echo statements. (This also applies to mv if you decide to ignore my first suggestion above.)
I get the same results

Code:
dir=$(dirname "$fn")
fnm=$(basename "$fn")

as doing

Code:
dir=${fn%/*}
fnm=${fn##*/}

How is one faster than the latter? What does `dirname` and `basename` work?

I did a test and variable substitution is faster as you say.

Last edited by kristinu; 02-18-2018 at 07:08 PM..
# 10  
Old 02-18-2018
Quote:
Originally Posted by kristinu
That's correct. I am just doing echo to check the commands created are correct so
when I happy with how things are set up, I can then invoke them properly.

---------- Post updated at 05:12 PM ---------- Previous update was at 04:24 PM ----------

Should I do `cd` to the directory so that I can run `mseed2sac`? I am not following
very well how to use `find $pwd`
First, I suggested using find $PWD ...; not find $pwd .... Your shell sets PWD to an absolute pathname of your current working directory when it starts up and updates it every time you successfully execute a cd command. (When you successfully execute cd, your shell also sets OLDPWD to an absolute pathname of the directory you were in before you executed cd.)

There is no reason to think that the value assigned to the variable pwd will have any value assigned to it unless your script does that. And $pwd will not be updated by your shell when you execute cd!

Your code does the following:
  1. read a relative pathname of an input file to be processed from /tmp/wrk,
  2. runs mseed2sac,
  3. calculates the directory where the output file(s) should be located,
  4. create that directory (if it doesn't already exist), and
  5. move the output file(s) to that directory.

My suggestion is to change that to:
  1. read an absolute pathname of an input file to be processed from /tmp/wrk,
  2. calculate the directory where the output file(s) should be located,
  3. create that directory (if it doesn't already exist),
  4. cd to that directory, and
  5. run mseed2sac.
Note that no mv is included in the above suggestion.
This User Gave Thanks to Don Cragun For This Post:
# 11  
Old 02-18-2018
Quote:
Originally Posted by kristinu
... ... ...

I get the same results

Code:
dir=$(dirname "$fn")
fnm=$(basename "$fn")

as doing

Code:
dir=${fn%/*}
fnm=${fn##*/}

How is one faster than the latter? What does `dirname` and `basename` work?

I did a test and variable substitution is faster as you say.
Your code uses command substitution (i.e., $(command arguments)). That involves forking a shell, executing command, waiting for command to finish, reading the results from command, and assigning them to a variable.

My code uses variable expansion which is done entirely in the shell. The fork and exec done by command substitution is a very slow shell operation. The string manipulations done by variable expansions are much faster.
This User Gave Thanks to Don Cragun For This Post:
# 12  
Old 02-19-2018
Right, `dirname` and `basename` involve executing a system program.

---------- Post updated 02-19-18 at 06:51 AM ---------- Previous update was 02-18-18 at 06:57 PM ----------

Have made the changes and understand now what you meant when using `PWD` and `OLDPWD`.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Move several files into specific directories with a loop

Hello, I'm a first time poster looking for help in scripting a task in my daily routine. I am new in unix but i am attracted to its use as a mac user. Bear with me... I have several files (20) that I manually drag via the mouse into several named directories over a network. I've used rsync... (14 Replies)
Discussion started by: SonnyClark
14 Replies

2. Shell Programming and Scripting

How to delete all the files and folders inside all the directories except some specific directory?

hi, i have a requirement to delete all the files from all the directories except some specific directories like archive and log. for example: there are following directories such as A B C D Archive E Log F which contains some sub directories and files. The requirement is to delete all the... (7 Replies)
Discussion started by: Little
7 Replies

3. Solaris

A way to list directories that contain specific files.

Hi everyone My issue is this, I need to list all the sub directories in a directory that contains files that have the extension *.log, *.dat and *.out . After reviewing the output i need to delete those directories i do not need. I am running Solaris 10 in a bash shell. I have a script that I... (2 Replies)
Discussion started by: jsabo40
2 Replies

4. Shell Programming and Scripting

Find specific files only in current directory...not sub directories AIX

Hi, I have to find specific files only in the current directory...not in the sub directories. But when I use Find command ... it searches all the files in the current directory as well as in the subdirectories. I am using AIX-UNIX machine.Please help.. I am using the below command. And i am... (2 Replies)
Discussion started by: aakishore
2 Replies

5. Shell Programming and Scripting

running C program to output in multiple locations

Hi Guys, What I am looking at doing is to run a C program in my home directory, but output files in multiple directories BUT not at the same instance. For e.g. 1st instance: Run program.c and output results in path /aaa/bbb/ccc/ 2nd instance: Run program.c again and output results... (9 Replies)
Discussion started by: Jatsui
9 Replies

6. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

7. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

8. Shell Programming and Scripting

grep'ing for specific directories, and using the output to move files

Hello, this is probably another really simple tasks for most of you gurus, however I am trying to make a script which takes an input, greps a specific file for that input, prints back to screen the results (which are directory names) and then be able to use the directory names to move files.... (1 Reply)
Discussion started by: JayC89
1 Replies

9. Shell Programming and Scripting

List specific files from directories

Hello, I would like to list the files from all directories that has been modified more than 1 month ago, and whose name is like '*risk*log'. I think a script like this should work : ls -R | find -name '*risk*.log' -mtime 30 -type f But it tells me "no file found" though I can see some. ... (4 Replies)
Discussion started by: Filippo
4 Replies
Login or Register to Ask a Question