Running program and output files in specific directories


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Running program and output files in specific directories
# 1  
Running program and output files in specific directories

I have been running a program mseed2sac using the following command

Code:
cd IV
find . -type f -exec /swadmin/mseed2sac '{}' \;

The problem is that I end up with a lot of files in directory IV.
Instead I would like to select the designator HHZ, create a
directory IV.SAC and all the files output for mseed2sac
get stored there

The two directories IV and IV.SAC would be at the same level

Code:
IV.SAC
├── PTCC
    ├── HHZ.D
    │   ├── IV.PTCC..HHZ.D.2016.001.SAC
    │   ├── IV.PTCC..HHZ.D.2016.002.SAC
    │   ├── IV.PTCC..HHZ.D.2016.003.SAC
    │   ├── IV.PTCC..HHZ.D.2016.004.SAC
    │   ├── IV.PTCC..HHZ.D.2016.005.SAC
    │   ├── IV.PTCC..HHZ.D.2016.006.SAC
├── RAFF
    ├── HHZ.D
    │   ├── IV.RAFF..HHZ.D.2016.001.SAC
    │   ├── IV.RAFF..HHZ.D.2016.002.SAC
    │   ├── IV.RAFF..HHZ.D.2016.003.SAC
    │   ├── IV.RAFF..HHZ.D.2016.004.SAC
    │   ├── IV.RAFF..HHZ.D.2016.005.SAC
    │   ├── IV.RAFF..HHZ.D.2016.006.SAC

The original files would be in directory IV

Code:
├── IV
   ├── PTCC
   │   ├── BHE.D
   │   ├── BHN.D
   │   ├── BHZ.D
   │   ├── HHE.D
   │   ├── HHN.D
   │   ├── HHZ.D
   │   │    ├── IV.PTCC..HHZ.D.2016.001
   │   │    ├── IV.PTCC..HHZ.D.2016.002
   │   │    ├── IV.PTCC..HHZ.D.2016.003
   │   │    ├── IV.PTCC..HHZ.D.2016.004
   │   │    ├── IV.PTCC..HHZ.D.2016.005
   │   │    ├── IV.PTCC..HHZ.D.2016.006
   │   ├── LHE.D
   │   ├── LHN.D
   │   └── LHZ.D
   ├── RAFF
   │   ├── BHE.D
   │   ├── BHN.D
   │   ├── BHZ.D
   │   ├── HHE.D
   │   ├── HHN.D
   │   └── HHZ.D
   │   │    ├── IV.RAFF..HHZ.D.2016.001
   │   │    ├── IV.RAFF..HHZ.D.2016.002
   │   │    ├── IV.RAFF..HHZ.D.2016.003
   │   │    ├── IV.RAFF..HHZ.D.2016.004
   │   │    ├── IV.RAFF..HHZ.D.2016.005
   │   │    ├── IV.RAFF..HHZ.D.2016.006
   ├── RESU
   │   └── HHZ.D
   │   │    ├── IV.RESU..HHZ.D.2016.001
   │   │    ├── IV.RESU..HHZ.D.2016.002
   │   │    ├── IV.RESU..HHZ.D.2016.003
   │   │    ├── IV.RESU..HHZ.D.2016.004
   │   │    ├── IV.RESU..HHZ.D.2016.005
   │   │    ├── IV.RESU..HHZ.D.2016.006


Last edited by kristinu; 02-17-2018 at 12:42 AM.. Reason: Ordering Directory Structure
# 2  
As always:
  1. What operating system are you using?
  2. What shell are you using?
  3. What have you tried to solve this problem on your own?
One might guess that you will need to modify /swadmin/mseed2sac:
  1. Why haven't you shown us the man page for this utility?
  2. Why haven't you shown us the source code for this utility?
# 3  
mseed2sac

I am using Trisquel and writing a script in bash. I do not want to modify the program mseed2sac as I think it would be too much work and so arrange the file structure in a bash script.

For example when I do

Code:
 mseed2sac favr/hhz.d/iv.favr..hhz.d.2016.226
Wrote 8639767 samples to IV.FAVR..HHZ.D.2016.226.000003.SAC

I get the file IV.FAVR..HHZ.D.2016.226.000003.SAC at the location where I have run the program
mseed2sac.

Code:
/home/hagbard/swadmin/mseed2sac/mseed2sac -H
mseed2sac version: 2.2

Convert miniSEED data to SAC

Usage: mseed2sac [options] input1.mseed [input2.mseed ...]

 ## Options ##
 -V             Report program version
 -h             Show this usage message
 -H             Print an extended usage message
 -v             Be more verbose, multiple flags can be used
 -O             Overwrite existing output files, default creates new file names

 -k lat/lon     Specify station coordinates as 'Latitude/Longitude' in degrees
 -m metafile    File containing channel metadata (coordinates and more)
 -M metaline    Channel metadata, same format as lines in metafile
 -msi           Convert component inclination/dip from SEED to SAC convention
 -E event       Specify event parameters as 'Time[/Lat][/Lon][/Depth][/Name]'
                  e.g. '2006,123,15:27:08.7/-20.33/-174.03/65.5/Tonga'
 -l selectfile  Read a list of selections from file, used for subsetting

 -f format      Specify SAC file format (default is 2:binary):
                  1=alpha, 2=binary (host byte order),
                  3=binary (little-endian), 4=binary (big-endian)

 More options are available, to see their description use the -H option

 -N network     Specify the network code, overrides any value in the SEED
 -S station     Specify the station code, overrides any value in the SEED
 -L location    Specify the location code, overrides any value in the SEED
 -C channel     Specify the channel code, overrides any value in the SEED
 -r bytes       Specify SEED record length in bytes, autodetected by default
 -i             Process each input file individually instead of merged
 -ic            Process each channel individually, data should be well ordered
 -dr            Use the sampling rate derived from the time stamps instead
                  of the sample rate denoted in the input data
 -z zipfile     Write all SAC files to a ZIP archive, use '-' for stdout
 -z0 zipfile    Same as -z but do not compress archive entries

---------- Post updated at 09:18 AM ---------- Previous update was at 05:23 AM ----------

Have been trying to create a strong with the new directory name
using rename but it is not working very well.

Code:
./read-seed.sh iv/resu/hhz.d/iv.resu..hhz.d.2016.008
regex: 's/hhz/hhz.sac/g'
dir: iv/resu/hhz.d
odir:

Code:
stn="HHZ"
f="$1"
dir=$(dirname "${f}")

regex="'s/${stn}/${stn}.sac/g'"

odir=`echo "$dir" | rename $regex`

---------- Post updated at 09:37 AM ---------- Previous update was at 09:18 AM ----------

Now I have fixed it using sed

Code:
r=`echo "iv/resu/hhz.d/" | sed -e 's#hhz.d#hhz.d.sac#'`


Last edited by kristinu; 02-17-2018 at 07:31 AM..
# 4  
I'm very glad that you solved your problem.

We would have loved to have been able to help you, but with the information you provided, we weren't able to do much. I asked for the man page your mseed2sec. Instead of that you supplied us with a help message from that utility. Unfortunately, the help message says absolutely nothing about what the purpose of the utility is, what it will do with the input files, what, if any, output files it will produce, ...

And, then you told us that you wanted to invoke it using the format:
Code:
mseed2sac file1 file2 file2

which seemed very strange since there is no indication that feeding one file (file2 in this case) to the utility twice in one invocation would serve any useful purpose. But, you did later change this line to again just pass one pathname to your utility (without explaining why you made that change).
# 5  
Seed Format

You pass it a list of files in seed format (Standard for the Exchange of Earthquake Data)
and outputs a set of files in sac format (a format used by the Seismic Analysis Code SAC).

For example, running mseed2sac as follows
mseed2sac iv/cagr/hhz.d/iv.cagr..hhz.d.2016.208

The program then creates a number of files (in the directory where mseed2sac is run)
as shown in the list below. I want then to move to a new directory, for example to
`iv.sac/cagr/hhz.d/`

.d means day. So for 2016.208 means file for day 208.

Code:
iv.cagr..hhz.d.2016.208.000001.sac  iv.cagr..hhz.d.2016.208.043041.sac  iv.cagr..hhz.d.2016.208.070027.sac  iv.cagr..hhz.d.2016.208.121238.sac
iv.cagr..hhz.d.2016.208.131218.sac  iv.cagr..hhz.d.2016.208.132855.sac  iv.cagr..hhz.d.2016.208.132938.sac  iv.cagr..hhz.d.2016.208.133025.sac
iv.cagr..hhz.d.2016.208.133143.sac  iv.cagr..hhz.d.2016.208.133225.sac  iv.cagr..hhz.d.2016.208.133324.sac  iv.cagr..hhz.d.2016.208.133329.sac
iv.cagr..hhz.d.2016.208.133416.sac  iv.cagr..hhz.d.2016.208.133444.sac  iv.cagr..hhz.d.2016.208.133516.sac  iv.cagr..hhz.d.2016.208.133554.sac
iv.cagr..hhz.d.2016.208.133558.sac  iv.cagr..hhz.d.2016.208.133735.sac  iv.cagr..hhz.d.2016.208.133908.sac  iv.cagr..hhz.d.2016.208.133923.sac
iv.cagr..hhz.d.2016.208.134135.sac  iv.cagr..hhz.d.2016.208.134323.sac  iv.cagr..hhz.d.2016.208.134343.sac  iv.cagr..hhz.d.2016.208.134437.sac
iv.cagr..hhz.d.2016.208.134514.sac  iv.cagr..hhz.d.2016.208.134536.sac  iv.cagr..hhz.d.2016.208.134716.sac  iv.cagr..hhz.d.2016.208.134943.sac
iv.cagr..hhz.d.2016.208.135034.sac  iv.cagr..hhz.d.2016.208.135144.sac  iv.cagr..hhz.d.2016.208.135243.sac  iv.cagr..hhz.d.2016.208.135323.sac
iv.cagr..hhz.d.2016.208.135423.sac  iv.cagr..hhz.d.2016.208.135443.sac  iv.cagr..hhz.d.2016.208.135523.sac  iv.cagr..hhz.d.2016.208.135536.sac
iv.cagr..hhz.d.2016.208.135543.sac  iv.cagr..hhz.d.2016.208.135643.sac  iv.cagr..hhz.d.2016.208.135723.sac  iv.cagr..hhz.d.2016.208.135834.sac
iv.cagr..hhz.d.2016.208.135916.sac  iv.cagr..hhz.d.2016.208.140033.sac  iv.cagr..hhz.d.2016.208.140136.sac  iv.cagr..hhz.d.2016.208.140237.sac
iv.cagr..hhz.d.2016.208.140305.sac  iv.cagr..hhz.d.2016.208.140337.sac  iv.cagr..hhz.d.2016.208.140356.sac  iv.cagr..hhz.d.2016.208.140456.sac
iv.cagr..hhz.d.2016.208.140525.sac  iv.cagr..hhz.d.2016.208.140703.sac  iv.cagr..hhz.d.2016.208.140727.sac  iv.cagr..hhz.d.2016.208.140813.sac
iv.cagr..hhz.d.2016.208.140924.sac  iv.cagr..hhz.d.2016.208.140943.sac  iv.cagr..hhz.d.2016.208.141043.sac  iv.cagr..hhz.d.2016.208.141106.sac
iv.cagr..hhz.d.2016.208.141123.sac  iv.cagr..hhz.d.2016.208.141157.sac  iv.cagr..hhz.d.2016.208.141233.sac  iv.cagr..hhz.d.2016.208.141344.sac
iv.cagr..hhz.d.2016.208.141737.sac  iv.cagr..hhz.d.2016.208.141743.sac  iv.cagr..hhz.d.2016.208.141756.sac  iv.cagr..hhz.d.2016.208.141933.sac
iv.cagr..hhz.d.2016.208.141944.sac  iv.cagr..hhz.d.2016.208.142038.sac  iv.cagr..hhz.d.2016.208.142135.sac  iv.cagr..hhz.d.2016.208.142138.sac
iv.cagr..hhz.d.2016.208.142223.sac  iv.cagr..hhz.d.2016.208.142457.sac  iv.cagr..hhz.d.2016.208.142517.sac  iv.cagr..hhz.d.2016.208.142537.sac
iv.cagr..hhz.d.2016.208.142543.sac  iv.cagr..hhz.d.2016.208.142637.sac  iv.cagr..hhz.d.2016.208.142716.sac  iv.cagr..hhz.d.2016.208.142736.sac
iv.cagr..hhz.d.2016.208.142838.sac  iv.cagr..hhz.d.2016.208.142938.sac  iv.cagr..hhz.d.2016.208.142944.sac  iv.cagr..hhz.d.2016.208.143035.sac
iv.cagr..hhz.d.2016.208.143137.sac  iv.cagr..hhz.d.2016.208.143224.sac  iv.cagr..hhz.d.2016.208.144159.sac  iv.cagr..hhz.d.2016.208.144215.sac
iv.cagr..hhz.d.2016.208.144324.sac  iv.cagr..hhz.d.2016.208.144337.sac  iv.cagr..hhz.d.2016.208.144355.sac  iv.cagr..hhz.d.2016.208.144446.sac
iv.cagr..hhz.d.2016.208.144537.sac  iv.cagr..hhz.d.2016.208.144743.sac  iv.cagr..hhz.d.2016.208.144845.sac  iv.cagr..hhz.d.2016.208.144956.sac
iv.cagr..hhz.d.2016.208.145137.sac  iv.cagr..hhz.d.2016.208.145337.sac  iv.cagr..hhz.d.2016.208.145355.sac  iv.cagr..hhz.d.2016.208.145443.sac
iv.cagr..hhz.d.2016.208.145537.sac  iv.cagr..hhz.d.2016.208.145553.sac  iv.cagr..hhz.d.2016.208.145715.sac  iv.cagr..hhz.d.2016.208.145757.sac
iv.cagr..hhz.d.2016.208.150043.sac  iv.cagr..hhz.d.2016.208.150143.sac  iv.cagr..hhz.d.2016.208.150236.sac  iv.cagr..hhz.d.2016.208.150344.sac
iv.cagr..hhz.d.2016.208.150454.sac  iv.cagr..hhz.d.2016.208.150544.sac  iv.cagr..hhz.d.2016.208.150736.sac  iv.cagr..hhz.d.2016.208.150826.sac
iv.cagr..hhz.d.2016.208.151114.sac  iv.cagr..hhz.d.2016.208.151206.sac  iv.cagr..hhz.d.2016.208.151354.sac  iv.cagr..hhz.d.2016.208.151445.sac
iv.cagr..hhz.d.2016.208.151605.sac  iv.cagr..hhz.d.2016.208.151623.sac  iv.cagr..hhz.d.2016.208.151837.sac  iv.cagr..hhz.d.2016.208.152444.sac
iv.cagr..hhz.d.2016.208.153233.sac  iv.cagr..hhz.d.2016.208.153738.sac  iv.cagr..hhz.d.2016.208.154623.sac  iv.cagr..hhz.d.2016.208.155053.sac
iv.cagr..hhz.d.2016.208.155145.sac  iv.cagr..hhz.d.2016.208.155733.sac  iv.cagr..hhz.d.2016.208.155745.sac  iv.cagr..hhz.d.2016.208.155853.sac
iv.cagr..hhz.d.2016.208.155937.sac  iv.cagr..hhz.d.2016.208.161544.sac  iv.cagr..hhz.d.2016.208.164058.sac  iv.cagr..hhz.d.2016.208.164637.sac
iv.cagr..hhz.d.2016.208.164705.sac  iv.cagr..hhz.d.2016.208.164726.sac  iv.cagr..hhz.d.2016.208.165706.sac  iv.cagr..hhz.d.2016.208.171016.sac
iv.cagr..hhz.d.2016.208.171135.sac  iv.cagr..hhz.d.2016.208.171513.sac  iv.cagr..hhz.d.2016.208.171556.sac  iv.cagr..hhz.d.2016.208.171723.sac
iv.cagr..hhz.d.2016.208.172005.sac  iv.cagr..hhz.d.2016.208.172044.sac  iv.cagr..hhz.d.2016.208.172438.sac  iv.cagr..hhz.d.2016.208.180648.sac
iv.cagr..hhz.d.2016.208.192841.sac

---------- Post updated at 01:07 PM ---------- Previous update was at 11:26 AM ----------

Now I have updated the code as follows which transfer the generated files
to the correct directory. Any improvements or possible problems with my
code would be very welcome.

Code:
# Counts the number of files to process
totfcn=$(find . -type f | tee /tmp/wrk | wc -l)

i=0; j=0
while read fn; do    
  printf -v XXX "%0*d" $((60 * ++i / totfcn))
  printf "\r[%-60s]" "${XXX//0/*}"
  printf "%4d%%" $((100 * ++j / totfcn))
  #${dir_mseed2sac}/mseed2sac $fn
done < /tmp/wrk
printf "\n"

while read fn; do    
  #${dir_mseed2sac}/mseed2sac $fn
  dir=$(dirname "$fn")  # Gets directory path
  fnm=$(basename "$fn") # Gets filename excl. path
  rgx_nwk="s/${nwk}/${nwk}.sac/g"
  odir_nwk=`echo "$dir" | sed -e $rgx_nwk`
  ofl_nwk="${odir_nwk}/${fnm}"

  echo "fn: $fn"
  if [ -d "$odir_nwk" ]; then
    echo "Directory already exists: $odir_nwk"
  else
    echo "+ dir: $dir"
    echo "+ fnm: $fnm"
    echo "+ mkdir -p $odir_nwk"
  fi
  echo "+ mv ${fnm}.* ${odir_nwk}/"
done < /tmp/wrk


Last edited by kristinu; 02-18-2018 at 03:33 PM..
# 6  
Why don't you move each result file right after having created it in the first loop?

Please be aware that the nwk variable used in 11 places is not defined anywhere...
# 7  
Latest Update

Yes, I had that for test. The latest version is now. Naturally I will remove all the print
values and just keep the progress bar.

Code:
nwk="iv"
incl_nm="*hhz*"
# Counts the number of files to process
totfcn=$(find . -type f -name $incl_nm | tee /tmp/wrk | wc -l)

i=0; j=0
while read fn; do    

  dir=$(dirname "$fn")  # Gets directory path
  fnm=$(basename "$fn") # Gets filename excl. path
  rgx_nwk="s/${nwk}/${nwk}.sac/g"
  odir_nwk=`echo "$dir" | sed -e $rgx_nwk`
  ofl_nwk="${odir_nwk}/${fnm}"

  echo -e "\n\nfn: $fn"
  if [ -d "$odir_nwk" ]; then
    echo "Directory already exists: $odir_nwk"
  else
    echo "+ dir: $dir"
    echo "+ fnm: $fnm"
    echo "+ mkdir -p $odir_nwk"
  fi

  echo "+ mseed2sac $fn"
  #echo "${dir_mseed2sac}/mseed2sac $fn"
  echo "+ mv ${fnm}.* ${odir_nwk}/"

  printf -v XXX "%0*d" $((60 * ++i / totfcn))
  printf "\r[%-60s]" "${XXX//0/*}"
  printf "%4d%%" $((100 * ++j / totfcn))

done < /tmp/wrk
printf "\n"

Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #31
Difficulty: Easy
Alan Turing was the founder of IBM.
True or False?

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Move several files into specific directories with a loop

Hello, I'm a first time poster looking for help in scripting a task in my daily routine. I am new in unix but i am attracted to its use as a mac user. Bear with me... I have several files (20) that I manually drag via the mouse into several named directories over a network. I've used rsync... (2 Replies)
Discussion started by: SonnyClark
2 Replies

2. Shell Programming and Scripting

How to delete all the files and folders inside all the directories except some specific directory?

hi, i have a requirement to delete all the files from all the directories except some specific directories like archive and log. for example: there are following directories such as A B C D Archive E Log F which contains some sub directories and files. The requirement is to delete all the... (7 Replies)
Discussion started by: Little
7 Replies

3. Solaris

A way to list directories that contain specific files.

Hi everyone My issue is this, I need to list all the sub directories in a directory that contains files that have the extension *.log, *.dat and *.out . After reviewing the output i need to delete those directories i do not need. I am running Solaris 10 in a bash shell. I have a script that I... (2 Replies)
Discussion started by: jsabo40
2 Replies

4. Shell Programming and Scripting

Find specific files only in current directory...not sub directories AIX

Hi, I have to find specific files only in the current directory...not in the sub directories. But when I use Find command ... it searches all the files in the current directory as well as in the subdirectories. I am using AIX-UNIX machine.Please help.. I am using the below command. And i am... (2 Replies)
Discussion started by: aakishore
2 Replies

5. Shell Programming and Scripting

running C program to output in multiple locations

Hi Guys, What I am looking at doing is to run a C program in my home directory, but output files in multiple directories BUT not at the same instance. For e.g. 1st instance: Run program.c and output results in path /aaa/bbb/ccc/ 2nd instance: Run program.c again and output results... (9 Replies)
Discussion started by: Jatsui
9 Replies

6. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

7. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

8. Shell Programming and Scripting

grep'ing for specific directories, and using the output to move files

Hello, this is probably another really simple tasks for most of you gurus, however I am trying to make a script which takes an input, greps a specific file for that input, prints back to screen the results (which are directory names) and then be able to use the directory names to move files.... (1 Reply)
Discussion started by: JayC89
1 Replies

9. Shell Programming and Scripting

List specific files from directories

Hello, I would like to list the files from all directories that has been modified more than 1 month ago, and whose name is like '*risk*log'. I think a script like this should work : ls -R | find -name '*risk*.log' -mtime 30 -type f But it tells me "no file found" though I can see some. ... (4 Replies)
Discussion started by: Filippo
4 Replies

Featured Tech Videos