Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.


Bash to copy subfolder and files to matching directory


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Bash to copy subfolder and files to matching directory

The bash executes but returns no results and the set -xv showed while the $run variable in blue, was extracted correctly, the $match value in green, was not, rather both values in home/cmccabe/Desktop/f1 were extracted not just the matching.

There will always be an exact match from the $run to the $match but there may be more than one. In all cases I am trying to cp the $run to the corresponding $match. When a match is found the contents of the $run subfolder and .zip and copied to $match. I havent added this as I am not sure how. Thank you Smilie.


run directory ---- this is $run ----
Code:
R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions   --- top level ---
   f1  --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- zip file in f1 ---
   f2   --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- zip file in f2 ---
   f3   --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- --- zip file in f3 ---

match directory ---- this is $match ----
Code:
R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions
R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions

Code:
# grab run directory
set -xv
for dir in /home/cmccabe/medex.logs/ponc/*/; do
  if [ -d $dir ]; then
     run=$(basename $dir)
  fi
# find match and copy run
cd /home/cmccabe/Desktop/f1
match=$(find * -maxdepth 1 -type d)
   if [ $run = $match ]
    then
   #cp -v $run /home/cmccabe/Desktop/f1/$match
  fi
done

bash [too many arguments

set -xv

Code:
+ '[' R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions = R_2019_01_23_13_22_58_user_S5-0271-93-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ']'


Last edited by cmccabe; 03-14-2019 at 11:26 AM.. Reason: fixed format
# 2  
Your code typo corrected. It is POSIX shell compliant, and it works with Bash too.

Code:
#!/bin/sh

# POSIX shell compliant.

# grab run directory.
set -xv

for dir in /home/cmccabe/medex.logs/ponc/*/ ; do


  if [ -d "$dir" ] ; then
     run="$(basename "$dir")"
  fi


  # find match and copy run.
  # the program exits 1, in case the directory can not be accessed, or does not exist.
  cd /home/cmccabe/Desktop/f1 || exit 1


  # https://github.com/koalaman/shellcheck/wiki/SC2035
  match="$(find ./* -maxdepth 1 -type d)"


  if [ "$run" = "$match" ]
  then
    # cp -v "$run" /home/cmccabe/Desktop/f1/"$match"
  fi


done

This User Gave Thanks to johnprogrammer For This Post:
# 3  
I think I finished your program. Test it, and tell us if it works.

Code:
#!/bin/sh

# POSIX shell compliant.

# grab run directory.
set -xv

for dir in /home/cmccabe/medex.logs/ponc/*/ ; do


  if [ -d "$dir" ] ; then
     run="$(basename "$dir")"
  fi


  # the program exits 1, in case the directory can not be accessed, or does not exist.

  cd /home/cmccabe/Desktop/f1 || exit 1


  # https://github.com/koalaman/shellcheck/wiki/SC2035
  
  findOutput="$(find ./* -maxdepth 1 -type d)"


  # find match and copy run.

  for match in $findOutput
  do
  
    if [ "$run" = "$match" ]
    then

      cp -Rv "$run" /home/cmccabe/Desktop/f1/"$match"
  
    fi
  
  done


done

exit 0


Last edited by johnprogrammer; 03-14-2019 at 08:11 PM..
This User Gave Thanks to johnprogrammer For This Post:
# 4  
The script executes and moves the copies the matching directory (parent) only. The sub-folders within it are not copied over.
So the filename in green below is copied but not the subdirectories within it. Thank you Smilie.

Code:
R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions   --- top level ---
   f1  --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- zip file in f1 ---
   f2   --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- zip file in f2 ---
   f3   --- subfolder under R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions ---
    zip --- --- zip file in f3 ---

match directory ---- this is $match ----
Code:
R_2019_01_23_13_22_58_user_S5-0271-93-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions
R_2019_01_23_13_22_58_user_S5-0271-96-v5.6_Oncomine_Childhood_Cancer_Research_DNA_and_Fusions


Last edited by cmccabe; 03-15-2019 at 10:17 AM..
# 5  
See the following proposal, with some comments:
Code:
#!/bin/sh

# grab run directory.

# POSIX shell compliant.

# external commands to be found in
PATH=/bin:/usr/bin

# debug:
#set -xv

# dryrun:
dryrun="echo"

# exit in case the directory can not be accessed
cd /home/cmccabe/Desktop/f1 || exit 1

for dir in /home/cmccabe/medex.logs/ponc/*/
do

  # the */ filters for directories, the following checks for a null match
  [ -d "$dir" ] || continue

  run=$(basename "$dir")

  # find match and copy run.

  # https://github.com/koalaman/shellcheck/wiki/SC2035
  # A * glob can overflow "too many arguments"
  # get two levels * and */*
  #match=$(find . -mindepth 1 -maxdepth 2 -type d)
  #if [ "$run" = "$match" ]
  # can this ever be true? $match is newline-separated!
  
  # test one level; the 2nd level would be */"$run", it could loop thru both with
  #for match in "$run" */"$run"
  match=$run
  if [ -d "$match" ]
  then
    # the following will copy to $match/$run/
    #$dryrun cp -Rv "$dir" "$match"
    # the following will copy to $match/
    $dryrun cp -Rv "$dir" "$(dirname "$match")"
  fi

done

This User Gave Thanks to MadeInGermany For This Post:
# 6  
There are logical errors in the code.

For example
Code:
for dir in /home/cmccabe/medex.logs/ponc/*/ ; do

  if [ -d "$dir" ] ; then
     run="$(basename "$dir")"
  fi

$run is always *

Please tell where the source directories to be copied are located (the full directory path), and what is the target directory (the full directory path).

Last edited by johnprogrammer; 03-15-2019 at 11:08 AM..
This User Gave Thanks to johnprogrammer For This Post:
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

Delete all files if another files in the same directory has a matching occurrence of a specific word

he following are the files available in my directory RSK_123_20141113_031500.txt RSK_123_20141113_081500.txt RSK_126_20141113_041500.txt RSK_126_20141113_081800.txt RSK_128_20141113_091600.txt Here, "RSK" is file prefix and 123 is a code name and rest is just timestamp of the file when its... (7 Replies)
Discussion started by: kridhick
7 Replies

3. Shell Programming and Scripting

Copy files matching multiple conditions

Hello How do i copy files matching multiple conditions. Requirement is to search files starting with name abc* and def* and created on a particular date or date range given by the user and copy it to the destination folder. i tried with different commands. below one will give the list ,... (5 Replies)
Discussion started by: NarayanaPrakash
5 Replies

4. UNIX for Dummies Questions & Answers

[SOLVED] Copy subfolder to another directory preserving parent name

Hi, I have the following folder structure: Folder6100/Data Folder6100/Data Folder6120/Data Folder6120/Data Folder6140/Data Folder6140/Data Folder6160/Data Folder6160/Data Folder6180/Data Folder6180/Data Folder6200/Data Folder6220/Data Folder6240/Data Folder6260/Data... (2 Replies)
Discussion started by: alex2005
2 Replies

5. Shell Programming and Scripting

Bash script to copy apache log files to client directory

Our Apache log files are written to a location on the server that we as clients have no access. Don't ask. Every month, I have to e-mail the administrator to have him manually copy our Apache log files to a directory in our file space. You can probably guess how efficient it is to do things this... (3 Replies)
Discussion started by: gregraven
3 Replies

6. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did... (4 Replies)
Discussion started by: santosh2626
4 Replies

7. UNIX for Dummies Questions & Answers

copy all files matching the request and change the extension at the same time

Hi everyone When I'm starting my script I'm giving to it two parameters: script.sh ext1 ext2 I need to copy all files in a directory fitting ext1, to the same folder, with the same names, but with the changed extension to ext2. Till now I've just managed to do it for only 1 file, but I... (16 Replies)
Discussion started by: vacuity93
16 Replies

8. Shell Programming and Scripting

Link multiple files from different subfolder to a new subfolder

Hi, I have the following subfolder with files: /data/a/1/xxx.txt /data/b/2/yyy.txt /data/c/3/zzz.txt And i have a set of new folders which have exactly the same structure as above but different disk without the files: /data_02/a/1/ /data_02/b/2/ /data_02/c/3/ Now i would like to... (6 Replies)
Discussion started by: total_ysf
6 Replies

9. Shell Programming and Scripting

Delete all files if another files in the same directory has a matching occurence of a specific word

Hello, I have several files in a specific directory. A specific string in one file can occur in another files. If this string is in other files. Then all the files in which this string occured should be deleted and only 1 file should remain with the string. Example. file1 ShortName "Blue... (2 Replies)
Discussion started by: premier_de
2 Replies

10. Shell Programming and Scripting

[Help]RegEx, Putty, Copy Files Insensitive Matching, SSH

Alright, basically we're in the whole where we can't tar/gzip a folder since its to big so how do I copy files to a new folder for example I got files from a-Z, i want to copy all files which starts with a A or a into another folder heres file structure ./backups/A ./backups/B... (11 Replies)
Discussion started by: Lamonte
11 Replies

Featured Tech Videos