Bash to check directory and create missing folder from file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to check directory and create missing folder from file
# 8  
Old 09-30-2019
STRING is empty. I believe this is because of / character before and after ref variable:

Code:
$ echo "test" | awk 'match($0, /est/) { print $1 }'
test
$ echo "test" | awk -v ref=/est/ 'match($0, ref) { print $1 }'
$ echo "test" | awk -v ref=est 'match($0, ref) { print $1 }'
test

Notice how the / characters are not needed when the 2nd match argument is a variable.
This User Gave Thanks to Chubler_XL For This Post:
# 9  
Old 10-02-2019
I made some edits to the code and am closer but am not performing the $STRING to $folder comparison correctly and the wrong folder gets created. I don't know if there is a better way but hopefullt I am closer Thank you Smilie.

Code:
DIR=/home/cmccbe/folder
for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
  S5=${RDIR##*/}  ## store run with no path as S5
  TRIMSTR=${RDIR%%-v5.6*}  ## trim folder match in RDIR from -v5.6 and store in TRIMSTR
    mv "$RDIR" "$TRIMSTR"  ## move trimmed folder to directory
  cd "$TRIMSTR"
   folders=`ls -d *[0-9]*/`  ## list folders
    STRING=$(awk -F '\n' -v RS="" -v ref="$S5" '$0 ~ ref {print $2}' "$DIR"/f1)
if [[ $folders == $STRING* ]] ## only execute folder on match
    then
      continue
  if [[ $folders != $STRING* ]] ## only execute folder on mis-match
   then
     mkdir -p "$TRIMSTR"/variants
 fi
  fi
done

set -x
Code:
+ for rdir in '"$dir"/R_2019*'
+ s5=R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx
+ trimstr=/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ mv /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx 
/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ cd /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
++ ls --color=auto -d 00-1111-xxx-xxx/ 00-2222-yyy-yy/
+ folders='00-1111-xxx-xxx/
00-2222-yyy-yy/'
++ awk -F '\n' -v RS= -v ref=R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx '$0 ~ ref {print $2}' 
/home/cmccabe/folder/f1
+ string='zzzz_0005 00-1111-xxx-xxx'
+ [[ 00-1111-xxx-xxx/
00-2222-yyy-yy/ != zzzz_0005 00-1111-xxx-xxx* ]]
+ mkdir -p '/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000- 
123/zzzz_0005 00-1111-xxx-xxx/folder'

desired
Code:
  --- $TRIMSTR/$STRIG/folder ---
R_2019_00_00_00_00_00_xxxx_x0-00000-123
       00-1111-xxx-xxx
           folder
       00-2222-yyy-yy
           folder
       00-3333-zz-zz
           folder

--- Post updated at 06:53 AM ---

I made some edits to the code and am closer but am not performing the $STRING to $folder comparison correctly and the wrong folder gets created. Thank you Smilie.

Code:
DIR=/home/cmccbe/folder
for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
  S5=${RDIR##*/}  ## store run with no path as S5
  TRIMSTR=${RDIR%%-v5.6*}  ## trim folder match in RDIR from -v5.6 and store in TRIMSTR
    mv "$RDIR" "$TRIMSTR"  ## move trimmed folder to directory
  cd "$TRIMSTR"
   folders=`ls -d *[0-9]*/`  ## list folders
    STRING=$(awk -F '\n' -v RS="" -v ref="$S5" '$0 ~ ref {print $2}' "$DIR"/f1)
  if [[ $folders != $STRING* ]] ## only execute folder on mis-match
   then
     mkdir -p "$TRIMSTR"/$STRING/folder
  fi
done

set -x
Code:
+ for rdir in '"$dir"/R_2019*'
+ s5=R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx
+ trimstr=/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ mv /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx 
/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ cd /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
++ ls --color=auto -d 00-1111-xxx-xxx/ 00-2222-yyy-yy/
+ folders='00-1111-xxx-xxx/
00-2222-yyy-yy/'
++ awk -F '\n' -v RS= -v ref=R_2019_00_00_00_00_00_xxxx_x0-00000-123- 
v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx '$0 ~ ref {print $2}' 
/home/cmccabe/folder/f1
+ string='zzzz_0005 00-1111-xxx-xxx'
+ [[ 00-1111-xxx-xxx/
00-2222-yyy-yy/ != zzzz_0005 00-1111-xxx-xxx* ]]
+ mkdir -p '/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000- 
123/zzzz_0005 00-1111-xxx-xxx/folder'

desired
Code:
  --- $TRIMSTR/$STRING/folder ---
R_2019_00_00_00_00_00_xxxx_x0-00000-123
       00-1111-xxx-xxx
           folder
       00-2222-yyy-yy
           folder
       00-3333-zz-zz
           folder

thought process
Code:
DIR=/home/cmccabe/Desktop/new    --- define directory ---
for RDIR in "$DIR"/R_2019* ; do  ---  /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx ---
  S5=${RDIR##*/}  --- R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx ---
  TRIMSTR=${RDIR%%-v5.6*}  --- --- R_2019_00_00_00_00_00_xxxx_x0-00000-123 ---
    mv "$RDIR" "$TRIMSTR"  --- /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx >>> /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6
  cd "$TRIMSTR"   --- look in /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6 ---
   folders=`ls -d *[0-9]*/`  --- 00-1111-xxx-xxx/ 00-2222-yyy-yy/ --- 
     STRING=$(awk -F '\n' -v RS="" -v ref="$S5" '$0 ~ ref {print $2}' "$DIR"/f1)   --- match R_2019 from $S5 line in f1 and print 00-1111-xxx-xxx/ 00-2222-yyy-yy 00-3333-zz-zz to $STRING --
if [[ $folders == $STRING/* ]] ## only folder on match --- 00-1111-xxx-xxx/ 00-2222-yyy-yy are skipped as they match $STRING ---
    then
      continue
   if [[ $folders != $STRING/* ]] ## only execute folder on mis-match   --- 00-3333-zz-zz is not equal to $STRING  ---
    then
      mkdir -p "$TRIMSTR"/variants  --- 00-3333-zz-zz/variants is made in /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6 ---
   fi
  fi
done


Last edited by cmccabe; 10-02-2019 at 05:10 PM.. Reason: fixed format, updated awk, added thought process
# 10  
Old 10-02-2019
I'm still a little confused on what you want to happen.

You end up with variables as follows:
folders
Code:
00-1111-xxx-xxx/
00-2222-yyy-yy/

string
Code:
zzzz_0005 00-1111-xxx-xxx

You seem to imply there is some sort of looping thru the folders variable and a comparison with stringwhich then results in creating folders. However you are just performing a single comparison between the two variables.
This User Gave Thanks to Chubler_XL For This Post:
# 11  
Old 10-02-2019
Quote:
You seem to imply there is some sort of looping thru the folders variable and a comparison with string which then results in creating folders. However you are just performing a single comparison between the two variables.
Yes that is correct. If the $folder variable does not match $STRING then it does not exist, so it is created (this will happen ocassionally). If the $folder variable does match $STRING then it already exists and can be skipped (this will be the majority of the time). Thank you Smilie.
# 12  
Old 10-02-2019
Is this closer to what you need?

Code:
DIR=/home/cmccbe/folder
for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
  S5=${RDIR##*/}  ## store run with no path as S5
  TRIMSTR=${RDIR%%-v5.6*}  ## trim folder match in RDIR from -v5.6 and store in TRIMSTR
    mv "$RDIR" "$TRIMSTR"  ## move trimmed folder to directory
  cd "$TRIMSTR"
  STRING=$(awk -F '\n' -v RS="" -v ref="$S5" '$0 ~ ref {print $2}' "$DIR"/f1)
  for folder in *[0-9]*/ ; do
      folder=${folder%/}   ## remove trailing slash
      if ! [[ " $STRING " == *\ $folder\ * ]]
      then
           ## folder is not within $STRING so make variants folder and leave
           mkdir -p "$TRIMSTR"/variants
           break
       end
  done
done

This User Gave Thanks to Chubler_XL For This Post:
# 13  
Old 10-03-2019
I added set -x and the $STRING variable looks to contain only one of the $2 values from f1, instead of the three so the directory created is not the right one. Thank you very much Smilie.

f1
Code:
yyy_009 00-0000-xxx-xxx-xxx
yyy_004 00-0011-xxx-xxx-xxx
R_2019_00_00_00_00_00_xxxx_x0-00000-200_yyyyyyy

zzzz_0000 00-3333-zz-zz  --- not printed to $STRING currently, but should be for the comparison ---
zzzz_0005 00-1111-xxx-xxx  --- printed to $STRING currently ---
zzzz_0003 00-2222-yyy-yy   --- not printed to $STRING currently, but should be for the comparison  ---
R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx   ---- this is matched ---

$folder
Code:
zzzz_0005 00-1111-xxx-xxx  --- this is correct ---
zzzz_0003 00-2222-yyy-yy

so since
Code:
00-3333-zz-zz

is not in folder 00-3333-zz-zz/variants is created

Code:
+ for RDIR in '"$DIR"/R_2019*'
+ S5=R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx
+ TRIMSTR=/home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ mv /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ cd /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123
++ awk -F '\n' -v RS= -v ref=R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx '$0 ~ ref {print $2}' /home/cmccabe/Desktop/new/f1
+ STRING='zzzz_0005 00-1111-xxx-xxx'
+ for folder in '*[0-9]*/'
+ folder=00-1111-xxx-xxx
+ [[  zzzz_0005 00-1111-xxx-xxx  == *\ 00-1111-xxx-xxx\ * ]]
+ for folder in '*[0-9]*/'
+ folder=00-2222-yyy-yy
+ [[  zzzz_0005 00-1111-xxx-xxx  == *\ 00-2222-yyy-yy\ * ]]
+ mkdir -p /home/cmccabe/Desktop/new/R_2019_00_00_00_00_00_xxxx_x0-00000-123/00-2222-yyy-yy/variants
+ break

# 14  
Old 10-03-2019
I believe you will need an awk script like this:

Code:
STRING=$(awk -F '\n' -v RS="" -v ref="$S5" '$0 ~ ref {d=split($0, val, " "); for(i=2;i<d;i+=2) printf "%s ",val[i]; printf "\n"}' "$DIR"/f1)

This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create directory and sub-directory with awk and bash

In the below I am trying to create a parent directory using the R_2019 line from f1 if what above it is not empty. I then create sub-directories under each parent if there is a match between $2 of f1 and $2. Inside each sub-folder the matching paths in $3 and $4 in f2are printed. If there is no... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Create automated scan of specific directory using bash

I am trying to use bash to automate the scan of a specific directory using clamav. Having this in place is a network requirement. The below is an attempt to: 1. count the extensions (.txt, .jpeg) in a directory and write them to a virus-scan.log (section in bold) 2. scan each folder in the... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. Shell Programming and Scripting

Bash to select oldest folder in directory and write to log

In the bash below the oldest folder in a directory is selected. If there are 3folders in the directory /home/cmccabe/Desktop/NGS/test and nothing is done to them (ie. no files deleted, renamed) then the bash correctly identifies f1 as the oldest. However, if something is done to the folder then... (4 Replies)
Discussion started by: cmccabe
4 Replies

4. Shell Programming and Scripting

Bash to add folder to exsisting folders in directory

I am trying to create subdirectories in each folder in /home/cmccabe/Desktop/NGS/test/*. When I use echo I can see each folder in the directory, but I can not seem to add the commented out portion in bold. These are the sub-directories and sub-folders I am trying to add to each folder in... (1 Reply)
Discussion started by: cmccabe
1 Replies

5. Shell Programming and Scripting

Bash to select oldest folder in directory automatically and log process

The `bash` below uses the oldest folder in the specified directory and logs it. The goes though an analysis process and creates a log. My problem is that if there are 3 folders in the directory folder1,folder2,folder3, the bash is using folder2 for the analysis eventhough folder1 is the oldest... (0 Replies)
Discussion started by: cmccabe
0 Replies

6. Shell Programming and Scripting

Create a folder under different user directory

Hello All, I have to write a shell script and use it in informatica. The script has to perform below actions: The script gets executed from edw user. Through the script, a DT folder has to be created under edw_sca user. Is this scenario possible through a SHELL script or not. ... (2 Replies)
Discussion started by: bghosh
2 Replies

7. Shell Programming and Scripting

how to check if a directory exists or not.if not need to create it

Hi, I am using solaris 10 OS and bash shell.just checking for small code snippet which follows below. /export/home/vomappservers/spa/common/5.0 /export/home/vomappservers/spa/common/scripts /export/home/vomappservers/spa/tools /export/home/vomappservers/spa/scm5.0/SCCS... (5 Replies)
Discussion started by: muraliinfy04
5 Replies

8. Shell Programming and Scripting

check if multiple directories exist else create missing directories

Hi , I 'm trying to check if multiple directories exist on a server, if not create the missing ones and print " creating missing directory. how to write this in a simple script, I have made my code complex if ; then taskStatus="Schema extract directory exists, checking if SQL,Count and... (7 Replies)
Discussion started by: ramky79
7 Replies

9. Shell Programming and Scripting

To check whether a directory is exist and if it is not, create it

Hi, I want to write a shell script to check whether a directory (say A) is existing in a given location and if it is not, create it. (3 Replies)
Discussion started by: sabya
3 Replies

10. Shell Programming and Scripting

Use awk to create new folder in current directory

Alright, I am sure this is a laughable question, but I don't know so I am going to ask anyway. I have a little script I am writing to take information from one source, recode it in a certain way, and print to files for each subject I have data for. This all works perfectly. I just want to put... (6 Replies)
Discussion started by: ccox85
6 Replies
Login or Register to Ask a Question