Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Bash to check directory and create missing folder from file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to check directory and create missing folder from file
# 1  
Old 09-27-2019
Bash to check directory and create missing folder from file
In the below bash I am trying to ensure that all folders (represented by $folders) in a given directory are created. In the file f1 the trimmed folder will be there somewhere (will be multiple trimmed folders).
When that trimmed folder is found (represented by $S5) the the contents of $2 printed to $STRING. If $STRING is the same as $folders then
that folder is already created in the directory and it can be skipped (this will be the case most of the time). If $STRING is not the same as $folders then the missing is created in the directory. The code seems to work as expected untill the If done, that is the variables are set correctly, but the actual match is portion below is not working and I am not sure how to mkdir. Thank you Smilie.


Code:
DIR=/home/cmccabe/folder

for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
  TRIMSTR=${RDIR%%-v5.6*}  ## trim folder match in RDIR from -v5.6 and store in TRIMSTR
     mv "$RDIR" "$TRIMSTR"  ## move trimmed folder to directory
     S5=${TRIMSTR##*/}  ## store run with no path as S5
# Iterate over each sample folder in run directory
cd "$TRIMSTR"
    folders=folders=`ls -d *[0-9]*/`  ## list only directories begining with digit 
done
for f in "$DIR"/f1; do STRING=$(awk -v ref="/$S5/" 'match($0, ref) {print $2}' "$f"); done
nl="  ## added newline boundry seperator 
"
    if [[ $nl$folders$nl == *$n$STRING/$nl* ]] || continue # only execute file on match
       else mkdir $STRING 
    fi

directory structure where variables come from
Code:
/path/to/run/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx

    --- /path/to/run/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123  --- this is $TRIMSTR and the blue is $S5

      --- 00-1111-xxx-xxx 00-2222-yyy-yy 00-3333-zz-zz id test   --- these are within $TRIMSTER id and test are not stored in $folders

f1
Code:
yyy_009 00-0000-xxx-xxx-xxx
yyy_004 00-0011-xxx-xxx-xxx
R_2019_00_00_00_00_00_xxxx_x0-00000-200_yyyyyyy

zzzz_0000 00-3333-zz-zz
zzzz_0005 00-1111-xxx-xxx
zzzz_0003 00-2222-yyy-yy
R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx

current
Code:
00-1111-xxx-xxx
00-2222-yyy-yy

desired
Code:
00-1111-xxx-xxx
00-2222-yyy-yy
00-3333-zz-zz

Last edited by cmccabe; 09-29-2019 at 08:31 AM.. Reason: updated script with comments
# 2  
Old 09-27-2019
You have a done statement on line 11 that doesn't match to anything.

S5=$(cut -d/ -f6 <<<"$TRIMSTR") ## store run with no path as S5
S5 will be empty because TRIMSTR doesn't contain 6 entries.

if [[ $folders = ${STRING[*]} ]]; then # only execute file on match
folders will contain space separated list of folders with a digit in their name, STRING is not an array, what are you trying to match here?
These 2 Users Gave Thanks to Chubler_XL For This Post:
cmccabe KD_999
# 3  
Old 09-27-2019
I made some edits to the code and am trying to match $TRIMSTER to the line in file f1. I think I should be looking for the pattern as f1 the file name is longer but includes [ICODE]$TRIMSTER[ICODE]. When a match is foung the values in$2 up to the empty line (which always seperates each block). I though those were being read into array $STRING which is then compared to $folders which is the sud-folders within $TRIMSTER. If $STRING[*] = $folders that means it is already there and can be skipped. However, if it is not equal then it is not there and a new folders is made in $TRIMSTR.. Thank you Smilie.
Last edited by RavinderSingh13; 09-28-2019 at 03:20 PM..
# 4  
Old 09-28-2019
A few suggestions:
Code:
for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
  test -d "$RDIR" || continue  ## is a directory, or jump to the next cycle

As you see it is continue not next.
Code:
S5=${TRIMSTR##*/}  ## strip path and store in S5

Code:
folders=`ls -d *[0-9]*/`

The trailing / ensures only directories match. But the trailing / is now part of the names.

To match a string in a list you can use a case-esac or as you tried, a [[ == ]]
The RHS takes the * wildcards.
For an exact match use boundaries. Here $folders is newline-separated, so we enclose the STRING in newlines, and we must enclose the LHS (folder list) in newlines so the first and last element can be matched.

Code:
nl="
"
    if [[ $nl$folders$nl == *$n$STRING$nl* ]]

If folder names have a trailing / you can add it to the STRING
Code:
    if [[ $nl$folders$nl == *$n$STRING/$nl* ]]

I hope this is helpful rather than confusing.
These 2 Users Gave Thanks to MadeInGermany For This Post:
cmccabe KD_999
# 5  
Old 09-29-2019
I updated the script in the original post with the helpful suggestions and added set -x. There are some syntax error and the $STRING variable doesn't get populated, though it looks like the expected values are being used. Thank you Smilie.


Code:
set -x
cmccabe@Satellite-M645:~$ DIR=/home/cmccabe/folder
+ DIR=/home/cmccabe/folder
cmccabe@Satellite-M645:~$ 
cmccabe@Satellite-M645:~$ for RDIR in "$DIR"/R_2019* ; do  ## # matching "R_2019*" to operate on desired directory and expand
>   TRIMSTR=${RDIR%%-v5.6*}  ## trim folder match in RDIR from -v5.6 and store in TRIMSTR
>      mv "$RDIR" "$TRIMSTR"  ## move trimmed folder to directory
>      S5=${TRIMSTR##*/}  ## store run with no path as S5
> # Iterate over each sample folder in run directory
> cd "$TRIMSTR"
>     folders=folders=`ls -d *[0-9]*/`  ## list only directories begining with digit 
> done
+ for RDIR in '"$DIR"/R_2019*'
+ TRIMSTR=/home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ mv /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123-v5.6_xxx_xxx_xxx_xxx_xxx_xxx_xxxx /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ S5=R_2019_00_00_00_00_00_xxxx_x0-00000-123
+ cd /home/cmccabe/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123
++ ls --color=auto -d 00-1111-xxx-xxx/ 00-2222-yyy-yy/
+ folders='folders=00-1111-xxx-xxx/
00-2222-yyy-yy/'
cmccabe@Satellite-M645:~/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123$ for f in "$DIR"/f1; do STRING=$(awk -v ref="/$S5/" 'match($0, ref) {print $2}' "$f"); done
+ for f in '"$DIR"/f1'
++ awk -v ref=/R_2019_00_00_00_00_00_xxxx_x0-00000-123/ 'match($0, ref) {print $2}' /home/cmccabe/folder/f1
+ STRING=
cmccabe@Satellite-M645:~/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123$ nl="
> "
+ nl='
'
cmccabe@Satellite-M645:~/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123$     if [[ $nl$folders$nl == *$n$STRING/$nl* ]] || continue # only execute file on match
>        else mkdir $STRING 
bash: syntax error near unexpected token `else'
cmccabe@Satellite-M645:~/folder/R_2019_00_00_00_00_00_xxxx_x0-00000-123$     fi

# 6  
Old 09-30-2019
The syntax requires then between if and else.
And what does the for loop with STRING?
You probably want
Code:
## added newline boundry seperator
nl="
"
for f in "$DIR"/f1
do
    STRING=$(awk -v ref="/$S5/" 'match($0, ref) {print $2}' "$f")
    if [[ $nl$folders$nl != *$n$STRING/$nl* ]] ## only execute file on match
    then
       mkdir $STRING 
    fi
done

Note that in [[ ]] the == operator is a glob match, and != is the opposite (no match). Some shells allow = instead of == but == is the standard in [[ ]].
(While in [ ] the = operator is true if the strings are equal, and the != is the opposite (not equal).) Some shells allow == instead of = but = is the standard in [ ].
Last edited by MadeInGermany; 09-30-2019 at 07:37 AM..
These 2 Users Gave Thanks to MadeInGermany For This Post:
cmccabe KD_999
# 7  
Old 09-30-2019
set x showed the same as the previous post, I initially thought there may be multiple files so used a for, but quickly realized that wasn't going to work. Is there a better approach? Thank you Smilie.

Code:
for f in "$DIR"/all ; do
>     STRING=$(awk -v ref="/$S5/" 'match($0, ref) {print $2}' "$f")
>     if [[ $nl$folders$nl != *$n$STRING/$nl* ]] ## only execute file on match
>     then
>        mkdir -p $STRING 
>     fi
> done
+ for f in '"$DIR"/f1'
++ awk -v ref=/R_2019_00_00_00_00_00_xxxx_x0-00000-123/ 'match($0, ref) {print $2}' /home/cmccabe/Desktop/new/f1
+ STRING=
+ [[ 

 != */
* ]]
+ mkdir -p

This User Gave Thanks to cmccabe For This Post:
KD_999
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create directory and sub-directory with awk and bash

In the below I am trying to create a parent directory using the R_2019 line from f1 if what above it is not empty. I then create sub-directories under each parent if there is a match between $2 of f1 and $2. Inside each sub-folder the matching paths in $3 and $4 in f2are printed. If there is no... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Create automated scan of specific directory using bash

I am trying to use bash to automate the scan of a specific directory using clamav. Having this in place is a network requirement. The below is an attempt to: 1. count the extensions (.txt, .jpeg) in a directory and write them to a virus-scan.log (section in bold) 2. scan each folder in the... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. Shell Programming and Scripting

Bash to select oldest folder in directory and write to log

In the bash below the oldest folder in a directory is selected. If there are 3folders in the directory /home/cmccabe/Desktop/NGS/test and nothing is done to them (ie. no files deleted, renamed) then the bash correctly identifies f1 as the oldest. However, if something is done to the folder then... (4 Replies)
Discussion started by: cmccabe
4 Replies

4. Shell Programming and Scripting

Bash to add folder to exsisting folders in directory

I am trying to create subdirectories in each folder in /home/cmccabe/Desktop/NGS/test/*. When I use echo I can see each folder in the directory, but I can not seem to add the commented out portion in bold. These are the sub-directories and sub-folders I am trying to add to each folder in... (1 Reply)
Discussion started by: cmccabe
1 Replies

5. Shell Programming and Scripting

Bash to select oldest folder in directory automatically and log process

The `bash` below uses the oldest folder in the specified directory and logs it. The goes though an analysis process and creates a log. My problem is that if there are 3 folders in the directory folder1,folder2,folder3, the bash is using folder2 for the analysis eventhough folder1 is the oldest... (0 Replies)
Discussion started by: cmccabe
0 Replies

6. Shell Programming and Scripting

Create a folder under different user directory

Hello All, I have to write a shell script and use it in informatica. The script has to perform below actions: The script gets executed from edw user. Through the script, a DT folder has to be created under edw_sca user. Is this scenario possible through a SHELL script or not. ... (2 Replies)
Discussion started by: bghosh
2 Replies

7. Shell Programming and Scripting

how to check if a directory exists or not.if not need to create it

Hi, I am using solaris 10 OS and bash shell.just checking for small code snippet which follows below. /export/home/vomappservers/spa/common/5.0 /export/home/vomappservers/spa/common/scripts /export/home/vomappservers/spa/tools /export/home/vomappservers/spa/scm5.0/SCCS... (5 Replies)
Discussion started by: muraliinfy04
5 Replies

8. Shell Programming and Scripting

check if multiple directories exist else create missing directories

Hi , I 'm trying to check if multiple directories exist on a server, if not create the missing ones and print " creating missing directory. how to write this in a simple script, I have made my code complex if ; then taskStatus="Schema extract directory exists, checking if SQL,Count and... (7 Replies)
Discussion started by: ramky79
7 Replies

9. Shell Programming and Scripting

To check whether a directory is exist and if it is not, create it

Hi, I want to write a shell script to check whether a directory (say A) is existing in a given location and if it is not, create it. (3 Replies)
Discussion started by: sabya
3 Replies

10. Shell Programming and Scripting

Use awk to create new folder in current directory

Alright, I am sure this is a laughable question, but I don't know so I am going to ask anyway. I have a little script I am writing to take information from one source, recode it in a certain way, and print to files for each subject I have data for. This all works perfectly. I just want to put... (6 Replies)
Discussion started by: ccox85
6 Replies
Login or Register to Ask a Question