Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Copying specific file types to specific folders

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copying specific file types to specific folders
# 1  
Old 03-05-2018
Copying specific file types to specific folders
I am trying to write a script that cycles through a folder containing many folders and when inside each one it's supposed to copy all the .fna.gz files to a folder elsewhere if the file and the respective folder have the same name.


Code:
for fldr in /home/playground/genomes/* ; do

    find . -name *fna.gz | while read f ; do

        f_name=$( echo $fldr | cut -d '/' -f 7 | cut -d '_' -f 1) ;

        #echo $f_name ;

        if [ "$f_name" == /home/playground/port/"$f_name" ]; then

        cp -r "$f" /home/playground/port/"$f_name"/ ;

        else

        continue    

        fi

    done

done

This script so far doesn't actually copy anything over at all.
# 2  
Old 03-05-2018
Your script doesn't copy anything because
Code:
f_name=$( echo $fldr | cut -d '/' -f 7 | cut -d '_' -f 1)

results in a null value, at least is does if you are using the expansion
Code:
/home/playground/genomes/*

Andrew
# 3  
Old 03-05-2018
Code:
f_name

is supposed to be the name, so it's used as a reference point because these files are something like "Long_John_Silver" but the folders into which they're supposed to be placed are like "Long".
# 4  
Old 03-05-2018
Also note, that this if statement will never be true:

Code:
 if [ "$f_name" == /home/playground/port/"$f_name" ]; then

Perhaps you meant:

Code:
 if [ "$f" == /home/playground/port/"$f_name" ]; then

Remember I showed you this construct: basenm=${fldr##*/} followed by f_name=${basenm%%_*} in this thread for extracting the filename proceeding _ from a full path.

This trick should server you well in this script to replace the echo + double cut command substitution.
# 5  
Old 03-06-2018
This is what I have now, but still no success.

Code:
 

cd /home/scripts/playground/genomes_2 ;

for fldr in /home/scripts/playground/genomes_2/* ; do

    find . -name *fna.gz | while read f ; do

        #f_name=$( echo $fldr | cut -d '/' -f 7 | cut -d '_' -f 1) ;

        basenm=${fldr##*/} ; f_name=${basenm%%_*} ; 

        #echo $f_name ;

        if [ "$f" == /home/scripts/playground/port/"$f_name" ] ; then

        cp $f /home/scripts/playground/port/"$f_name"/ ;

        else

        continue    

        fi

    done

done

# 6  
Old 03-06-2018
I'm losing the plot here. Could you please show an example or two of:
  1. $fldr;
  2. $f_name;
  3. $f
One of the problems I'm having is that if $fldr has values such as
  • long_john_silver
  • jack_of_lantern
  • nick_swift
then f_name will have values like
  • long
  • jack
  • nick
and yet you appear to be looking for values of $f_name that match $f and therefore end in fna.gz. That is not going to work. Perhaps if the line
Code:
if [ "$f" == /home/scripts/playground/port/"$f_name" ] ; then

was actually
Code:
if [ "${f%fna.gz}" == /home/scripts/playground/port/"$f_name" ] ; then

you may stand a better chance? But without looking at actual names you are trying to match we can't tell you why your code doesn't work.

Andrew
These 2 Users Gave Thanks to apmcd47 For This Post:
Mr_Keystrokes RudiC
# 7  
Old 03-06-2018
Solved...
I realised that I was copying all the files into all the folders based on the type of file i.e. .fna.gz, so I specified what type of fna.gz file I want to be read and then copied. There is no need for an if statement since the specificity is implied through the parameter expansion. It now works perfectly.

Code:
 
 
    for fldr in /home/scripts/playground/genomes_2/* ; do

    basenm=${fldr##*/} ; f_name=${basenm%%_*} ; 

    find . -name $f_name*fna.gz | while read f ; do

        cp -r $f /home/scripts/playground/port/$f_name/ ;

    

    done

    done
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Searching for file types by count in specific folder in RHEL 6

So I'm trying to search for the top 10 or 15 items under a directory by file type. I want to run a command on a directory and get something like the following: Example of expected output.. .PDF: 100, .txt: 95, .word: 80.. What would be the best way of going about this? I've searched around... (2 Replies)
Discussion started by: shackle101
2 Replies

2. Shell Programming and Scripting

Search for specific file type in subdirectory with multiple folders

I have a directory that is in the below order (the --- is not part of the directory tree, only there to help illustrate: DATE --- main level Folder1 --- level under DATE plugin_out --- level under Folder1 variantCaller_out.40 --- level under plugin_out 001,002,003 --- level under... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

Bash to move specific files from folders in find file

I have a directory /home/cmccabe/nfs/exportedReports that contains multiple folders in it. The find writes the name of each folder to out.txt. A new directory is then created in a new location /home/cmccabe/Desktop/NGS/API, named with the date. What I am trying to do, unsuccessfully at the moment,... (7 Replies)
Discussion started by: cmccabe
7 Replies

4. Shell Programming and Scripting

Extract specific line in an html file starting and ending with specific pattern to a text file

Hi This is my first post and I'm just a beginner. So please be nice to me. I have a couple of html files where a pattern beginning with "http://www.site.com" and ending with "/resource.dat" is present on every 241st line. How do I extract this to a new text file? I have tried sed -n 241,241p... (13 Replies)
Discussion started by: dejavo
13 Replies

5. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

6. Shell Programming and Scripting

split file content into specific folders

Hi I have a large text file and I want to split its content into multiple flies. this large file contains several blocks of codes separated by a comment line for each block. this comment line represents a directory path So, when separate these blocks each into a separate file, This output... (7 Replies)
Discussion started by: turki_00
7 Replies

7. UNIX for Dummies Questions & Answers

Help with copying specific parts of a file structure

Hello. I need help with copying part of a file structure to another directory while still keeping the structure. For example look below: ../folder1/sub1/txt.txt ../folder1/sub2/pic.png ../folder2/sub1/pic.png ../folder2/sub2/txt.txt So in this I would like to copy only the directories and... (3 Replies)
Discussion started by: the
3 Replies

8. Shell Programming and Scripting

Assigning a specific format to a specific column in a text file using awk and printf

Hi, I have the following text file: 8 T1mapping_flip02 ok 128 108 30 1 665000-000008-000001.dcm 9 T1mapping_flip05 ok 128 108 30 1 665000-000009-000001.dcm 10 T1mapping_flip10 ok 128 108 30 1 665000-000010-000001.dcm 11 T1mapping_flip15 ok 128 108 30... (2 Replies)
Discussion started by: goodbenito
2 Replies

9. Shell Programming and Scripting

Copying specific files from remote m/c to specific folders

Hi All, I am trying to rsync some of the latest files from remote m/c to my local linux box. Folder structure in my remote m/c looks like this /pub/Nightly/Package/ROLL/WIN /pub/Nightly/Package/SOLL/sol /pub/Nightly/Package/SOLL/linux Each of the folder contains gzip files which on daily... (0 Replies)
Discussion started by: jhoomsharabi
0 Replies

10. UNIX for Dummies Questions & Answers

How do I grep in specific file types?

I have a directory with file types ending .log, .mml, .gll, .dll . How can I grep expressions only in say the .log files? (3 Replies)
Discussion started by: bbbngowc
3 Replies
Login or Register to Ask a Question