Copy files based on specific word in a file name & its extension and putting it in required location


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copy files based on specific word in a file name & its extension and putting it in required location
# 8  
Old 10-21-2016
There can be multiple files, and the structure of the filename can be different but it has hyr in the filename. So if filename contains hyr word and it has .md and .db extension; it will move to TUM/HYR folder; if extension is .txt it should convert using dd command to edcdic and move it to TUM/HYR
if filename contains par word and it has .md and .db extension; it will move to TUM/par folder; if extension is .txt it should convert using dd command to edcdic and move it to TUM/PAR
if filename contains mar word and it has .md and .db extension; it will move to TUM/mar folder; if extension is .txt it should convert using dd command to edcdic and move it to TUM/MAR..
Can anyone help me with the entire script so that I can test it...

One important thing I missed is; all these files and folders are in Hadoop HDFS.

Main folder is MUM
SUBFOLDERS ARE HYR PAR MAR... there can be multiple such subfolders
But the subfolders and the files in it has a match with filename,like if HYR is a subfolder it will hv a file name which has hyr_20162001.md and db in it, so the script can pick word hyr from filename.

So script will read main MUM folder and the all the subfolders one by one and copy file into out directory which is :-
TUM/HYR for hyr files with .md .d
TUM/PAR for par files with .md .db
TUM/MAR for mar files with .md .db
TUM/SAR for sar files with .md .db
remember there can be other subfolders as well but it has a standard like lets assume I hv other subfolders called ABC so the file in it will be abc_20162019.db and md and it will be copied into TUM/MUM folder..

Note all these files and folders are in Hadoop so it will be Hadoop commands. Not sure if above unix command like find and all will work or not
# 9  
Old 10-22-2016
Just repeating the info given in post#1 doesn't really help. Info on the contents of directories might be extremely beneficial, like will there be "hyr" files in the "PAR" subdir, and e.g. a (partial) directory listing.
Adding lately that *nix commands probably don't work is, hmmm, debatable.
# 10  
Old 10-22-2016
Apologies RudiC, to answer to ur question.
hyr file name will be there in HYR subfolder, par filename will be in PAR subfolder and so on... I'm not sure about unix command will work or not... Actually these main folder MUM and it's subfolders N files are in Hadoop and I want to copy these to destination folders as mentioned in my earlier post...
Hope that clarifies...
# 11  
Old 10-23-2016
Hello all,
A quick solution will let me move ahead...
Do share a script for further analysis
# 12  
Old 10-23-2016
Quote:
Originally Posted by prajaktaraut
Hello all,
A quick solution will let me move ahead...
Do share a script for further analysis
The UNIX & Linux Forums is here to help you learn how to write your own code; the volunteers here are not your unpaid programming staff.

RudiC has given you code that you should be able to mold into something that will meet your requirements. RudiC also asked questions that you have not answered that would provide information that would greatly simplify code that I would write if I were trying to solve your problem. You were asked to show us the output of ls (which would clearly demonstrate the format of your filenames). You have shown us one relative pathname (TUM/ABC/abc_20162019.db), if all of the filenames you want to process follow this filename format (i.e., 3 lowercase letters; <underscore>; 6 decimal digits; and one of the three filename extensions: .db, .md, and .txt), then code could be significantly simplified. If we knew that there are no files in those directories that do not conform to the above filename format, code could be simpler still.

If you would like our help to complete your project, please answer the questions that have been asked, show us the code you have tried, show us what is working, and explain clearly what still needs to be done.
This User Gave Thanks to Don Cragun For This Post:
# 13  
Old 10-23-2016
Apology Don Cragun and RudiC. But I did replied to the questions raised by RudiC. File name will contain project_abc_20162012.db & md and .txt as well.
So again if I would reiterate, the main Folder is MUM and the subfolder is ABC anf this subfolder u will hv 3 or more files and these files will be copied to output folder which is TUM/ABC by searching abc word from the filename and .db & .md in it.
This folder can contain other files as well but they will stay there in the same folder only the file which matches the above criteria will move to output folders.
As of now I'm copying these files manually...
I search files in MUM/ABC folder which contains abc word in filename and which has .md and .db extension and copy those to TUM/ABC folders.
Same process I do it for other subfolders par sar mar.
Key point- the folder name has the relavant filenaming convention files in it.like PAR subfolder will hv project_par_20162010.db .md and .txt files in it which has to be moved to TUM/ PAR destination folder.
Iam hoping this clarifies..

Most important thing- all these files and folders are in Hadoop HDFS.

Sincere apology for the trouble...
# 14  
Old 10-24-2016
Try to replace for i in MUM/*/* in post#7 with
Code:
for i in MUM/*/*.{db,md,txt}

and post where this approach doesn't meet your needs. This is all I can (and will) do until you stop repeating the well known info and give the necessary ones.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete all files of a particular name & extension except one file.

I wish to delete all files that starts with "body<any number of digits>.xml" except body65.xml on Linux 7 bash shell So, from the below files body64.xml body.sh body65.xml body655.xml body565.xml body66.xml hello65.xml My command should delete all files except the below. body.sh... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. Shell Programming and Scripting

Find and Copy file of specific location

Dear All, I need to transfer all files present in one location to another but those files should be of specific extension like. Find and copy all files of extension .xls, .pdf, .txt from location usr/tmp to location /per/Treat (6 Replies)
Discussion started by: yadavricky
6 Replies

3. UNIX for Dummies Questions & Answers

Display the .csv extension files based on .done extension fine

Hi All, I want to fetch the files based on .done file and display the .csv files and Wil take .csv files for processing. 1.I need to display the .done files from the directory. 2.next i need to search for the .Csv files based on .done file.then move .csv files for the one directory ... (2 Replies)
Discussion started by: girija.g6
2 Replies

4. Shell Programming and Scripting

How to copy files from one location to another based on a priority?

Hi Gurus, I am a newbie to shell scripting and I am facing a problem right now.I have to automate the copy of files based on a priority.The scenario is as below: 1) There will be files from Mon-Fri with Mon file being named as abc_def_01_YYYYMMDD and Tue file being abc_def_02_YYYYMMDD and so... (4 Replies)
Discussion started by: vikramgk9
4 Replies

5. Shell Programming and Scripting

Deleting lines in a fixed length file where there is a word at specific location

I have a big file having 100 K lines. I have to read each line and see at 356 character position whethere there is a word "W" in it. If it is their then don't delete the line otherwise delete it. There are two lines as one Header and one trailer which should remain same. Can somebody... (5 Replies)
Discussion started by: mohit kanoongo
5 Replies

6. Shell Programming and Scripting

How to find a word and move it a specific location in xml file using perl?

Hi friends, I have one XML file having below structure :- INput XML file :- <?xml version="1.0" encoding="UTF-8"?> <START> <A=value1> <attr name1="a1"> </A> <B=value2> <attr name2="b1"> <attr name3="c1"> </B> </START> output xml file should be === (3 Replies)
Discussion started by: harpal singh
3 Replies

7. Shell Programming and Scripting

Using sed to replace a word at specific location

I'm try to change a the prohibit to aix for the lines starting with ssh and emagent and rest should be the same. Can anyone please suggest me how to do that using a shell script or sed passwd account required /usr/lib/security/pam_prohibit passwd session required ... (13 Replies)
Discussion started by: pjeedu2247
13 Replies

8. UNIX for Dummies Questions & Answers

how to copy files and record original file location?

:EDIT: I think my post name should have been labeled: how to copy files and record original file location. not "retain". Hello, this is my first post! I searched the forums a lot before posting, but was unable to answer my question. Here's my problem: There are several hundred text files... (4 Replies)
Discussion started by: willie8605
4 Replies

9. Shell Programming and Scripting

Shell Script for Copy files from one location to another location

Create a script that copies files from one specified directory to another specified directory, in the order they were created in the original directory between specified times. Copy the files at a specified interval. (2 Replies)
Discussion started by: allways4u21
2 Replies

10. Shell Programming and Scripting

Bash copy file contents into an existing file at a specific location

Hi all I need to copy the entire contents of one file into an existing file at a specific location. I know the exact line number where I need to put it. It appears I would use either sed or awk to do this, but I have been unsuccessful so far: File A line 1 line 2 line 3 line 4 ... (6 Replies)
Discussion started by: gshepherd7
6 Replies
Login or Register to Ask a Question