File copy script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File copy script
# 1  
Old 09-24-2013
File copy script

Looking for help on a script that does the following. Reads a text file with ~5000 lines that look something like this:

Code:
/folder/path/path2/pathX/file.lua,d/1.2699.gz
/folder/path/path4/pathX/file2.lua,d/1.2699.gz
/folder/path/path2/pathX/data/file3.lua,d/1.2699.gz

And copies the file from the following path (/mnt/cifs appended) to the above path:
Code:
/mnt/cifs/folder/path/path2/pathX/file.lua,d/1.2699.gz
/mnt/cifs/folder/path/path4/pathX/file2.lua,d/1.2699.gz
/mnt/cifs/folder/path/path2/pathX/data/file3.lua,d/1.2699.gz

In each of these, the following are 0755 folders that may or may not exist:
Code:
/folder/path/path2/pathX/file.lua,d/
/folder/path/path4/pathX/file2.lua,d/
/folder/path/path2/pathX/data/file3.lua,d/

And each copied file should be chown’d user:group.

I have a script like working for a specific file value (e.g. 1.2699.gz across multiple directories), but wanted to get this working across multiple files in multiple directories read from a text file.

Thanks!

Last edited by Scott; 09-24-2013 at 10:42 PM.. Reason: Please use code tags
# 2  
Old 09-25-2013
Code:
xargs -n1 -I% sh -c 'test -f % && mv /mnt/cifs/% %' < infile

--ahamed
# 3  
Old 09-25-2013
So what is your script that is working for a specific file?

What shell are you using?

The script ahamed101 provided may work for you, but it won't create missing target directories and it won't set the owner and group of the destination files correctly if the desired owner and group don't match the effective UID and GID of the process running the script (assuming you change the mv to cp).

Last edited by Don Cragun; 09-25-2013 at 01:27 AM.. Reason: Note additional difference...
# 4  
Old 09-25-2013
Quote:
Originally Posted by Don Cragun
So what is your script that is working for a specific file?

What shell are you using?

The script ahamed101 provided may work for you, but it won't create missing target directories and it won't set the owner and group of the destination files correctly if the desired owner and group don't match the effective UID and GID of the process running the script (assuming you change the mv to cp).
Oops!
Yes, I didn't notice the requirement of creating the missing directories.

--ahamed
# 5  
Old 09-25-2013
Quote:
Originally Posted by Don Cragun
So what is your script that is working for a specific file?

What shell are you using?

The script ahamed101 provided may work for you, but it won't create missing target directories and it won't set the owner and group of the destination files correctly if the desired owner and group don't match the effective UID and GID of the process running the script (assuming you change the mv to cp).
I take the file missing error log, parse it down to a simple list of files called cleanerror.txt. And run a script that looks like this to restore the files:

Code:
#!/bin/bash

file='1.2699.gz'
files=`cat cleanerror.txt`
filesdir=`sed -e 's/$file//g' cleanerror.txt`

for i in $filesdir; do
mkdir -p $i
done

for i in $files; do
cp "/mnt/cifs/$i" $i
chown user:group $i
done

Using bash (running CentOS6) but can use Perl. Wrote this just now, it seems to do what I need with some file tests even if it feels hacky (only using Perl for getting the file/directory so I don't have to sed/awk, shelling out for almost everything else).

Code:
#!/usr/bin/perl

use File::Basename;
use File::Copy;

$result = open (IN, "file.txt");
while (defined ($dataLine = <IN>)) {
        chomp ($dataLine);
        my $dir = dirname($dataLine);
        system("mkdir -p $dir");
        system("chown user:group $dir");
        my $oldPath = "/mnt/cifs" . $dataLine;
        copy($oldPath,$dataLine);
        system("chown user:group $dataLine");
        system("chmod 644 $dataLine");
}


Last edited by kwalan; 09-25-2013 at 05:04 AM..
# 6  
Old 09-25-2013
Do you have the pax utility on CentOS6?

Do you really want all copied files to all be owned by a single user and group, or do you want to copy the user and group settings from each of the source files to the corresponding destination files?

If pax is available, you're running as root, and you'd like to copy the owner, group, mode, and timestamp settings of the source files to your destination files, I would think that something like:
Code:
sed 's|^|/mnt/cifs|' < cleanerror.txt | pax -rw -p e -s '|^/mnt/cifs||' /.

should do what you want. (NOTE that I have not tested this command line on my system when running with root privileges. Without root privileges, I get warnings that I can't change the ownership of a file [even when the requested change is a no-op] when I try this on OS X.)
# 7  
Old 09-25-2013
Don't have the pax utility.

Yeah, the destination directories and files are all owned by a single user and group (non-root service account for the application). The source is a CIFS-mounted network drive from the old server and could be mounted as the non-root service account's UID, but I'm okay with having the implicitness of chown'ing in the script because I'm only concerned with how the files end up rather than what permission state they're coming from.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script for File Copy

Hello All, This is my first post to unix.com. I have requirement to copy multiple *.dmp files from one server to other server. I have that other server NFS filesystem mapped to source server. Dump files are so huge almost 20TB. There are around 15-20 files each of 1.1TB. I want to copy these... (6 Replies)
Discussion started by: Amit Bansode
6 Replies

2. Shell Programming and Scripting

Help with file copy script

Hello, I am still pretty new at writing shell scripts and may have bitten off more than I can chew. I am trying to write a script that figures out if a file exists (a known name) in directory a, then copy it to my selected destination, otherwise, look in directory b and check there. If it is in... (14 Replies)
Discussion started by: r34lj4k3
14 Replies

3. Shell Programming and Scripting

Shell script to copy file

Dear all, I have a database with thousands of files with the structure of name is: Filename_hour_year.abc Filename_hour_year_1.abc .............. So what I need is how to write a script that all file with contain the character "_1" will copy to "_2" For example: file name:... (7 Replies)
Discussion started by: hainguyen1402
7 Replies

4. UNIX for Dummies Questions & Answers

Script to copy file small interval file

HI Guys whenever i post a transaction , abcd.in file is stored in the temp and vanishes when the transaction is compleated . the abcd.in file stays for 1 sec in temp is der any solution to capture the abcd.in file to another directory cp -r /tmp/abcd.in /tmp/smith.in when i used the... (1 Reply)
Discussion started by: kalyankalyan
1 Replies

5. Shell Programming and Scripting

Match File and Copy File Script (Homework, Closed)

Can you please help on this? I am looking for the shell script which does following:- step 1: It should open the file /u/manish/input/FileIndex.dat and read line by line step 2: Once first line is read (for ex: File1), we have to find a file, that contains this matching... (4 Replies)
Discussion started by: teteguru1
4 Replies

6. UNIX for Dummies Questions & Answers

Shell script to search for text in a file and copy file

Compete noob question.... I need a script to search through a directory and find files containing text string abcde1234 for example and then copy that file with that text string to another directory help please :eek: (9 Replies)
Discussion started by: imeadows
9 Replies

7. Shell Programming and Scripting

copy file from script file to remote computer

hi, i want copy one or group of file from a computer to others, but i have some problem that do not allow me to do this. i do this by scp command like this : scp <file name> root@cpName:destinationAddress but the problem is that it do not it automatically. it means when it is connecting to... (4 Replies)
Discussion started by: MShirzadi
4 Replies

8. Shell Programming and Scripting

Script to capture new lines in a file and copy it to new file

Hi All, I have a file that gives me new line/output every 5 minutes. I need to create a script that capture new line/output besides "IN CRON_STATUS", in this case the new output is "begin ------ cron_status.sh - -----------". I want this script to capture the line starting from "begin ------... (0 Replies)
Discussion started by: fara_aris
0 Replies

9. Shell Programming and Scripting

Script look for file ,sort and copy to other direcotry

Hi I am in a situation to write a shell script that looks for the flat files continuosly and if files exists in that directory sort on the files and get the latest file(File comes with timestamp at the end) and copy the latest file to the other directory and again copy the next one to the same... (0 Replies)
Discussion started by: reddi22
0 Replies

10. Shell Programming and Scripting

Help:Copy file from one to other using script

Hi Frineds, Through my DTS process I am generating one file at one directory like, /sqlsrvr_demo/dts/put/transaction_fact.csv now I want to copy this files(only when the files size is greater than 0 kb) to some other directory like /sqlsrvr_demo/dts/get/transaction_fact.csv Can... (2 Replies)
Discussion started by: sunnysunny
2 Replies
Login or Register to Ask a Question