Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Transferring files to directories

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transferring files to directories
# 1  
Old 02-06-2018
Transferring files to directories
I have a large number of files with file names of the format

Code:
iv.epoz.hhe.d.2018.028.000000.sac
iv.epoz.hhn.d.2018.028.000000.sac
iv.epoz.hhz.d.2018.028.000000.sac
iv.epoz.hhe.d.2018.029.000000.sac
iv.epoz.hhn.d.2018.029.000000.sac
iv.epoz.hhz.d.2018.029.000000.sac
mn.clta.hhe.d.2018.030.000000.sac
mn.clta.hhn.d.2018.030.000000.sac
mn.clta.hhz.d.2018.030.000000.sac
etc

The file names are set up such that d.2018.030 is the 30th day of 2018

I want to have a folder of each day in the format d.year.day such
as for example d.2018.028 and transfer the files there. And so on
for each day.
# 2  
Old 02-06-2018
Any attempts / ideas / thoughts from your side? With more than 900 posts in these fora we'd suppose you had at least a faint vision of the way to go...?
# 3  
Old 02-06-2018
Expanding on what RudiC has already said...

With over 900 posts, you know the standard questions for every thread in this forum...
  1. What operating system are you using?
  2. What shell are you using?
  3. What have you tried to solve this problem on your own?
And a few more questions related to this thread...
  1. Are all of the files to be moved located in a single directory?
  2. If not, in what directories are they located?
  3. When you say you have a large number of files, are there so many that if you run the commands year=2018, day=030, and then ls -1 *.d.$year.$day.*.sac that you get an error from your shell saying something like "argument list too long"? Note that you should try this with year and day values for the day with the largest number of files you'll need to process.
# 4  
Old 02-06-2018
Code:
fnames=`ls *.sac`

for f in $fnames; do
  day=`echo "$f" | grep -oP 'd+.[\d]+.[\d]+'`
  echo "file: $f"
  echo "day: $day"
  ftyp=`echo "${f:0:3}"`

  # Do not remake directory if it already exists 
  # Skip pzs files
  if [[ ! -e $day ]] && [ $ftyp != "pzs" ]; then
    mkdir $day
    echo "Created directory $day"
  elif [[ ! -d $day ]]; then
    echo "$dir already exists but is not a directory"
  fi

  # Copies file to station directory 
  if [ $ftyp != "pzs" ]; then
    cp $f $day
  else
    echo "Detected pzs file"
  fi
  echo "" 

done

Have done something as above. Seems to work
# 5  
Old 02-06-2018
Quote:
Originally Posted by kristinu
Code:
fnames=`ls *.sac`

for f in $fnames; do
  day=`echo "$f" | grep -oP 'd+.[\d]+.[\d]+'`
  echo "file: $f"
  echo "day: $day"
  ftyp=`echo "${f:0:3}"`

  # Do not remake directory if it already exists 
  # Skip pzs files
  if [[ ! -e $day ]] && [ $ftyp != "pzs" ]; then
    mkdir $day
    echo "Created directory $day"
  elif [[ ! -d $day ]]; then
    echo "$dir already exists but is not a directory"
  fi

  # Copies file to station directory 
  if [ $ftyp != "pzs" ]; then
    cp $f $day
  else
    echo "Detected pzs file"
  fi
  echo "" 

done

Have done something as above. Seems to work
Note that if you would change the code marked in red above to the following:
Code:
for f in *.sac; do
  ... ... ...
  ftyp=${f:0:3}

your code would use fewer system resources, run faster, and produce exactly the same results.

We could probably make several other suggestions that might help you improve the performance of your script. But since you refuse to tell us what operating system and shell you're using, there is no reason for us to waste our time making guesses that might not work in your environment.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

Rcp error while transferring files

i am trying to use a rsh script that uses the rcp command to copy files from server to server but it always produce the following error: HOSTNAME: protocol failure due to unexpected closure from server end Error: failed to copy /test to HOST2 system Error: failed to execute command on remote... (5 Replies)
Discussion started by: Portabello
5 Replies

2. Shell Programming and Scripting

Script for transferring files

Hi Guys, I have to transfer a few files in my system . The commands to be used are as follows . Will it be possible to send the output of the following in the form of a mail . cd /export/home/teja ls -lrt Quote.java* mv Quote.java Quote.java.20121023 cp /tmp/Quote.java . ls -lrt... (2 Replies)
Discussion started by: Ravi_Teja
2 Replies

3. UNIX for Dummies Questions & Answers

Transferring files from one linux server into another

Hello , I want to transfer files from one linux server into another , I got it working using SCP command , but I have to type in password for each and every file . All the remote severs have the same password , so is there a way that I can transfer all these files by typing my password only once ? (5 Replies)
Discussion started by: RaviTej
5 Replies

4. Shell Programming and Scripting

Transferring Files via ssh (bash)

I am trying to transfer my website using ssh(hostbasedauthentication) using: sudo ssh -o "PasswordAuthentication no" -o "HostbasedAuthentication yes" -l testuser 192.168.3.1 "find /var/www/vhosts/mywebsite.com -depth | grep -f include| cpio -oavc| gzip" > $backup_dir/fullwwwsite$date.cpio.gz ... (5 Replies)
Discussion started by: metallica1973
5 Replies

5. UNIX for Dummies Questions & Answers

Mget issue: not transferring more than 2 files

I am trying to transfer about 3000 files from a window platfrom to my linux server. the scrip is pretty simple ftp to the source server (windows) cd to source directory lcd to destination directory mget *.jpg what I get in the log file is: mget 07421001.jpg? 200 PORT command successful.... (4 Replies)
Discussion started by: Doveman
4 Replies

6. AIX

Transferring files via RS232 cable

Current setup: Some old AIX standalone machine (F series) My WinXP laptop Null modem cable from laptop to AIX machine (console port). On Hypterminal on WinXP I can connect to the AIX machine (serial connection), but when I try to transfer file, it will just sit there. I'm guessing the... (3 Replies)
Discussion started by: apra143
3 Replies

7. AIX

Transferring files between Windows and AIX

How do I transfer Plain Text and/or BMP image files between my WindowsXP PC and my AIX 4.1 PowerPC? I have no network or USB options, just a Floppy disk drive and a CD drive on each machine. Is it possible at all? Any help would be gratefully received:) (7 Replies)
Discussion started by: Pennant Man
7 Replies

8. Shell Programming and Scripting

FTP - To handle error while transferring files

Hi, I had written an FTP script where in I loop through the directories and transfer the files from each and every directory of Windows to UNIX. Now the problem is when 1. The connection is unable to be established I should return some error codes 2. When there is some system... (1 Reply)
Discussion started by: mahalakshmi
1 Replies

9. UNIX for Dummies Questions & Answers

Transferring hidden files with scp

Is there a way to scp hidden files only ( with regex ) without specifying the name of the hidden files ? Many thanks in advance ! :) (13 Replies)
Discussion started by: matrixmadhan
13 Replies

10. UNIX for Dummies Questions & Answers

transferring files to and from remote computer

Hi all, i first have to ssh into my university account and then through there another ssh into my office computer, from my home computer. I have been trying to transfer files to and from with no such luck. How do i send a file from home to my office computer. Do I have to send it to my... (6 Replies)
Discussion started by: yogi1
6 Replies
Login or Register to Ask a Question