Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

move first 1000 files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting move first 1000 files
# 1  
Old 08-17-2010
move first 1000 files
Hi i have a folder with 27,000 images. I need to process each image using imagemagick. I have got image magick working. I don't want to attempt to process all 27,000 images in one go and thought i would try doing it with 1000 images at a time. Is there some way i can list only the first 1000 entries and move them to a new folder for processing

Code:
cd /folder
for i in `ls -1`
do
    mv $i  ../temp/$i
done

That sort of thing but only do 1000
# 2  
Old 08-17-2010
Hi.

Try:

Code:
ls | head -1000 | xargs -I{} mv {} ../temp

# 3  
Old 08-17-2010
thanks ill give that a try
# 4  
Old 08-31-2010
Script to move first 1000 files
Hello ,

I have used the above command to write a script that can move the first thousand files.
However , the script is eating all the files including iteslf in the directory from where it is run.
Pls correct the below script. Thanks in advance.

Code:
#!/bin/bash
PATH=$PATH:/usr/bin:/usr/sbin
# Variables
LOGFILE=/tmp/r_bkp_log.out
U_PATH=/source/In7_backlog
# MAIN
while true
do
  DATE=$( date +"%a %d-%m-%Y %H:%M:%S" )
  UR=$( ls /source/In7_bkp| wc -l )
  if [ $UR -eq 0 ]
  then
    /usr/bin/cd $U_PATH
    /usr/bin/echo /usr/bin/pwd
    ls | head -1000 | xargs -I{} mv {} ../In7_bkp
    echo "Moving next 1000 files " >> $LOGFILE
  fi
  /usr/bin/echo "" >> $LOGFILE
  /usr/bin/echo "$DATE: loader:\t$UR" >> $LOGFILE
  sleep 60
done

Last edited by Scott; 08-31-2010 at 02:44 PM.. Reason: Please use code tags (and indent your code)
# 5  
Old 08-31-2010
Hi.

It's hard to say what exactly your script is doing.

It's representing paths using an odd mix of variables, hard-coded absolute paths and hard-coded relative paths - all seemingly to the same location.

If the directory you are running the script from contains the files you are moving, then either qualify what files you are moving to exclude the script, using ls, or using grep -v to exclude it.
# 6  
Old 08-31-2010
Actually , I am trying run this script from a /source/tools where I have lot of other scripts available.

Now the above said script needs to check one dir called /source/In7_bkp and see if the count there is zero.

Incase it is then it needs to move files from /source/In7_backlog to the above said directory location.

There is a process that handles the files from In7_bkp. So I am trying check the count every 1 min and see if the process had finished the 1000 files that we put.
Incase it does then , next 1000 files are moved.

However, this script has eaten all the files in tools dir as well.
I dunno if I implemented the logic properly for my requirement.
# 7  
Old 08-31-2010
Hi.

I made a couple of changes to your script, and hope it helps Smilie

Code:
#!/bin/bash
# Variables

LOGFILE=/tmp/r_bkp_log.out
M_PATH=/source               # Main directory
S_PATH=$M_PATH/In7_backlog   # Source directory
T_PATH=$M_PATH/In7_bkp       # Target directory

cd $S_PATH

# MAIN
while true
do
  DATE=$( date +"%a %d-%m-%Y %H:%M:%S" )
  UR=$( ls $T_PATH | wc -l )
  if [ $UR -eq 0 ]
  then
    echo $PWD   # I didn't know what "/usr/bin/echo /usr/bin/pwd" was meant to be doing!
    ls | head -1000 | xargs -I{} mv {} $T_PATH
    echo "Moving next 1000 files " >> $LOGFILE
  fi
  echo >> $LOGFILE
  echo "$DATE: loader:\t$UR" >> $LOGFILE
  sleep 60
done

This User Gave Thanks to Scott For This Post:
rmv
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

2. Shell Programming and Scripting

Creating a sequence of numbers in a line for 1000 files

Hi, I try to explain my problem , I have a file like this: aasdsaffsc23 scdsfsddvf46567 mionome0001.pdb asdsdvcxvds dsfdvcvc2324w What I need to do is to create 1000 files in which myname line listing a sequence of numbers from 0001 to 1000. So I want to have : nomefile0001.txt that must... (10 Replies)
Discussion started by: danyz84
10 Replies

3. UNIX for Dummies Questions & Answers

Split files into smaller ones with 1000 hierarchies in a single file.

input file: AD,00,--,---,---,---,---,---,---,--,--,--- AM,000,---,---,---,---,---,--- AR, ,---,--,---,--- AA,---,---,---,--- AT,--- AU,---,---,--- AS,---,--- AP,---,---,--- AI,--- AD,00,---,---,---, ,---,---,---,---,---,--- AM,000,---,---,--- AR,... (6 Replies)
Discussion started by: kcdg859
6 Replies

4. UNIX for Dummies Questions & Answers

Remove files from tar archive which are more than 1000 days old.

I am not able to extract/remove files older than 1000 days from a tar archive in linux system. #!/usr/bin/perl @file_list = `find /home/x/tmp/ -name *xxMsg* -ctime +7`; $file_name = '/home/x/tmp/new_archive.tar'; for... (1 Reply)
Discussion started by: DannyV
1 Replies

5. Shell Programming and Scripting

Move all files except sys date (today) files in Solaris 10

I want to move all files from one directory to another directory excluding today (sysdate files) on daily basis. file name is in pattern file_2013031801, file_2013031802 etc (2 Replies)
Discussion started by: khattak
2 Replies

6. Shell Programming and Scripting

How do you check for a particular string in 1000's of files?

Please forgive my ignorance on scripting. I am trying to determine (via a script) if a certain string of characters is present . The string that I am looking for is a constant length. Here is the string I am searching for: Model_Type={t}, ModelName={m} I need to determine if the above... (2 Replies)
Discussion started by: dlundwall
2 Replies

7. Shell Programming and Scripting

Adding a line to 1000's of files right after x amt of characters.

I am trying to add a single line of text to every file in a particular folder. There are thousands of files in the folder. Each file contains this same start of the first line: {d "%w- %d %m-, %Y - %T"} <some message here> with the rest of the text following the second curly bracket... (10 Replies)
Discussion started by: dlundwall
10 Replies

8. Shell Programming and Scripting

Script to find the string contain in which file 1000 files.

Hi Greetings i have 1000 files (xmlfiles) and i need to find which file contain the string over 1000 file all file end with txt i tried with grep -c "string" *.txt i am getting an error -bash: /bin/egrep: Argument list too long i have put an script like below #!/bin/bash for line... (9 Replies)
Discussion started by: venikathir
9 Replies

9. UNIX for Dummies Questions & Answers

Move same files and issue ls -al command on remaining files

I know I can use an ls -l junk1 command to get a listing of all files in the directory junk1, but I was wondering how I'd go about going through the files in junk1 in a for-in loop and issuing the ls -l command on them one by one. This is what I have so far: for file in $(ls -a $1) do ls... (1 Reply)
Discussion started by: Trinimini
1 Replies

10. Shell Programming and Scripting

Creating 1000 files using a script

I would like to write a script that would create 1000 files like clm0001.txt to clm1000.txt. All the files would contain the same contents. How do I achieve this using a script?Please help. (2 Replies)
Discussion started by: ggayathri
2 Replies
Login or Register to Ask a Question