Ordering batch number


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ordering batch number
# 1  
Old 08-13-2013
Ordering batch number

Hi,

Could some one please help to order the batch number in sequence.

I will be getting bunch of files with batch number in folder1 which are not in sequence.

I need to move all files from folder1 to folder2 with batch number in sequence.

Header record looks like
Code:
 PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000013

example.
Code:
/tmp/folder1
1
2
4
3
5
6
8
9
7
10

Code:
/tmp/folder2
1
2
3
4
5
6
7
8
9
10

I need to run this for every few hours.
# 2  
Old 08-13-2013
The following does what you want, if you need to compare the DATE field as well it will obviously require modification.

Code:
#!/usr/bin/perl

use strict;
use warnings;
use File::Copy;
my %batch;
opendir(my $src, "/tmp/folder1")|| die "Could not open /tmp/folder1: $!\n";
while (my $f=readdir($src)){
    if ($f !~/^\.+/){
          open (my $transaction , "<", "/tmp/folder1/$f")|| die "Could not open file \"/tmp/folder1/$f: $!\n";
          my $head=readline($transaction);
          close($transaction);
          $batch{$f}=$1 if ($head=~/BATCHNO=(\d+)/);
    }
}
my $index=0;
for my $file (sort {$batch{$a}<=>$batch{$b}} keys %batch){
    copy("/tmp/folder1/$file", "/tmp/folder2/$index");
    $index++;
}

# 3  
Old 08-13-2013
Thanks for the perl script. But I am looking for shell programming.
# 4  
Old 08-13-2013
You could start with

Code:
grep BATCHNO /tmp/folder1/* | sort -t"=" -nk 4

and then pick the filename in this ordered list to move to folder2.
# 5  
Old 08-13-2013
Was in a Perl frame of mind, something straightforward like this?

Code:
 for j in $(for i in /tmp//folder1/* ; do echo $i,$(head -1 $i| sed "s/.*BATCHNO=//");done | sort -k2 -t, | cut -d, -f1) ; do cp $j /tmp/folder2/$k; k=$(( $k + 1 )); done

# 6  
Old 08-13-2013
Thanks both. Above command correctly searching for BATCHNO and sorting in order, but when coping the files to folder2 the files are not copied on sequence. When writting to folder2 we should get following result

Code:
/tmp/folder2 $ ls -tr| more 
1
2
3
4
5
6
7
8
9
10

# 7  
Old 08-13-2013
This is what i came with. If i correctly understood your question.

You have a file like :
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000014
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000009
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000017

And you need to order it by the BATCHNO as in :
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000009
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000014
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000017


Hope this will help you. Feel free to modify the script to put the file in another path.


Code:
#!/bin/bash
#Date :
#Author
#Purpose : Sort batch file
#Usage : ./sortbatch.sh <batchfile>

BATCHFILE=$1
ORDEREDBATCH=`pwd`/ordered_`date +%F_%H:%M`_$BATCHFILE

#Verify if a file was given
if [ $# -lt 1 ]; then
        echo "Usage : ./sortbatch.sh <batchfile>"
        exit 1
fi

#Verify if the file exists
if [ ! -e $BATCHFILE ]; then
        echo "File does not exist"
        exit 1
fi

#Read batch numbers
BATCHNRS=`awk '{print $4}' $BATCHFILE | sort | uniq`

#Copy to another file the lines in order of the batch number
for i in $BATCHNRS; do
        awk -v batch="$i" '{if ($4 == batch ) print $0}' $BATCHFILE >> $ORDEREDBATCH
done

#End

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Start process on X number of files and then wait for the next batch

Thanks for RudiC for his extraordinary help on organizing files in a batch of 10 using below code. FL=($(ls)); for ((i=0;i<=${#FL};i++)); do for j in ${FL:$i:10}; do $batch ${j} ${j}.txt done; echo "Pausing for next iteration"; echo... (6 Replies)
Discussion started by: busyboy
6 Replies

2. Shell Programming and Scripting

Random ordering

1 2 4 5 3 I would like to use a script so that i can randomly rearrange these numbers such as 3 5 2 4 1 Thanks! (3 Replies)
Discussion started by: johnkim0806
3 Replies

3. Shell Programming and Scripting

ordering

file1 1 SNP2 3 1 SNP3 3 1 SNP5 4 2 SNP1 4 2 SNP4 4 file2 SNP1 1 1 1 SNP5 5 5 5 SNP4 4 4 4 SNP2 2 2 2 SNP3 1 1 1 desired output (1 Reply)
Discussion started by: johnkim0806
1 Replies

4. Shell Programming and Scripting

Executing a batch of files within a shell script with option to refire the individual files in batch

Hello everyone. I am new to shell scripting and i am required to create a shell script, the purpose of which i will explain below. I am on a solaris server btw. Before delving into the requirements, i will give youse an overview of what is currently in place and its purpose. ... (2 Replies)
Discussion started by: goddevil
2 Replies

5. Shell Programming and Scripting

Re ordering lines - Awk

Is it possible to re-order certain rows as columns (of large files). Few lines from the file for reference. input Splicing Factor: Tra2beta, Motif: aaguguu, Cutoff: 0.5000 Sequence Position Genomic Coordinate K-mer Score 97 chr1:67052604 uacuguu 0.571 147... (3 Replies)
Discussion started by: quincyjones
3 Replies

6. Shell Programming and Scripting

Re-ordering data

input Predictions for job: 1299399580 ********************************************** gg18_qqq10_100017878_100017978_- ============================================================================== zzz Factor: XXX, ttt: crsmsgw, Cutoff: 0.6429 seqe Position fff Coordinate K-mer Score ... (3 Replies)
Discussion started by: quincyjones
3 Replies

7. Shell Programming and Scripting

ordering a data file

With an input file like this: How can I get an output like this? (In the quoted examples, the "_" sign represents an empty space) Note that there are some minus signs and no spaces, in the example above the first character of the first line is an empty space, so each number spans 10... (16 Replies)
Discussion started by: lego
16 Replies

8. Homework & Coursework Questions

word ordering problem HELP please (linux)

Hi guys I need you ,please help me i have to do this for tomorow and i don't understand how to do Q1 : Order the words of RADIO.txt by frequency Q2 : Order the words of RADIO.txt in alphabétique order Q3 : Order the words of RADIO.txt par ordre "rhymique" (exemple, put togeder words which are... (1 Reply)
Discussion started by: Lili
1 Replies

9. Shell Programming and Scripting

proper ordering of o/p values

Hi, Below is my script which creates a file: #!/bin/sh if then echo "Enter bill period " echo "Syntax: sh cpd.sh G08" exit fi sqlplus uname/pwd@dbname <<EOF set WRAP off set FEEDBACK off set PAGESIZE 0 set VERIFY off (14 Replies)
Discussion started by: ss_ss
14 Replies

10. UNIX for Dummies Questions & Answers

Append value(batch number) to start of records

Hi all, I am new to unix shell scripting and I am trying to append batch number that comes in Trailer record to the detailed record. TR|20080312|22881 |000000005|20080319|2202 LN|20080312|077777722220 |0000100000017|ABS LN|20080312|000799439326 |0000709943937|AA TR|20080313|22897 ... (6 Replies)
Discussion started by: kiran_418
6 Replies
Login or Register to Ask a Question