Program to tar files to packages less then 200000 bytes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Program to tar files to packages less then 200000 bytes
# 1  
Old 06-03-2010
Program to tar files to packages less then 200000 bytes

Trying to write program to archive files from a file system.
I got some problem with appropriate selection of files in regard to size.

Algorithm basic :
-----------------
1. Remotely executes on machine ls cmd and its output redirects and sorts by size of a files :
Code:
# cd ~/usr/local/inp/mails/inbox; ls -la | sort -n" > ./iw1_output.log_$$

2. Program filters this file and finally creates new file named log.January.1.array with structure below :
Code:
1000;plikA
3000;plikB
4000;plikC
5000;plikD
6000;plikE
180000;plikF
185000;plikG
189000;plikH
190000;plikI

A dot doesn't mean compund variable in my example.

3.Then I want to construct engine which will sum size of files displayed above in first column and if it less or equal to 200000 then perform tar <list of files from 2nd column> and if greater then sum each line from 1 column then divide entire size of values in column 1 by 200000 to designate how many tar archives we need($nrOfIterations). Then perform division of population in column 1 to meet <=200000 criteria and again check previos condition.

Code:
for n in `cat log.January.1.array`
do
        (( number=`print $n | awk -F\; '{print $1}'` ))
        string=$string" "`print $n | awk -F\; '{print $2}'`
        (( filesSize=$filesSize+$number ))
done                   


(( nrOfIterations=filesSize/200000 ))
print "Szacunkowa ilosc iteracji wynosi: $nrOfIterations"        
print

if (( nrOfIterations>0 ));
then
        (( nrOfIterations=$nrOfIterations+1 ))
        print "Prawdziwa ilosc iteracji wynosi: $nrOfIterations"
        print   
fi


if (( filesSize<=200000 ));
then
# umiesc tar | tar do file systemu z SAMBA zamontowanego w /mnt
        print "tar cf JANUARY_`date '+%d%m%Y'`.emails.tar" "$string"
        else
                        for n in `cat log.January.1.array`
                        do
                                (( number=`print $n | awk -F\; '{print $1}'` ))
                                string=`print $n | awk -F\; '{print $2}'`
                                (( filesSize=$filesSize-number ))
                                (( mNumber=$mNumber+$number ))

                                if (( mNumber<=200000 ));         
                                then
                                        ArrayString1[$i]=`print "$string"`
                                        ArrayNumber1[0]=`print "$mNumber"`
                                        
                                        (( i=$i+1 ))    
                                        $(sed -e /"$n"/d <log.January.1.array >log.January.1.array.1)
                                        
                                else
                                        (( mNumber=0 ))
                                        (( nrOfIterations=$nrOfIterations-1 ))   
                                        
                                        $(cat log.January.1.array >log.January.1.array.1)           
                                fi
                                `mv log.January.1.array.1 log.January.1.array`
                        done

                        print ${ArrayString1[*]}
                        print

                        print ${ArrayNumber1[*]}
                        print
fi

Problems :
1. Dynamically reference variable names of $ArrayString1, $ArrayNumber1 by using variable $nrOfIterations.

2. Red colored code should exchange file log.January.1.array with file log.January.1.array.1 but unfortunatelly variable n in for loop references to file log.January.1.array before the exchange happens.

Last edited by Scott; 06-03-2010 at 09:55 AM.. Reason: Code tags, PLEASE!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Divide an EBCDIC files into multiple files based on value at 45-46 bytes

Hi All, I do have an EBCDIC file sent from the z/os , this file has records with different record types in it, the type of record is identified by bytes 45-46 like value 12 has employee record value 14 has salaray record and etc.... we do now want to split the big ebcdic file into multiple... (3 Replies)
Discussion started by: okkadu
3 Replies

2. Shell Programming and Scripting

Shell script - entered input(1-40 bytes) needs to be converted exactly 40 bytes

hello, suppose, entered input is of 1-40 bytes, i need it to be converted to 40 bytes exactly. example: if i have entered my name anywhere between 1-40 i want it to be stored with 40 bytes exactly. enter your name: donald duck (this is of 11 bytes) expected is as below - display 11... (3 Replies)
Discussion started by: shravan.300
3 Replies

3. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

4. UNIX for Dummies Questions & Answers

problem with packages that have .tar.gz in the extention

Hi Everybody , I want to install some packages on sun Solaris , I download the packages it was for example openssl.tar.gz it is not data stream or sun packages , so I search and I found command called make solve this problem , but the packages of this command not found on my machine, so I... (7 Replies)
Discussion started by: hard_revenge
7 Replies

5. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

6. Programming

Copying 1024 bytes data in 3-bytes chunk

Hi, If I want to copy a 1024 byte data stream in to the target location in 3-bytes chunk, I guess I can use the following script. dd bs=1024 count=3 if=/src of=/dest But, I would like to know, how to do it via a C program. I have tried this with memcpy(), that did not help. (3 Replies)
Discussion started by: royalibrahim
3 Replies

7. HP-UX

Make command performs badly and refuses to compile on 200000 files

I am trying to run make command on 200000 files in HP UX but it refuses to compile giving a message that " command line is too long .stop" I checked and found out that there is a limit imposed by the operating system on the command line .for Eg refer following link : The maximum length of... (6 Replies)
Discussion started by: madhur.tripathi
6 Replies

8. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies

9. UNIX for Dummies Questions & Answers

Uninstalling packages (.tar.gz)

Hi, I still don't know if there's a standard way to do this. I'm hoping there's a *more or less* standard way to do this across linux platforms but the platform in question is Gentoo. Let's say I download a source tarball as tar.gz and I do the typical gunzip app_to_install.tar.gz tar xvf... (2 Replies)
Discussion started by: d11wtq
2 Replies

10. UNIX for Dummies Questions & Answers

Moving files till the size is less than 200000.

Hi, I need your help. Suppose I have a directory called /home/rooh This directory contains files and directories. For ex drwxr-xr-x 3 rooha arboradm 96 Apr 6 03:24 batches drwxr-xr-x 2 rooha arboradm 96 Apr 6 03:21 worker -rw-rw-rw- 1 rooha arboradm ... (1 Reply)
Discussion started by: rooh
1 Replies
Login or Register to Ask a Question