Sponsored Content
Top Forums Shell Programming and Scripting Program to tar files to packages less then 200000 bytes Post 302426885 by presul on Thursday 3rd of June 2010 07:38:44 AM
Old 06-03-2010
Program to tar files to packages less then 200000 bytes

Trying to write program to archive files from a file system.
I got some problem with appropriate selection of files in regard to size.

Algorithm basic :
-----------------
1. Remotely executes on machine ls cmd and its output redirects and sorts by size of a files :
Code:
# cd ~/usr/local/inp/mails/inbox; ls -la | sort -n" > ./iw1_output.log_$$

2. Program filters this file and finally creates new file named log.January.1.array with structure below :
Code:
1000;plikA
3000;plikB
4000;plikC
5000;plikD
6000;plikE
180000;plikF
185000;plikG
189000;plikH
190000;plikI

A dot doesn't mean compund variable in my example.

3.Then I want to construct engine which will sum size of files displayed above in first column and if it less or equal to 200000 then perform tar <list of files from 2nd column> and if greater then sum each line from 1 column then divide entire size of values in column 1 by 200000 to designate how many tar archives we need($nrOfIterations). Then perform division of population in column 1 to meet <=200000 criteria and again check previos condition.

Code:
for n in `cat log.January.1.array`
do
        (( number=`print $n | awk -F\; '{print $1}'` ))
        string=$string" "`print $n | awk -F\; '{print $2}'`
        (( filesSize=$filesSize+$number ))
done                   


(( nrOfIterations=filesSize/200000 ))
print "Szacunkowa ilosc iteracji wynosi: $nrOfIterations"        
print

if (( nrOfIterations>0 ));
then
        (( nrOfIterations=$nrOfIterations+1 ))
        print "Prawdziwa ilosc iteracji wynosi: $nrOfIterations"
        print   
fi


if (( filesSize<=200000 ));
then
# umiesc tar | tar do file systemu z SAMBA zamontowanego w /mnt
        print "tar cf JANUARY_`date '+%d%m%Y'`.emails.tar" "$string"
        else
                        for n in `cat log.January.1.array`
                        do
                                (( number=`print $n | awk -F\; '{print $1}'` ))
                                string=`print $n | awk -F\; '{print $2}'`
                                (( filesSize=$filesSize-number ))
                                (( mNumber=$mNumber+$number ))

                                if (( mNumber<=200000 ));         
                                then
                                        ArrayString1[$i]=`print "$string"`
                                        ArrayNumber1[0]=`print "$mNumber"`
                                        
                                        (( i=$i+1 ))    
                                        $(sed -e /"$n"/d <log.January.1.array >log.January.1.array.1)
                                        
                                else
                                        (( mNumber=0 ))
                                        (( nrOfIterations=$nrOfIterations-1 ))   
                                        
                                        $(cat log.January.1.array >log.January.1.array.1)           
                                fi
                                `mv log.January.1.array.1 log.January.1.array`
                        done

                        print ${ArrayString1[*]}
                        print

                        print ${ArrayNumber1[*]}
                        print
fi

Problems :
1. Dynamically reference variable names of $ArrayString1, $ArrayNumber1 by using variable $nrOfIterations.

2. Red colored code should exchange file log.January.1.array with file log.January.1.array.1 but unfortunatelly variable n in for loop references to file log.January.1.array before the exchange happens.

Last edited by Scott; 06-03-2010 at 09:55 AM.. Reason: Code tags, PLEASE!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Moving files till the size is less than 200000.

Hi, I need your help. Suppose I have a directory called /home/rooh This directory contains files and directories. For ex drwxr-xr-x 3 rooha arboradm 96 Apr 6 03:24 batches drwxr-xr-x 2 rooha arboradm 96 Apr 6 03:21 worker -rw-rw-rw- 1 rooha arboradm ... (1 Reply)
Discussion started by: rooh
1 Replies

2. UNIX for Dummies Questions & Answers

Uninstalling packages (.tar.gz)

Hi, I still don't know if there's a standard way to do this. I'm hoping there's a *more or less* standard way to do this across linux platforms but the platform in question is Gentoo. Let's say I download a source tarball as tar.gz and I do the typical gunzip app_to_install.tar.gz tar xvf... (2 Replies)
Discussion started by: d11wtq
2 Replies

3. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies

4. HP-UX

Make command performs badly and refuses to compile on 200000 files

I am trying to run make command on 200000 files in HP UX but it refuses to compile giving a message that " command line is too long .stop" I checked and found out that there is a limit imposed by the operating system on the command line .for Eg refer following link : The maximum length of... (6 Replies)
Discussion started by: madhur.tripathi
6 Replies

5. Programming

Copying 1024 bytes data in 3-bytes chunk

Hi, If I want to copy a 1024 byte data stream in to the target location in 3-bytes chunk, I guess I can use the following script. dd bs=1024 count=3 if=/src of=/dest But, I would like to know, how to do it via a C program. I have tried this with memcpy(), that did not help. (3 Replies)
Discussion started by: royalibrahim
3 Replies

6. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

7. UNIX for Dummies Questions & Answers

problem with packages that have .tar.gz in the extention

Hi Everybody , I want to install some packages on sun Solaris , I download the packages it was for example openssl.tar.gz it is not data stream or sun packages , so I search and I found command called make solve this problem , but the packages of this command not found on my machine, so I... (7 Replies)
Discussion started by: hard_revenge
7 Replies

8. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

9. Shell Programming and Scripting

Shell script - entered input(1-40 bytes) needs to be converted exactly 40 bytes

hello, suppose, entered input is of 1-40 bytes, i need it to be converted to 40 bytes exactly. example: if i have entered my name anywhere between 1-40 i want it to be stored with 40 bytes exactly. enter your name: donald duck (this is of 11 bytes) expected is as below - display 11... (3 Replies)
Discussion started by: shravan.300
3 Replies

10. Shell Programming and Scripting

Divide an EBCDIC files into multiple files based on value at 45-46 bytes

Hi All, I do have an EBCDIC file sent from the z/os , this file has records with different record types in it, the type of record is identified by bytes 45-46 like value 12 has employee record value 14 has salaray record and etc.... we do now want to split the big ebcdic file into multiple... (3 Replies)
Discussion started by: okkadu
3 Replies
All times are GMT -4. The time now is 06:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy