Problem to read archive

11-03-2009

Registered User

4, 0

Join Date: Nov 2009

Last Activity: 5 November 2009, 6:20 AM EST

Posts: 4

Thanks Given: 0

Thanked 0 Times in 0 Posts

Problem to read archive

Dear all,

I have this archive: cat file.txt

Code:

archive test  02  sequence 03        02length     52
archive test  02  sequence 04        02length     52
archive test  02  sequence 05        02length     52
teste arquivo 06 sequencia 08        06  length     54
teste arquivo 06 sequencia 09        06  length     54
teste arquivo 08 sequencia 01        08                                    length     88
teste arquivo 09 sequencia 01        09                           length     79

I have this shell....
----------------------------------

Code:

#!/Bin/ksh
dir_work=/aplic/tmp
arquivo=file.txt
cd ${dir_work}
cat $arquivo | while read registro
do
   campo=`echo $registro | cut -c38-39`
   echo $registro >>${campo}_${arquivo} 
done

-----------------------------------
I have a problem when I try to separate in 4 archives (Types 02, 06, 08 and 09).
I hope this results:

Code:

02_file.txt with 3 records
06_file.txt with 2 records
08_file.txt with 1 record
09_file.txt with 1 record.

Help me please.

Last edited by pludi; 11-03-2009 at 06:23 PM.. Reason: code tags, please...

cewa67

View Public Profile for cewa67

Find all posts by cewa67

11-03-2009

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

One problem with your script, is that it should be bin, not Bin. Also you have to use double quotes around $registro to maintain appropriate spacing.

Code:

#!/bin/ksh
dir_work=/aplic/tmp
arquivo=file.txt
cd ${dir_work}
cat $arquivo | while read registro
do
   campo=`echo "$registro" | cut -c38-39`
   echo $registro >>${campo}_${arquivo} 
done

If you want to retain the original spacing of the input file in the output files you'd also have to use double quotes:

Code:

echo "$registro" >>${campo}_${arquivo}

You can easily check what your script is doing by temporarily using

Code:

#!/bin/ksh -x

An alternative way to do it would be:

Code:

#!/bin/ksh
dir_work=/aplic/tmp
arquivo=file.txt
cd ${dir_work}
while read registro; do
   campo=${registro:37:2}
   echo $registro >>${campo}_${arquivo}
done<$arquivo

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

11-04-2009

Registered User

4, 0

Join Date: Nov 2009

Last Activity: 5 November 2009, 6:20 AM EST

Posts: 4

Thanks Given: 0

Thanked 0 Times in 0 Posts

Thanks.... It's solved part of my problem...
I have a header and trailer.....
Header

Code:

                                     00                                   02112009

Trailer

Code:

                                     99     00000061

The shell included header in Type 02 (didn't consider the spaces on the left side) and I lost the Trailer.....
The Trailer contains total amount of register type.

Last edited by Franklin52; 11-04-2009 at 08:18 AM.. Reason: Please use code tags!

cewa67

View Public Profile for cewa67

Find all posts by cewa67

11-04-2009

Registered User

2,759, 420

Join Date: Jun 2006

Last Activity: 13 September 2015, 8:58 PM EDT

Posts: 2,759

Thanks Given: 44

Thanked 420 Times in 408 Posts

Code:

$ cat file.txt |awk '{print $3}'  |sort |uniq -c |sort  -k2 |awk '{print $2"_file.txt with",$1, "records"}'
02_file.txt with 3 records
06_file.txt with 2 records
08_file.txt with 1 records
09_file.txt with 1 records

Code:

$ awk '{a[$3]+=1} END {for (i in a) print i"_file.txt with",a[i],"records" }' file.txt  |sort
02_file.txt with 3 records
06_file.txt with 2 records
08_file.txt with 1 records
09_file.txt with 1 records

Last edited by rdcwayx; 11-04-2009 at 08:27 AM..

rdcwayx

View Public Profile for rdcwayx

Find all posts by rdcwayx

11-04-2009

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

@cewa67: Try using IFS= read -r instead of read

Code:

while IFS= read -r registro

Code:

grep . *_*
00_file.txt:00 02112009
02_file.txt:archive test 02 sequence 03 02length 52
02_file.txt:archive test 02 sequence 04 02length 52
02_file.txt:archive test 02 sequence 05 02length 52
06_file.txt:teste arquivo 06 sequencia 08 06 length 54
06_file.txt:teste arquivo 06 sequencia 09 06 length 54
08_file.txt:teste arquivo 08 sequencia 01 08 length 88
09_file.txt:teste arquivo 09 sequencia 01 09 length 79
99_file.txt:99 00000061

If I put double quotes around $registro when writing to the output files, to preserve original spacing I get:

Code:

00_file.txt:                                     00                                   02112009
02_file.txt:archive test  02  sequence 03        02length     52
02_file.txt:archive test  02  sequence 04        02length     52
02_file.txt:archive test  02  sequence 05        02length     52
06_file.txt:teste arquivo 06 sequencia 08        06  length     54
06_file.txt:teste arquivo 06 sequencia 09        06  length     54
08_file.txt:teste arquivo 08 sequencia 01        08                                    length     88
09_file.txt:teste arquivo 09 sequencia 01        09                           length     79
99_file.txt:                                     99     00000061

Last edited by Scrutinizer; 11-04-2009 at 09:14 AM..

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

11-04-2009

Registered User

4, 0

Join Date: Nov 2009

Last Activity: 5 November 2009, 6:20 AM EST

Posts: 4

Thanks Given: 0

Thanked 0 Times in 0 Posts

I use IFS= read -r instead of read and I get the results.. Thanks.
But I have other problem... the performance...
Processed 94.000 records in 3 minutes but I have 93.000.000 records.
How I solve this?

cewa67

View Public Profile for cewa67

Find all posts by cewa67

11-04-2009

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

Then why didn't you say so? I thought you specifically wanted a shell script

. Try this.

Code:

awk '{print>substr($0,38,2)"_file.txt"}' file.txt

You can make it even faster if you use mawk.

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

Shell Programming and Scripting

Problem to read archive

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to archive logs and sftp to another archive server

Discussion started by: Arjun Goswami

2. Solaris

Boot-archive Problem

Discussion started by: dinu

3. Shell Programming and Scripting

Extracting from archive | compressing to new archive

Discussion started by: chebarbudo

4. Shell Programming and Scripting

Archive::Tar problem

Discussion started by: amcrisan

5. Programming

read file from tar.gz archive

Discussion started by: krylin

6. Shell Programming and Scripting

Read specific file from a zip archive without extracting

Discussion started by: sridharg

7. Solaris

problem in creating flash archive

Discussion started by: girish.batra

8. Shell Programming and Scripting

read list of filenames from text file, archive, and remove

Discussion started by: fxvisions

9. Shell Programming and Scripting

Read from file then purge or archive.

Discussion started by: kayarsenal