Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sorting rules on a text section

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sorting rules on a text section
# 1  
Old 12-04-2006
Java Sorting rules on a text section
Hi all

My text file looks like this:

Code:
start doc
...                    (certain number of records)
REC3|Emma|info|
REC3|Lukas|info|
REC3|Arthur|info|
...                     (certain number of records)
end doc
start doc
...                     (certain number of records)
REC3|Tamira|info|
REC3|Henry|info|
...     
end doc

As you see, the text file contains different document sections, and some of the records in there are tagged with "REC3" which is a unique record code. REC3 lines are always in a row, but obviously there are as many REC3 sections as there are documents.

What I want to do is to sort these REC3 lines per document using the quickest UNIX command possible. The expected output file is:

Code:
start doc
...                    (certain number of records)
REC3|Arthur|info|
REC3|Emma|info|
REC3|Lukas|info|
...                     (certain number of records)
end doc
start doc
...                     (certain number of records)
REC3|Henry|info|
REC3|Tamira|info|
...     
end doc

I don't want to touch any of the other record types. Does any of you have any suggestion about how to do this? I would be very grateful. I had in mind perl had some good potential to make it simple, but I´m no expert. I tried a couple of things with ksh/awk but it tended to become complex and unflexible.
Last edited by Indalecio; 12-04-2006 at 11:13 AM..
# 2  
Old 12-04-2006
you could do this by having few intermediate steps like create temporary files for each document sections and then sort the necessary temporary file as per your need and later combind them in the same order. Hope this would help.

--Manish
# 3  
Old 12-04-2006
input
Code:
start doc
...                    (certain number of records)
REC3|Emma|info|
REC3|Lukas|info|
REC3|Arthur|info|
...                     (certain number of records)
end doc
start doc
...                     (certain number of records)
REC3|Tamira|info|
REC3|Henry|info|
...     
end doc

output:
Code:
 
start doc
...                    (certain number of records)
REC3|Arthur|info|
REC3|Emma|info|
REC3|Lukas|info|
...                     (certain number of records)
end doc
start doc
...                     (certain number of records)
REC3|Henry|info|
REC3|Tamira|info|
...
end doc

code :
Code:
#/bin/ksh
set -x
found=0
set -A arr ""
while read line
do
	rec3=${line##REC3|}
	if [[ ${#rec3} -lt ${#line} ]] ; then
		arr[$found]="$line"
		found=$found+1	    
	    continue
	fi
	if [[ $found -gt 0 ]] ;then
		
		cat tmp.tmp
		let i=0
		while [[ $i -lt $found ]]
		do
		    printf "%s\n" ${arr[i]} 
		    i=$i+1
		done | sort -k2.1,2.30 -t'|' 
		found=0
        set -A arr ""
	fi
	echo "$line"
done < filename

# 4  
Old 12-05-2006
Thank you for your response

The thing is, I had already tried simple file manipulation like the following:
Code:
#!/usr/bin/ksh

new_block="N"
rm *.tmp

while read line; do
   record=`echo $line | cut -d '|' -f 1`
   if [[ "$record" = "REC3" ]]; then 
     if [[ "$new_block" = "Y" ]]; then
         echo $line >> REC3.tmp
      else
         new_block="Y"
         echo $line > REC3.tmp
      fi
   else
      if [[ "$new_block" = "Y" ]]; then
         new_block="N"
         sort REC3.tmp > finalREC3.tmp
         cat rest.tmp finalREC3.tmp >> output.tmp         
      fi
      echo $line >> rest.tmp
    fi
done < $1
cat rest.tmp >> output.tmp

But this is incredibly slow! If I run this on a 20 MB file I can wait for a few minutes... I just wonder if a more powerful command would do the same job a bit quicker (I mentioned perl but it could be anything).
# 5  
Old 12-05-2006
For your info, the above command took 40 minutes to make the job, which is not acceptable.

I came to another solution, check this out:
Code:
gawk 'BEGIN {FS="|";blockREC3="N"}{if ($1 == "REC3"){if (blockREC3 == "Y"){j++;ind[j]=$0} else {blockREC3="Y";j=1;ind[j\
]=$0} } else {if (blockREC3 == "Y") {blockREC3="N";n=asort(ind);for (i = 1; i <= n; i++) {print ind[i]}}; print $0}}' $1 > outp\
ut.tmp

I was trying to find something that awk could do, unfortunately the asort command is only supported by the gawk version. So I haven't tested it since I need to install gawk first. But I think this may save a lot of time.

EDIT: I tested it, it took 2 sec Smilie
Last edited by Indalecio; 12-05-2006 at 05:40 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sorting blocks by a section of the identifier

I have a file that should be sorted by a string (shown in red in my example below) in the identifier. The RS is ^@M0, something like this: @M04961:22:000000000-B5VGJ:1:1101:9280:7106 1:N:0:86 GGCATGAAAACATACAAACCGTCTTTCCAGAAATTGTTCCAAGTATCGGCAACAGCTTTATCAATACCATGAAAAATATCAACCACACCAGAAGCAGCAT... (16 Replies)
Discussion started by: Xterra
16 Replies

2. Shell Programming and Scripting

Sorting indented text files

Hello, I'm trying to find a solution or a proper tool for the following job: I need to sort a text document with indented sections, so all levels of indentation are sorted independently for each section. Particularly, I need this for Cisco routers' running config files to compare them with... (2 Replies)
Discussion started by: kobel
2 Replies

3. Shell Programming and Scripting

Grep text and see all section

Hello I am looking for a way to look in files and to grep text and see the all section of the text ! Sharon123 deed 10000 class 360 ! sharon456 2000 deed ! Sharon789 live 3000 ! To grep "deed "an see the all section (5 Replies)
Discussion started by: sharong
5 Replies

4. Shell Programming and Scripting

Extracting text from within a section of text using AWK

I have a command which returns the below output. How can I write a script to extract mainhost and secondhost from this output and put it into an array? I may sometimes have more hosts like thirdhost. I am redirecting this output to a variable. So I guess there should be a awk or sed command to... (7 Replies)
Discussion started by: heykiran
7 Replies

5. UNIX for Dummies Questions & Answers

Sorting arrays horizontally without END section, awk

input: ref001, Europe, Belgium, 1001 ref001, Europe, Spain, 203 ref001, Europe, Germany, 457 ref002, America, Canada, 234 ref002, America, US, 87 ref002, America, Alaska, 652 Without using an END section, I need to write all the info related to the same ref number ($1)and continent ($2) on... (9 Replies)
Discussion started by: lucasvs
9 Replies

6. UNIX Desktop Questions & Answers

Problem in sorting a text file

Hi; I have a text file like this: 1 10 11 2 3 4 M X Y When I sort it numerically using sort -n, it looks like this: Y X M 1 2 3 4 10 (3 Replies)
Discussion started by: a_bahreini
3 Replies

7. Shell Programming and Scripting

sorting based on a specified column in a text file

I have a tab delimited file with 5 columns 79 A B 20.2340 6.1488 8.5086 1.3838 87 A B 0.1310 0.0382 0.0054 0.1413 88 A B 46.1651 99.0000 21.8107 0.2203 89 A B 0.1400 0.1132 0.0151 0.1334 114 A B 0.1088 0.0522 0.0057 0.1083 115 A B... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

8. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

9. Shell Programming and Scripting

Sorting a text file

In unix how to sort in reverse order based on second field in a text file. $ cat data1 David:501 Albie:503 Shaun:502 The expected output: Albie:503 Shaun:502 David:501 Please help :) (4 Replies)
Discussion started by: jon2ryhme
4 Replies

10. UNIX for Dummies Questions & Answers

how can i extract text section via grep

hi, i have a very long text file. i need to extract with grep command a certain part. for example text file include 1ooo rows: 1.... 2... 3... . . . 1000 i want to view with grep only rows 50-100. any ideas will be appreciated thanks... (8 Replies)
Discussion started by: meny
8 Replies
Login or Register to Ask a Question