Rearranging


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rearranging
# 1  
Old 04-27-2011
Rearranging

Hello,

I spent all day trying to write a script and cannot find the solution Smilie

I have plenty files looking like this:
Code:
several hundred
lines precede 
the following interesting 
Bla xxx:
Blub[0][0] = -7537.37687
Blub[0][1] = -100.644746
Blub[0][2] = -3247.61954
.
.
.
Blub[0][274] = 1324.82567
Blub[1][0] = -967.12936
.
.
.
Blub[13][274] = 805.250982

I need to extract only the numbers in the last column and to rearrange them into 14 columns (this is for all files the same) with an equal number of lines (in the first case: 275. But this number can vary between files!).

This is what I managed so far:
Code:
#!/bin/bash

### one example file:
declare i=TestFile.txt

datend=`awk 'END { print NR }' ${i}`
findline=`awk '/^Bla xxx:/ {zeile=NR} END{print zeile}' ${i}`
findline=`expr ${findline} + 1`
nolines=`expr ${datend} - ${findline}`
### either:
#cat $i | grep "^Bla xxx:" -A $nolines | tail -$nolines  > ${i%.*}_tmp.txt
### or:
awk -F: 'NR >= v1 && NR <= v2' v1="${findline}" v2="${datend}" $i > ${i%.*}_tmp.txt

####At this point I have only the last column extracted.
#awk '{print $3}' $ECHO "${i%.*}_tmp.txt" > ${i%.*}_onlynumbers.txt

Before the last (commented) step above I wanted to move on with something like this:
Code:
for (( j=0; j<=13; j++ ))
do
findfirst=`awk '/^Blub && $j/ {line=NR} {print line}' $ECHO "${i%.*}_tmp.txt"`	
findlast=`awk '/^Blub && $j/ {line=NR} END{print line}' $ECHO "${i%.*}_tmp.txt"`	
awk -F: 'NR >= v3 && NR <= v4' v3="${findfirst}" v4="${findlast}" $ECHO "${i%.*}_tmp.txt" >> test.txt
### or using this approach:
awk -v 'FS=^Bar[?|]?' '{print $2}' $ECHO "${i%.*}_tmp.txt" >> test.txt
done

in order to find the first line where $j is in [], I thought I could declare the [ as something like a space.
But it is not working.
So basically my plan is to find first all 275 lines belonging to j=0 and then to j=1 and so on until j=13 and grap from those lines only the numbers in the last column and as the last step using the command "paste" and put all 14 columns in a new file.

Does anyone have a suggestion for me how I can solve that?

Thanks a lot in advance!

Best,
tempestas

Last edited by tempestas; 04-27-2011 at 01:02 PM..
# 2  
Old 04-27-2011
here's something to start with:

nawk -f temp.awk myFile

temp.awk:

Code:
BEGIN {
  FS="(\\[)|(\\])|(=)"
  OFS=","
}
NF==6 {
  a[$2,$4]=$NF
  maxI=($2>maxI)?$2:maxI
  maxJ=($4>maxJ)?$4:maxJ
}
END {
  for(i=0;i<=maxI;i++)
    for(j=0;j<=maxJ;j++)
       printf("%s%c", a[i,j], (j==maxJ)?ORS:OFS)
}


Last edited by vgersh99; 04-27-2011 at 03:31 PM..
# 3  
Old 04-28-2011
Hi,

sorry for the late reply. I had to install nawk in order to run the script.

Thanks vgersh99 for the script! It works (as you probably know). But actually, I do not fully understand how it works Smilie
Can you explain it a little bit for a beginner like me? Smilie Smilie

Also in your first version you specified the number of lines and now you edit so that it finds out automatically how many lines there are? Is that correct?

And when I wanna rearrange this 1 column into 14 columns with an equal number of lines, do I write that in my main script from where I work on all files in that folder or do I add this feature in the .awk script?

Thanks a lot for your help!

Best,
tempestas
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Rearranging the column

I have a text file like this, I would like to rearrange the first column (Name) according to the third column(percentage)in descending order. I mean methionine with the highest percentage should be the first one to appear under the name column. But I also want to exclude the headers from this... (2 Replies)
Discussion started by: cathum
2 Replies

2. Shell Programming and Scripting

Rearranging Data Set

Hello everybody, I've got the following problem: The data set I have is an ASCII file containing a header over 4 lines and the actual data comprised of dezimal numbers in a 1000x1000 grid (1000 lines and 1000 columns). Since I want to plot the data in GMT I need to convert it into the... (3 Replies)
Discussion started by: Evilknievel
3 Replies

3. Shell Programming and Scripting

Help with rearranging file with script

Hi Guys I normally do thins with a Windows program but I am trying to rearrange a filename based on delimiters in Ubuntu. Example v017 __ Detective Academy Q #133 Murder in the Village Of Suspension Bridges &&& Part 9.cbz = Detective Academy Q v017 #133 Murder in the Village Of Suspension... (9 Replies)
Discussion started by: itschrisonline
9 Replies

4. Shell Programming and Scripting

Rearranging of values as desired

i have a file as below :- 100 D 22 100 T 33 100 C 89 101 C 55 101 D 44 102 D 88 103 T 22 103 C 13 output format :- <number> <D value> <C Value> <T Value> if no value then zero. I want output as :- 100 22 33 89 101 44 55 0 102 88 0 0 103 0 13 22 (3 Replies)
Discussion started by: satishmallidi
3 Replies

5. Shell Programming and Scripting

Counting and rearranging the rows

Hi, I have a file that I re-arranged using awk and unix commands to produce a file that looks like this: JOE JOE JOE JOE JOE BOB BOB HI HI HI I want to count how many of the same rows there are and print it on the second column while only maintaining the original name once. The... (5 Replies)
Discussion started by: phil_heath
5 Replies

6. Shell Programming and Scripting

Doubt with rearranging file through awk

Filename1.xml NO 2012-11-16 02:00:27 20121115/pathname/ asia Filename1.rec YES 2012-11-16 01:20:24 20121115/pathname asia FIleName2.xml YES 2012-11-16 01:20:25 20121115/pathaname asia if the file content are... (6 Replies)
Discussion started by: manas_ranjan
6 Replies

7. Shell Programming and Scripting

Rearranging into new columns (awk?)

Hi experts, I've used several solutions from this forum to delete nonsense and rearrange data in the project file I'm working on. I'm hoping you guys can give me some tips on further rearranging the data (I've seen a few solutions by searching, but one specific item has me stumped, which is only... (5 Replies)
Discussion started by: coryvp
5 Replies

8. UNIX for Dummies Questions & Answers

Rearranging whole columns

Hello all, I have a text file that is arranged: name 3 7 2 9 5 jim a d e g k max d g u x g rob f w v k o This is just an example as my real file has >1000 individuals and >64,000 columns. I need to rearrange the file so that the columns appear in numerical order so that name... (3 Replies)
Discussion started by: doobedoo
3 Replies

9. Shell Programming and Scripting

Rearranging columns

Hi, I have an input file as follows : input.txt abcdTXXqwe axdfSYYrew dasgTXXqwt gtfsTYYwer gadfSXXerw gwerSYYTXX Now I have to get four output files. output1.txt should have the first four cloumns, Where the rows containing 5th column as T and 6th-7th columns as XX output2.txt... (5 Replies)
Discussion started by: sudhamacs
5 Replies

10. Shell Programming and Scripting

Rearranging fields from a pipe

I have a large file that I am pulling only certain fields out of but my output I would like to rearrange the field order via a pipe. I have been looking through the site and man pages and have come to a loss. I am running on HP cut -c33-38,44-46,62-65,91-98 <file> | grep -e <value> >... (4 Replies)
Discussion started by: bthomas
4 Replies
Login or Register to Ask a Question