Transpose info that is within blocks


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transpose info that is within blocks
# 8  
Old 03-24-2012
Many thanks Scrutinizer,

I still have to test it with real large files (150MB), but it seems that works with the
sample of 1000 lines I have access now.

I have understood better your code now, only have some doubts (in red below)
Code:
awk '{$1=$1}1' infile |   #If the file is about 150MB this pipe still will work?
awk '
BEGIN{
  header="Type Of Record|ChargeableDuration|DateForStartOfCharge|ExchangeIdentity|RecordSequenceNumber|TimeForStartOfCharge|TimeForStopOfCharge|GSMCallReferenceNumber|MSCAddress|OutgoingRoute|CalledPartyNumber|NetworkCallReference|CallingPartyNumber|IncomingRoute"
  print header
  m=split(header,H,"|")
}

$1!="BeginOfRecord"{ next} # I think this means, go to next record if $1 is not exactly to BeginOfRecord?

{
  F[1]=$7
  for(i=8;i<=NF;i++) {
    if($i==H[2])  F[2]=$(i+11)":"$(i+19)":"$(i+27)
    if($i==H[3])  F[3]=$(i+8)
    if($i==H[4])  F[4]=$(i+2)
    if($i==H[5])  F[5]=$(i+2)
    if($i==H[6])  F[6]=$(i+11)":"$(i+19)":"$(i+27)
    if($i==H[7])  F[7]=$(i+11)":"$(i+19)":"$(i+27)
    if($i==H[8])  F[8]=$(i+2)
    if($i==H[9])  F[9]=$(i+22)
    if($i==H[10]) F[10]=$(i+2)
    if($i==H[11]) F[11]=$(i+22)
    if($i==H[12]) F[12]=$(i+2)
    if($i==H[13]) F[13]=$(i+22)
    if($i==H[14]) F[14]=$(i+2)
  }
  $0=x  #What does it mean this statement?
  for(i=1;i<=m;i++){  # which is the value of m? I dont see it with a value assisned previously
    $i=F[i] #Here you assign the array values to each field to be printed in the output?
    delete F[i] #Delete array values for memory and speed issues or why?
  }
}1 # This "1" at the end is to print the operations established within the for loop?
' RS= OFS=\| # If the blocks were separated by some string like "*** End of block" or only one blank line, how would be RS? RS="*** End of block"? or RS=" "

Many thanks for all your great help and support.

Best regards
# 9  
Old 03-24-2012
Quote:
If the file is about 150MB this pipe still will work?
I don't expect there will be problem with that..
Quote:
think this means, go to next record if $1 is not exactly to BeginOfRecord
Correct
Quote:
$0=x #What does it mean this statement?
Set the record to "" (x is uninitialized an thus contains "")
Quote:
#Here you assign the array values to each field to be printed in the output?
Correct
Quote:
# This "1" at the end is to print the operations established within the for loop
It means, perform the default action, which is {print $0}, i.e. print the current record
Quote:
# If the blocks were separated by some string like "*** End of block" or only one blank line, how would be RS? RS="*** End of block"? or RS=" "
RS can only be a single character, unless your awk has a special extension, so that it can be an extended regular expression. gawk and mawk can do this, asterisks would need to be escaped probably. Setting RS to " " does not make sense, since there are plenty of spaces within the record..
# 10  
Old 03-24-2012
Thank you again Scrutinizer,

I only fortgot to put in red the question,
Code:
delete F[i] #Delete array values for memory and speed issues or why?

Thanks a lot again.

Regards
# 11  
Old 03-25-2012
To remove values from there current line before processing the next line, otherwise you may end up using old values from a previous line if one or more fields are missing in the current line, while these fields should be empty.. Good luck, I hope this helps you find a solution..

Last edited by Scrutinizer; 03-25-2012 at 02:24 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 12  
Old 03-25-2012
Quote:
Originally Posted by Scrutinizer
To remove values from there current line before processing the next line, otherwise you may end up using old values from a previous line if one or more fields are missing in the current line, while these fields should be empty.. Good luck, I hope this helps you find a solution..
Hello again,

Certainly yes it helps a lot to get the solution I was looking for. Really many thanks to be accesible to help to other persons.

Honestly very appreciated and gratefull.

Best regards

---------- Post updated at 09:39 PM ---------- Previous update was at 02:43 AM ----------

Hello again to all,

Is there a way to run the script having the input file outside of it?

The Scrutinizerīs script is of this form:
Code:
awk '{$1=$1}1' inputfile | awk ' ...' RS= OFS=\| > output

If it is possible, I would like to run the script as follow:
Code:
Script.sh inputfile > output 

Thanks in advance for your help.
# 13  
Old 03-26-2012
You could try this:
Code:
awk '{$1=$1}1' "$1" | awk ' ...' RS= OFS=\|

# 14  
Old 03-26-2012
Hi Scrutinizer,

Thank you for your help.

Doesn't work, only prints the headers.

I've been trying to use within the second awk script gsub in order to emulate the first awk command your're using (awk '{$1=$1}1').
Code:
gsub(/^[ \t]*/, "", $0) # I'm inserting this after BEGIN {} 
gsub(/[ \t]*$/, "", $0) # I'm inserting this after BEGIN {}

Thanks in advance for any advice.

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Blocks into table

please help, I have a huge file with blocks of data which I need to convert to a tabular format. Input sample id: GO:0000017 name: alpha-glucoside transport namespace: biological_process def: "The directed movement of alpha-glucosides into, out of or within a cell, or between... (3 Replies)
Discussion started by: ritakadm
3 Replies

2. Shell Programming and Scripting

Row blocks to column blocks

Hello, Searched for a while and found some "line-to-column" script. My case is similar but with multiple fields each row: S02 Length Per S02 7043 3.864 S02 54477 29.89 S02 104841 57.52 S03 Length Per S03 1150 0.835 S03 1321 0.96 S03 ... (9 Replies)
Discussion started by: yifangt
9 Replies

3. Shell Programming and Scripting

Transpose lines from individual blocks to unique lines

Hello to all, happy new year 2013! May somebody could help me, is about a very similar problem to the problem I've posted here where the member rdrtx1 and bipinajith helped me a lot. https://www.unix.com/shell-programming-scripting/211147-map-values-blocks-single-line-2.html It is very... (3 Replies)
Discussion started by: Ophiuchus
3 Replies

4. Shell Programming and Scripting

how to split this file into blocks and then send these blocks as input to the tool called Yices?

Hello, I have a file like this: FILE.TXT: (define argc :: int) (assert ( > argc 1)) (assert ( = argc 1)) <check> # (define c :: float) (assert ( > c 0)) (assert ( = c 0)) <check> # now, i want to separate each block('#' is the delimeter), make them separate files, and then send them as... (5 Replies)
Discussion started by: paramad
5 Replies

5. UNIX for Dummies Questions & Answers

Convert 512-blocks to 4k blocks

I'm Unix. I'm looking at "df" on Unix now and below is an example. It's lists the filesystems out in 512-blocks, I need this in 4k blocks. Is there a way to do this in Unix or do I manually convert and how? So for container 1 there is 7,340,032 in size in 512-blocks. What would the 4k block be... (2 Replies)
Discussion started by: rockycj
2 Replies

6. Shell Programming and Scripting

Help with Script using Command Blocks

Hello, I am trying to create a shell script that use command block (donīt really know if this is the correct way to say it), but while one version works fine, the other one is not working at all. So let me show an example of this "command block" Iīm using and its working ok: cat << _EOF_ `echo... (7 Replies)
Discussion started by: Alexis Duarte
7 Replies

7. Shell Programming and Scripting

Removing blocks from a file

I have a file like the one below. Each record is separated with > In between I have lines consisting of 3 numeric values separated by a space. I need to take each block between the > sign and read the first number in the line. Then take the first after the > sign and the last before the >... (7 Replies)
Discussion started by: kristinu
7 Replies

8. Shell Programming and Scripting

How to read text in blocks

Hi, I have file which contains information written in blocks (every block is different). Is it possible to read every block one by one to another file (one block per file). The input is something like this <block1> <empty line> <block2> <empty line> ... ... ... <block25> <empty... (0 Replies)
Discussion started by: art84_)LV
0 Replies

9. Shell Programming and Scripting

Delete blocks with no data..

Hi, I tried this but could not get it... here is what I need I have an xml where I get all the data in blocks but some times I get empty blocks with no data...shown below..I need to delete only those blocks with no data, I tried couple of ways but could not do it..any help is appreciated...... (1 Reply)
Discussion started by: mgirinath
1 Replies
Login or Register to Ask a Question