Stacking Columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Stacking Columns
# 8  
Old 03-20-2013
Quote:
Originally Posted by vgersh99
hm..... I really don't see how the first sample/result relates to the second one. I (for one) could not understand the first one at all - maybe others could have guessed better given your last explanation - I cannot....
So was my "guess" good-enough?
I wish I could explain it better. I will try out your suggestion in a bit. I apreciate the effort.

Mike
# 9  
Old 03-20-2013
Quote:
Originally Posted by Michael Stora
I guess I could have been more clear in my first example, but the second example seems clear to me.
I agree. I wasn't familiar with stacking in this context, and, as you noticed, there was an error with a missing header in the original post. I also agree that your second example is clear.

The following is a more generalized version of vgersh99's approach:
Code:
awk '

NR == 1 {
    split($0, labels)
    pr_rec(h1 FS h2)
}

NR > 1 {
    for (i=a; i<=b; i++)
        pr_rec(labels[i] FS $i)
}

function pr_rec(s, r, i) {
    for (i=1; i<a; i++)
        r = r $i FS
    r = r s
    for (i=b+1; i<=NF; i++)
        r = r FS $i
    print r
}

' FS=, a=2 b=4 h1=' Label' h2=' Value' filename

a: 1-based index of first column to stack
b: 1-based index of last column to stack
h1: Name of first new column
h2: Name of second new column

The values assigned in the code correspond to your second example's data.

If I were going to deploy this code, in the NR==1 section I would also add a sanity check to confirm that a is positive and not greater than b, and that b is less than NF (the number of columns in the first line). If any of the checks fail, print a diagnostic to stderr and abort.

Regards,
Alister

Last edited by alister; 03-20-2013 at 08:40 PM..
These 2 Users Gave Thanks to alister For This Post:
# 10  
Old 03-20-2013
I think the following example the poster provided is clear enough.

Get attribute names from first input row. Print special invariant first output row. For "Bob" input row, print separate output line for each attribute, with format: Bob, Attribute_Name, Attribute_Value, ID. Then, do same thing for "Jill" input row. Keep going until no more input rows.

If I'm wrong about the logic, please correct me. But it seems straightforward format to me.

Code:
Name, Sex, Height, Weight, ID 
Bob, M, 75, 220, 21345 
Jill, F, 61, 120, 43563 
Frank, M, 61, 300, 57864

Code:
Name, Label, Value, ID
Bob, Sex, M, 21345
Bob, Height, 75, 21345
Bob, Weight, 220, 21345
Jill, Sex, F, 43563
Jill, Height, 61, 43563 
Jill, Weight, 120, 43563 
Frank, Sex, M, 57864 
Frank, Height, 61, 57864 
Frank, Weight, 300, 57864

Sorry I don't have a solution. But this looks like a setup for awk.
# 11  
Old 03-20-2013
Alister, it works great, It works on a huge database file in a fraction of a second.

I had no idea you would pass an array like that--nice!

I am trying to understand a couple things:
1. Why does r = r $i FS not result in a leading comma at the beginning of the line when the initial value of r passed to the function is ,?
2. What is the difference between the i passed to the function and the i in the loops? I don't think they can be the same variable for my understanding of the code to function.

Mike
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Adding columns from 2 files with variable number of columns

I have two files, file1 and file2 who have identical number of rows and columns. However, the script is supposed to be used for for different files and I cannot know the format in advance. Also, the number of columns changes within the file, some rows have more and some less columns (they are... (13 Replies)
Discussion started by: maya3
13 Replies

2. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ... (4 Replies)
Discussion started by: Khushbu
4 Replies

3. Shell Programming and Scripting

Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All, I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns. So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies

4. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

5. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

6. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

7. Shell Programming and Scripting

Combine columns from many files but keep them aligned in columns-shorter left column issue

Hello everyone, I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble. I have many files each having two columns and hundreds of rows. first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
Discussion started by: isildur1234
5 Replies

8. UNIX for Advanced & Expert Users

Help in Deleting columns and Renaming Mutliple columns in a .Csv File

Hi All, i have a .Csv file in the below format startTime, endTime, delta, gName, rName, rNumber, m2239max, m2239min, m2239avg, m100016509avg, m100019240max, metric3min, m100019240avg, propValues 11-Mar-2012 00:00:00, 11-Mar-2012 00:05:00, 300.0, vma3550a, a-1_CPU Index<1>, 200237463, 0.0,... (9 Replies)
Discussion started by: mahi_mayu069
9 Replies

9. Shell Programming and Scripting

Single command for add 2 columns and remove 2 columns in unix/performance tuning

Hi all, I have created a script which adding two columns and removing two columns for all files. Filename: Cust_information_1200_201010.txt Source Data: "1","Cust information","123","106001","street","1-203 high street" "1","Cust information","124","105001","street","1-203 high street" ... (0 Replies)
Discussion started by: onesuri
0 Replies

10. Shell Programming and Scripting

stacking case options

possible to stack case commands ? I get an error, s. comments below select choice in search delete quit do case $choice in search) select option in title director year do case $option in title) ..... break ;; ... (2 Replies)
Discussion started by: forever_49ers
2 Replies
Login or Register to Ask a Question