The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here. Shell Script Page.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Stumped JeffR Shell Programming and Scripting 3 05-30-2008 06:01 PM
RegEx question has me stumped tolmark UNIX for Dummies Questions & Answers 7 08-18-2007 03:20 PM
httpd.conf - stumped Cameron IP Networking 2 04-25-2002 02:31 AM

Reply
 
Submit Tools LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 05-16-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
I Am Stumped, Please Help

I have a CSV file that I am trying to parse with awk in a table. I think I am going about this the wrong way. I can't get the awk arrays to output with a tab between them in one line. They print out vertically and I need them to print out horizontally. WHAT AM I DOING WRONG?? I want to know if it's
possible to do in awk, because it will become a function in a bigger script if this ever works out.

csv input file:
Code:
175,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
176,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
177,14 ,0,2 ,7  ,0,2 ,11 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
178,5  ,0,2 ,2  ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
179,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
18 ,8  ,0,1 ,13 ,0,2 ,3  ,0,2 ,3  ,0,2 ,   , ,  ,   , ,  ,3  ,0,2 ,3  ,0,2 ,3  ,0,2 ,3  ,0,2 ,3
180,5  ,0,1 ,4  ,0,2 ,12 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
181,1  ,0,2 ,4  ,0,2 ,12 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
182,14 ,0,2 ,7  ,0,2 ,3  ,0,1 ,3  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
script:
Code:
#!/usr/bin/nawk -f
BEGIN {
FS=";"
}
{
x[$2*1"-"$4*1]=x[$2*1"-"$4*1] " " $32 * 1000 + $1"-1"ORS;
x[$5*1"-"$7*1]=x[$5*1"-"$7*1] " " $32 * 1000 + $1"-2"ORS;
x[$8*1"-"$10*1]=x[$8*1"-"$10*1] " " $32 * 1000 + $1"-3"ORS;
x[$11*1"-"$13*1]=x[$11*1"-"$13*1] " " $32 * 1000 + $1"-4"ORS;
x[$14*1"-"$16*1]=x[$14*1"-"$16*1] " " $32 * 1000 + $1"-5"ORS;
x[$17*1"-"$19*1]=x[$17*1"-"$19*1] " " $32 * 1000 + $1"-6"ORS;
x[$20*1"-"$22*1]=x[$20*1"-"$22*1] " " $32 * 1000 + $1"-7"ORS;
x[$23*1"-"$25*1]=x[$23*1"-"$25*1] " " $32 * 1000 + $1"-8"ORS;
x[$26*1"-"$28*1]=x[$26*1"-"$28*1] " " $32 * 1000 + $1"-9"ORS;
x[$29*1"-"$31*1]=x[$29*1"-"$31*1] " " $32 * 1000 + $1"-10"ORS;
}
END {
for ( i in x )
printf "%-3s%-4s\n%-8s\n%-7s\n","UFC ",i,"--------",x[i]
}
output:

Code:
output:

UFC 5-2
--------
 3175-1
 3176-1
 3178-1
 3179-1

UFC 5-1
--------
 3180-1

UFC 14-2
--------
 3177-1
 3182-1

UFC 8-1
--------
 3018-1

UFC 1-2
--------
 3181-1

UFC 4-1
--------
 3175-2
 3176-2
desired output:

Code:
UFC 5-2     UFC 5-1   UFC 14-2
--------   --------   --------
 3175-1     3180-1     3177-1
 3176-1                3182-1
 3178-1
 3179-1
I hope you get the idea of the desired output, too hard to put everything in thread. I want the output to be in a table format, and sorted by UFC numbers from left to right.

Thanks in advance.
Reply With Quote
Forum Sponsor
  #2 (permalink)  
Old 05-17-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 2,253
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Your script already formats the tables at the time it constucts them; the ORS variable evaluates to a newline (and changing it does not really solve anything, at least not trivially).

As a basic principle, you want something like (pseudocode)

Code:
separator=  # nothing
for table in tables
  print separator table.name
  separator=tab # or suitable number of spaces, or something
done
print "\n"

separator=  # nothing
for table in tables
  print separator "-" x (lenght table.name)   # print as many underlines as required
  separator=tab # or suitable number of spaces, or something
done
print "\n"

for i=0 to (however long the longest table is)
  separator=  # nothing
  for table in tables
    print separator
    if (table.value[i])
      table.value[i]
    else
      print ""   # print whitespace if we ran out of values for this table
    separator=tab # or suitable number of spaces, or something
  done
  print "\n"
done
As you can tell, this entails changing your data structure drastically, and possibly even changing to a different language. (Perl or Python would be easier than awk for this I think.)

A less drastic solution would be to output each table to a temporary file and use pr to print it in columnar format. On Linux, I use pr -bt 3 to tabulate into three columns; other platforms offer slightly different options.

Last edited by era; 05-17-2008 at 01:59 AM. Reason: pr -bt3
Reply With Quote
  #3 (permalink)  
Old 05-18-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
era,
Thanks soo much for the quick reply. Unfortunalely this post is a segment of a bigger awk script, that I hope to use as a awk function. Changing to a different language would be time prohibitive. I feel that the only thing that is wrong is the <TABS> between the defined awk arrays in this post that I can't seem to figure out. Does anyone know a way to ouput across a page and not down a page?? Is there another way to capture and sort/group the data like above?
Reply With Quote
  #4 (permalink)  
Old 05-18-2008
...@...
 

Join Date: Feb 2004
Location: NM
Posts: 3,289
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
You can embed perl (or python) scripts in awk, just like you can call any other command line utility. They might be hard to read if they get really big. But that is another issue.
Reply With Quote
  #5 (permalink)  
Old 05-19-2008
Registered User
 

Join Date: Dec 2006
Posts: 28
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Quote:
Originally Posted by timj123 View Post
I have a CSV file that I am trying to parse with awk in a table. I think I am going about this the wrong way. I can't get the awk arrays to output with a tab between them in one line. They print out vertically and I need them to print out horizontally. WHAT AM I DOING WRONG?? I want to know if it's
possible to do in awk, because it will become a function in a bigger script if this ever works out.

csv input file:
Code:
175,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
176,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
177,14 ,0,2 ,7  ,0,2 ,11 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
178,5  ,0,2 ,2  ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
179,5  ,0,2 ,4  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
18 ,8  ,0,1 ,13 ,0,2 ,3  ,0,2 ,3  ,0,2 ,   , ,  ,   , ,  ,3  ,0,2 ,3  ,0,2 ,3  ,0,2 ,3  ,0,2 ,3
180,5  ,0,1 ,4  ,0,2 ,12 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
181,1  ,0,2 ,4  ,0,2 ,12 ,0,2 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
182,14 ,0,2 ,7  ,0,2 ,3  ,0,1 ,3  ,0,1 ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,   , ,  ,3
script:
Code:
#!/usr/bin/nawk -f
BEGIN {
FS=";"
}
{
x[$2*1"-"$4*1]=x[$2*1"-"$4*1] " " $32 * 1000 + $1"-1"ORS;
x[$5*1"-"$7*1]=x[$5*1"-"$7*1] " " $32 * 1000 + $1"-2"ORS;
x[$8*1"-"$10*1]=x[$8*1"-"$10*1] " " $32 * 1000 + $1"-3"ORS;
x[$11*1"-"$13*1]=x[$11*1"-"$13*1] " " $32 * 1000 + $1"-4"ORS;
x[$14*1"-"$16*1]=x[$14*1"-"$16*1] " " $32 * 1000 + $1"-5"ORS;
x[$17*1"-"$19*1]=x[$17*1"-"$19*1] " " $32 * 1000 + $1"-6"ORS;
x[$20*1"-"$22*1]=x[$20*1"-"$22*1] " " $32 * 1000 + $1"-7"ORS;
x[$23*1"-"$25*1]=x[$23*1"-"$25*1] " " $32 * 1000 + $1"-8"ORS;
x[$26*1"-"$28*1]=x[$26*1"-"$28*1] " " $32 * 1000 + $1"-9"ORS;
x[$29*1"-"$31*1]=x[$29*1"-"$31*1] " " $32 * 1000 + $1"-10"ORS;
}
END {
for ( i in x )
printf "%-3s%-4s\n%-8s\n%-7s\n","UFC ",i,"--------",x[i]
}
output:

Code:
output:

UFC 5-2
--------
 3175-1
 3176-1
 3178-1
 3179-1

UFC 5-1
--------
 3180-1

UFC 14-2
--------
 3177-1
 3182-1

UFC 8-1
--------
 3018-1

UFC 1-2
--------
 3181-1

UFC 4-1
--------
 3175-2
 3176-2
desired output:

Code:
UFC 5-2     UFC 5-1   UFC 14-2
--------   --------   --------
 3175-1     3180-1     3177-1
 3176-1                3182-1
 3178-1
 3179-1
I hope you get the idea of the desired output, too hard to put everything in thread. I want the output to be in a table format, and sorted by UFC numbers from left to right.

Thanks in advance.
I think the trick is related to ORS. I remember I had a similar case when I had to put the output horizontally.
As far as I remember it was something like that

............................{ORS=","}{print variable}

for all the lines I applied this rule
Reply With Quote
  #6 (permalink)  
Old 05-19-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Thanks elthox, this will take me in a new direction that I think I can work with. I don't know if I need to start a new thread but there are 2 more issues that I need help with.

1) What would be the best way to sort the arrays? i.e. (UFC 1-1 UFC 1-2, UFC 2-1, etc.) as they output?

2) This script output a "null" array, basically it creates a all others that I don't want it to output. Basically all the null fields in the CSV file get bunch together. How do I get rid of that?

example:

UFC 0-0 #<- This should not be printed
--------
3001-3 3001-4 3001-5
Reply With Quote
  #7 (permalink)  
Old 05-19-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
awkologist,
I figured out number 2 by changing the END statement from:
Code:
END {
for ( i in x )
printf "%-3s%-4s\n%-8s\n%-7s\n","UFC ",i,"--------",x[i]
}
to:

Code:
END {
for ( i in x )
if ( i != "0-0" ) {
printf "\n%-3s%-4s\n%-8s\n%-7s\n","UFC ",i,"--------",x[i] }
}

I know I should have figured out sooner, but getting burnt out on this. If anyone has a quick down & dirty way to sort the arrays that would be great, I am pretty sure it can be done with for loops, but there might be a easier way.

Thanks
Reply With Quote
  #8 (permalink)  
Old 05-19-2008
Moderator
 

Join Date: Feb 2007
Posts: 1,394
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Piping the output to sort?
Reply With Quote
  #9 (permalink)  
Old 05-19-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
If I do that, the headers get moved around and don't "line-up" with the awk elements.
Reply With Quote
  #10 (permalink)  
Old 05-19-2008
Moderator
 

Join Date: Feb 2007
Posts: 1,394
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
O my, I only read the last 2 post and did'nt look at the desired output, I apologize.
Reply With Quote
  #11 (permalink)  
Old 05-19-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
does anyone know how to use the system command in awk? Been trying to sort with the following:

Code:
END {
for ( i in x )
if ( i != "0-0" ) {
printf "%s",i | system("sort -nk 1") }
can't get it working so that I can sort these arrays
Reply With Quote
  #12 (permalink)  
Old 05-19-2008
rubin's Avatar
Registered User
 

Join Date: Nov 2007
Posts: 134
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Use sort without the AWK system function:

Code:
...
printf "%s",i | "sort -nk 1" }
Reply With Quote
  #13 (permalink)  
Old 05-20-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Thanks ruben.

I am still having issues with sorting these arrays with awk. This is what I have now:

Code:
END {
for ( i in x )
if ( i != "0-0" ) {
printf "%s\n",i | "sort -nk 1 > /tmp/a.out 2>&1"}
while ( (getline < "/tmp/a.out") > 0 )
sort[$1]
for ( s in sort )
printf "%s\n",s }
I was hoping that awk printed out indices in the order received. When I check "/tmp/a.out" indices ARE in sorted order, but when I re-assign and loop thru them, they are not.

Any awkologist that can help would be very much appreciated.

Thanks
Reply With Quote
  #14 (permalink)  
Old 05-20-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 2,253
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
for (s in sort) basically randomizes the order in which they are processed. So, as the doctor said, "don't do that then".
Reply With Quote
  #15 (permalink)  
Old 05-20-2008
Registered User
 

Join Date: Jan 2008
Posts: 60
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!Reddit! Stumble this Post!Spurl this Post!
Thanks for the reply, if the pipe to sort and getline command won't work, is there another way to sort this for ouput?
Reply With Quote
Google UNIX.COM
Reply

Tags
linux

Thread Tools
Display Modes


The 50 most popular UNIX and Linux searches.
Google Search Cloud for The UNIX and Linux Forums
"inappropriate ioctl for device" 421 service not available, remote server has closed connection ^m autosys awk trim bash eval bash exec bash for loop boot: cannot open kernel/sparcv9/unix close_wait command copy/move folder in unix curses.h