Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google site



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Closed Thread
English Japanese Spanish French German Portuguese Italian Powered by Powered by Google
 
Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 04-17-2009
Registered User
 

Join Date: Sep 2006
Posts: 7
help with ksh/awk/sed script, random # of fields

Hello all, I'm working on an attendance callout script for a school district. I need to change our current layout for the vendor. Currently the data is in the form of:
studentid,period,building,
Heres a sample of some made up records:
500,1,30,
500,2,30,
500,3,30,
500,6,30,
7899,2,31,
9021,1,33,
9021,6,33,
907711,5,40,
907711,6,40,
I need to reformat this, omitting the header row, to look like the following:
500,1,2,3,6,30,
7899,2,31,
9021,1,6,33,
907711,5,6,40,
I've done stuff like this with awk in the past when the number of fields were a constant. Just not sure how to loop through this when the number of fields vary. The kids can have an unexcused in any number of periods throughout the day. Some of our buildings have over 9 periods configured for attendance purposes as well.

If anyone has some code to get me going in the right direction that would be great.

I will read the file first then do something like this to compare the current record with the next record to verify its the same kid. (beyond that I'm not sure how to proceed)


Code:
awk -F, '
$1 == lastid {
         do something
}
$1 != lastid {
        lastid=$1
        do something
}'

Thanks,
Sponsored Links
  #2 (permalink)  
Old 04-17-2009
awk awk is offline
Registered User
 

Join Date: Feb 2007
Posts: 134

Code:
awk -F, -v OFS=, '{ for (I=2; I<NF; I++)
                    {print $1, $I}
                  }' <<END |\
sort -kn1,1 -kn2,2 -t, -u |\
awk -F, 'NR==1{Save0=$1}
         Save0 == $1{Line=Line "," $2; next}
         Save0 != $1 {print Save0  Line;
                      Line="," $2;
                      Save0=$1;
                     }
         END{print Save0  Line;}'
500,1,30,
500,2,30,
500,3,30,
500,6,30,
7899,2,31,
9021,1,33,
9021,6,33,
907711,5,40,
907711,6,40,

Produced


Code:
500,1,2,3,6,30
7899,2,31
9021,1,6,33
907711,5,6,40

  #3 (permalink)  
Old 04-17-2009
Registered User
 

Join Date: Sep 2006
Posts: 7
thanks!

Thanks a lot awk, I'll get to give your code a go on Monday first thing.
  #4 (permalink)  
Old 04-18-2009
Moderator
 

Join Date: Feb 2007
Location: The Netherlands
Posts: 4,962
Assuming your data is sorted:


Code:
awk -F, '
NR==1{s=$1 FS $2; b=$3; next}
b!=$3{print s FS b; s=$1 FS $2; b=$3; next}
{s=s FS $2}
END{print s FS b}' file

Regards
  #5 (permalink)  
Old 04-20-2009
Registered User
 

Join Date: Jun 2007
Location: Beijing China
Posts: 1,133

Code:
nawk -F"," '{
if (_[$1","$3]==""){
  _[$1","$3]=$2
  next
}
_[$1","$3]=sprintf("%s,%s",_[$1","$3],$2)
}
END{
for (i in _){
 split(i,arr,",")
 print arr[1]","_[i],","arr[2]
} 
}' filename

  #6 (permalink)  
Old 04-21-2009
Registered User
 

Join Date: Sep 2006
Posts: 7
Quote:
Originally Posted by awk View Post
Code:
awk -F, -v OFS=, '{ for (I=2; I<NF; I++)
                    {print $1, $I}
                  }' <<END |\
sort -kn1,1 -kn2,2 -t, -u |\
awk -F, 'NR==1{Save0=$1}
         Save0 == $1{Line=Line "," $2; next}
         Save0 != $1 {print Save0  Line;
                      Line="," $2;
                      Save0=$1;
                     }
         END{print Save0  Line;}'
500,1,30,
500,2,30,
500,3,30,
500,6,30,
7899,2,31,
9021,1,33,
9021,6,33,
907711,5,40,
907711,6,40,

Thanks awk, works great for me. I was wondering what the <<END is in the first awk command? Is it like a here doc? I'm trying to understand this better to figure out what order the code is being executed in.

thanks again
  #7 (permalink)  
Old 04-21-2009
awk awk is offline
Registered User
 

Join Date: Feb 2007
Posts: 134
Ideally, I should have had the word END on a line by itself after the data. It shows the korn shell where to stop the input of data to the awk program.

Still the script terminated at that point, the shell script figures out the END is implied.

Sorry, I will remember to put it in future posts.
Sponsored Links
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
random script termination mph Shell Programming and Scripting 6 07-18-2008 09:03 AM
how do i generate random integer using only shell script sridhusha Shell Programming and Scripting 1 12-31-2007 03:46 AM
random function script prash_b Shell Programming and Scripting 4 09-15-2006 05:54 AM
Random files do not FTP in the shell script RLatham20 Shell Programming and Scripting 1 06-29-2006 01:07 PM
Random parameters passing in FTP script sourabhshakya Shell Programming and Scripting 2 06-12-2006 12:20 PM



All times are GMT -4. The time now is 03:32 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2010. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0