parsing data file picking out certain fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting parsing data file picking out certain fields
# 1  
Old 02-28-2008
can someone take a quick look at my code? Having issues figuring this out.

I have a file that is large and is broken up by groups of data. I want to take certain fields and display them different to make it easier to read. Given input file below:

HTML Code:
2008   fl01   LAC   2589   polk   doal
xx 2008q1 mx
     sect 25698541

     Sales 08 Dept group

        lead1    2008q1
        tom
        pat
        tim
        tad

        lead1  07q4   07q3   07q2   07q1   06q4   06q3   jan
        tom    0      96     0      3312   3624   0      312
        pat    0      17     0      0      30     0      30
        tim    357    03     04     25     3020   3120   20
        tad    1734   0      0      0      5213   5213   0

        lead1  feb    mar    apr    may    jun    jul    aug
        tom    0      96     0      0      0      0      0
        pat    0      17     0      0      0      0      0
        tim    357    23     5      7      8      14     70
        tad    1734   0      0      0      0      0      0

        lead1  sept   oct    nov    dec
        tom    0      0      460    92
        pat    0      0      240    0
        tim    0      21     1800   0
        tad    0      0      672    0

2008   fl01  LAC   2589    polk   doal
yy 2008q1 mx
     sect 2569852

     Sales 08 Dept group
I would like to display:

HTML Code:
lead1   07q4    07q1    06q4    06q3    sept    oct     nov
tim	357	25	3020	3120	0	21	1800 
tad	1734	0	5213	5213	0	0	672
Can anyone help out on this? I can't seem to get this right.
Thanks in advance.

Last edited by timj123; 03-03-2008 at 07:26 PM..
# 2  
Old 02-28-2008
Would this need to be reusable for some different data? It's hard to make sense of this currently. Thanks
# 3  
Old 02-28-2008
I don't understand your question. But I wanted the input file to print out differently. Problem I was having was that the id are slit between blank lines and data I wanted to parse have different NF. I was trying awk and going the route of arrays with no luck, because while some records I want $2 on other records $2 might be a field I don't want.

Does this make sense??

And thank you for your time.

Last edited by timj123; 02-29-2008 at 09:13 AM..
# 4  
Old 02-29-2008
Can a expert take a look at this, still having issues figuring this out.

thanks
# 5  
Old 03-01-2008
Can someone check my code below and help me fix? I can't seem to get this right. I am stuck, any help in making this easier for me, I would really appreciate it.

thanks

Code:
   $3 ~ /mx$/ && !s12 {
        printf "sales dept 08                                    %s %s\n",time,date
        print "-----------------"
        print "lead1   07q4    07q3    07q2  07q1  06q4    06q3  jan   nov   dec"
        s12++
        }
          s12 {
                if ( $5 ~ /07q1/ ) {
                   f1++
                   }
                if ( f1 ) {
                        s1[$1"_ld "]=$1 ; s1[$1"_7q4 "]=$2 ; s1[$1"_7q3 "]=$3
                        s1[$1"_7q2 "]=$4; s1[$1"_7q1 "]=$5 ; s1[$1"_6q4 "]=$6
                        s1[$1"_6q3 "]=$7 ; s1[$1"_jan "]=$8
                          for ( i in s1 ) { printf "|%s",i s1[i] }
                   }
                if ( ! NF ) {
                        f1=""
                   }
                if ( $5 ~ /dec$/ ) {
                        f3++
                   }
                if ( f3 ) {
                        s3[$1"_nov "]=$4 ; s3[$1"_dec  "]=$5
                           for ( i in s3 ) { printf "|%s",i s3[i] }
                   }
                if ( ! NF ) {
                        f3=""
                   }
                if ( $3 ~ /my/ ) {
                        print "\n"
                        f3=s12=""
                   }
              }

# 6  
Old 03-03-2008
anybody here?
# 7  
Old 03-04-2008
Hi,

A complex one, but should be ok for you.

Code:
grep -n lead e | cut -d":" -f1 > temp
for i in `cat temp`
do
t=`expr $i + 4`
sed -n "$i,${t}p" e >> 000${i}
done
rm temp
for i in `ls -l 000* | awk '{print $9}'`
do
	sort $i | awk '{
				if(NF<2)
				{
					NF=2
				}
				for(j=1;j<=NF;j++)
				{
					if($j=="")
						$j=0
					printf("%s ",$j)
				}
				printf("\n")
			}'> ${i}.temp
	rm $i
	mv ${i}.temp $i
done
ls -l 000* | awk '{print $9}' | sort -n | paste - - > tt
n=0
while read line
do
	n=`expr $n + 1`
	join $line > t.$n
done < tt
join t.* > final
rm 000*
rm t*
awk '{
if ($1=="tim" || $1=="tad" || $1=="lead1")
print $1" "$3" "$4" "$7" "$8" "$17" "$18" "$19
}' final
rm final

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

2. Shell Programming and Scripting

Parsing C Data Tipes from Input File

Im really beginner in this case, maybe someone can help me find the answer: if my input file like this: void main(int a, int b){ int x; double y; printf("file"); } and i want output like this: int a int b int x double y A awk script that can parse only data tipe, im confused. what... (2 Replies)
Discussion started by: radynaraya
2 Replies

3. Shell Programming and Scripting

Parsing data using keys from one file

I have 2 text files where I need to parse data from file 2 using the data from file 1. Below are my sample files File 1 (tab delimited) 257 350 670 845 725 1025 767 820 ... .... .... file 2 (tab delimited) 220..450 TA AB650 ABCED 520..850 GA AB720 ABCDE 700..1100 TC AB820 ABCDE... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

4. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

5. Shell Programming and Scripting

parsing a portion of Data from a text file

Hi All, I need some help to effectively parse out a subset of results from a big results file. Below is an example of the text file. Each block that I need to parse starts with "Output of GENE for sequence file 100.fasta" (next block starts with another number). I have given the portion of... (8 Replies)
Discussion started by: Lucky Ali
8 Replies

6. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi, I have a folder that contains many (multiple) files 1.fasta 2.fasta 3.fasta 4.fasta 5.fasta . . 100's of files Each such file have data in the following format for example: vi 1.fasta >AB_1 gi|15835212|ref|NP_296971.1| preprotein translocase subunit SecE... (3 Replies)
Discussion started by: Lucky Ali
3 Replies

7. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi All I have two files: file 1 >AB_1 MLKKPIIIGVTGGSGGGKTSVSRAILDSFPNARIAMIQHDSYYKDQSHMSFEERVKTNYDHPLAFDTDFM IQQLKELLAGRPVDIPIYDYKKHTRSNTTFRQDPQDVIIVEGILVLEDERLRDLMDIKLFVDTDDDIRII RRIKRDMMERGRSLESIIDQYTSVVKPMYHQFIEPSKRYADIVIPEGVSNVVAIDVINSKIASILGEV >AB_2... (5 Replies)
Discussion started by: Lucky Ali
5 Replies

8. Shell Programming and Scripting

urgent<parsing data from a excel file>

Hi all, I wud like to get ur assistance in retrieving lines containing l1.My excel dataset contains around 8000 lines.I converted it into a text tab delimiter file and got the lines containing l1,My output is a list of lines containing l1 saved in a outfile.Some of d lines from my outfile s... (5 Replies)
Discussion started by: sayee
5 Replies

9. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies

10. Shell Programming and Scripting

Parsing file and extracting the useful data block

Greetings All!! I have a very peculiar problem where I have to parse a big text file and extract useful data out of it with starting and ending block pattern matching. e.g. I have a input file like this: sample data block1 sample data start useful data end sample data block2 sample... (5 Replies)
Discussion started by: arminder
5 Replies
Login or Register to Ask a Question