Sorting multi-column values from a specific file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sorting multi-column values from a specific file
# 1  
Old 09-21-2010
Bug Sorting multi-column values from a specific file

Hi, all.
I need a shell script which gathers data from a remote XML file and then displays it according to my needs.. I need this for my job due to the fact that I need to keep track price changes of euro, usd, gold, etc.
The XML file I am talking about is located at this page: cnnturk dot com/finans/ticker/endeks.asp
The reason I am posting the URL is that I need to use curl to get this file and it does NOT have newlines after each tag. I thought that that would be a problem.
Here is what I need from the script:
1) curl to get the page
2) make use of sed, awk, etc. to display its contents in a more structured and readable manner as shown below:
A: 64.125 (% -0.26)
B: 81.130 (% -0.32)
C: 1.4930 (% 0.00)
D: 1.9590 (% 0.36)

I am looking forward to your insights..
Thanks!
# 2  
Old 09-21-2010
I'm not familiar with the syntax of churl, but assume that you can fetch the XML into a file to parse with awk. This is an example of one way to accomplish your task.

Code:
#!/usr/bin/env ksh

awk -v RS=">" '                      # we dont need closing tag mark; make it the record seperator
        /Percent/ {                    # data we are interested in has this field
                data = 0;
                gsub( " = ", "=" );             # easier to work with if there are no spaces round equal signs
                gsub( "\"", "" );               # for this case we dont expect blanks in the value so ditch quote marks
                for( i = 1; i < NF; i ++ )     # for each token that is var=value
                {
                        if( (x = split( $(i), a, "=" )) == 2 )   # split and save the value
                        {
                                data++;           # may not be necessary, but safe
                                var[a[1]] = a[2];
                        }
                }

                if( data )                    # if we saw data, print it
                {
                        printf( "%c %6.3f %6.3f%%\n", 65+idx, var["Price"], var["Percent"]);
                        idx++;
                }
        }' <current.xml

Change the printf() statement to print it in the format you need.
# 3  
Old 09-21-2010
this is perfect.. thank you very much, agama. you really helped me out... this is what I was looking.. well, one last thing, how can i store the value of DateTime attribute in a variable??
thanks "again"!!!
# 4  
Old 09-22-2010
Snarfing the date/time makes it a bit more complicated as the value has a space inside of the quotes. So, rewriting things a bit to allow for multiple tokens within quotes consider this brute force parsing example. You can use the date/time value as you need to. This just prints it out when it finds it.

Code:

#!/usr/bin/env ksh

awk -v RS=">" '

        function snarf_vars( )           # capture all name=value pairs on the current line. 
        {
                delete var;
                gsub( " = ", "=" );             # easier to deal without spaces
                for( i = 1; i <= NF; i ++ )
                {
                        if( (x = split( $(i), a, "=" )) == 2 )
                        {
                                $(i) = a[2];
                                for( ; i <= NF; i++ )     # snarf up all tokens in the quotes
                                {
                                        var[a[1]] = var[a[1]] $(i) " ";    # capture next token
                                        if( match( $(i), /"$/ ) )             # this has ending quote 
                                        {
                                                gsub( "\"", "", var[a[1]] );  # one replace of all quotes (does not handle escaped quotes)
                                                break;
                                        }
                                }
                        }
                }
        }

        /DateTime/{
                snarf_vars();
                printf( "%s \n", var["DateTime"] );
        }

        /Percent/ {
                snarf_vars();
                if( var["Price"] )
                        printf( "%c %6.3f %6.3f%%\n", 65+(idx++), var["Price"], var["Percent"]);
        }' <test.xml

The function unsets the var array, so if you want the date/time value later you need to specifically save it or it will be gone. This also makes no attempt to preserve the whitespace that it might find between quotes; for your needs that'll be OK, but if you want to use this in other situations it might not do the job.

Hope this helps.

Last edited by agama; 09-22-2010 at 01:31 AM.. Reason: better, shorter, more simple, snarf function
# 5  
Old 09-22-2010
Quote:
Originally Posted by canimsin
2) make use of sed, awk, etc. to display its contents in a more structured and readable manner as shown below:
A: 64.125 (% -0.26)
B: 81.130 (% -0.32)
C: 1.4930 (% 0.00)
D: 1.9590 (% 0.36)
Base on today's data sample:
Code:
awk -F\" '{for(i=1;++i<NF;){if($i~"Percent"){x=$(i+1)}if($i~"Price"){printf "%c: %s (%s %s)\n",65+_,$(i+1),"%",x;_++}}}' endeks.asp
A: 64.264 (% 0.22)
B: 81.470 (% 0.42)
C: 1.4840 (% -0.60)
D: 1.9840 (% 1.28)

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

DB2 Query -Convert multi values from column to rows

Hi Team I am using DB2 artisan tool and struck to handle multi values present in columns that are comma(,) separated. I want to convert those column values in separate rows . For example : Column 1 Column2 Jan,Feb Hold,Sell,Buy Expected Result Column1 ... (3 Replies)
Discussion started by: Perlbaby
3 Replies

2. Shell Programming and Scripting

Overwrite specific column in xml file with the specific column from adjacent line

I have an xml file dumped from rrd file, that I want to "patch" so the xml file doesn't contain any blank hole in the resulting graph of the rrd file. Here is the file. <!-- 2015-10-12 14:00:00 WIB / 1444633200 --> <row><v> 4.0419731265e+07 </v><v> 4.5045912770e+06... (2 Replies)
Discussion started by: rk4k
2 Replies

3. Shell Programming and Scripting

Sorting a specific column!

What im trying to do is sort the output by the number on the second column and than limit the result to only the first three lines. This is the code idlist="x23s52; f34233; 2343xs; 25x34; si342d" cntr=1 idcnt=$(print $nidlist |tr ';' '\n' |wc -l) numofgrps=0 while (($cntr <= $idcnt))... (3 Replies)
Discussion started by: ajetangay
3 Replies

4. Shell Programming and Scripting

Sum values of specific column in multiple files, considering ranges defined in another file

I have a file (let say file B) like this: File B: A1 3 5 A1 7 9 A2 2 5 A3 1 3 The first column defines a filename and the other two define a range in that specific file. In the same directory, I have also three more files (File A1, A2 and A3). Here is 10 sample lines... (3 Replies)
Discussion started by: Bastami
3 Replies

5. Shell Programming and Scripting

Count specific column values

Hi all: quick question! I have the following data that resembles some thing like this: i am tired tired am i what is up hello people cool I want to count (or at least isolate) all of the unique elements in the 2nd column. I have tried this: cut -f 2 | uniq 'input' which does... (3 Replies)
Discussion started by: owwow14
3 Replies

6. Shell Programming and Scripting

Extract values from a specific column to the end

Hello friends, I have a text file with many columns (no. columns vary from row to row) separated by space. I need to collect all the values from 18th column to the end from each line and group them as pairs and then numbering like below.. 1. 18th-col-value 19th-col-value 2. 20th-col-value ... (5 Replies)
Discussion started by: prvnrk
5 Replies

7. UNIX for Dummies Questions & Answers

Sorting columns for specific values

Dear All, i have a column with values in excel table: ATGC22327-p66 ATGC15922-p239 ATGC12710-p21743567 ATGC08037-p186 ATGC07969-p173 ATGC07345-p48534 ATGC02767-p254234 ATGC02124-p2177451 ATGC02124-p1459 ATGC01930-p3005 I need to... (6 Replies)
Discussion started by: AAWT
6 Replies

8. Shell Programming and Scripting

How to substract selective values in multi row, multi column file (using awk or sed?)

Hi, I have a problem where I need to make this input: nameRow1a,text1a,text2a,floatValue1a,FloatValue2a,...,floatValue140a nameRow1b,text1b,text2b,floatValue1b,FloatValue2b,...,floatValue140b look like this output: nameRow1a,text1b,text2a,(floatValue1a - floatValue1b),(floatValue2a -... (4 Replies)
Discussion started by: nricardo
4 Replies

9. UNIX for Dummies Questions & Answers

Switching the values in a specific column of a text file

Hi, I have a space de-limited text file. In the fifth column, I would like to switch "1"s with "2"s. How do I go about doing that? Thanks! Sample input: 0 311000259 0 0 1 1 0 311000397 0 0 1 2 0 311000491 0 0 2 1 0 311000516 0 0 2 1 0 311000541 0 0 1 1 0 311000558 0 0 2 1 0 311000566 0... (1 Reply)
Discussion started by: evelibertine
1 Replies

10. UNIX for Dummies Questions & Answers

Assistance with combining, sorting and saving multi files into one new file

Good morning. I have a piece of code that is currently taking multiple files and using the CAT.exe command to combine into one file that is then sorted in reverse order based on the 3rd field of the file, then displayed on screen. I am trying to change this so that the files are being combined into... (4 Replies)
Discussion started by: jaacmmason
4 Replies
Login or Register to Ask a Question