|
|||||||
| Forums | Search Forums | Register | Forum Rules | Man Pages | Albums | FAQ | Members | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
|
|
|
Thread Tools | Search this Thread | Display Modes |
|
#1
|
|||
|
|||
|
Problem with python script
I have a txt file of 8GB with same type of messages in it. for eg: Code:
MTQ_BQUOTE, Length: 40, Timestamp: 4:00:52.064 MsgKey: symbol: XXX | reportingExchange: 11 Symbol: XXX, hash 004C5746 QS Symbol: XXX, market 1 Security Type: EQUAL (1) Symbol Type: Equity.Share.Single.None Session: XX_XXX (0) Ticker Exchange: ABCD (11) => 11 Flags: x00000000 Bid: 2475, frac: 2, pure7 val: 24.75 Bid Size: 50 Ask: 0, frac: 0, pure7 val: --- Ask Size: 0 Quote Condition: x00 There are tons of same messages with different values in the file. Can you please help me with a python script that can saperate the whole message with unique Flags value in an output file. Last edited by jim mcnamara; 11-19-2012 at 04:20 PM.. |
| Sponsored Links | ||
|
|
#2
|
|||
|
|||
|
Does it have to be python? There are other suitable tools.
|
| Sponsored Links | ||
|
|
#3
|
|||
|
|||
|
UNIX?
It does not necessarily be python. I use a tool called Cygwin (UNIX like). Can it be UNIX command?
|
|
#4
|
||||
|
||||
|
can you explain more about " can saperate the whole message with unique Flags value in an output file."
|
| Sponsored Links | |
|
|
#5
|
|||
|
|||
|
Unique Flags value
Quote:
Code:
MTR_MKTSTAT, Length: 104, Timestamp: 3:30:00.000 MsgKey: statId: @ | type: T | basis: I | exchgId: O | modifier: T Symbol: @XXX, hash 4F495415 QS Symbol: @XXXX, market 236 Security Type: MKTSTATS (8) Symbol Type: Statistic.Share.Single.None Session: US_Day (3) Ticker Exchange: DTN (254) => 254 Flags: x00000000: Description: NASDAQ 3RD MKT ISSUES TICKS TOTAL Last: 446, frac: 0, pure7 val: 446 Open: 146, frac: 0, pure7 val: 146 High: 446, frac: 0, pure7 val: 446 Low: 146, frac: 0, pure7 val: 146 Close: 446, frac: 0, pure7 val: 446 Change: 0, frac: 0, pure7 val: --- Market Status: x01 Last Trade Time: 16:05:00.000 Last Trade Date: 10/4/2012 Open Time: 9:31:00.000 Close Time: 16:05:00.000 MTR_MKTSTAT, Length: 104, Timestamp: 3:30:00.000 MsgKey: statId: @ | type: T | basis: R | exchgId: O | modifier: T Symbol: @TROT, hash 4F525415 QS Symbol: @TROT, market 236 Security Type: MKTSTATS (8) Symbol Type: Statistic.Share.Single.None Session: US_Day (3) Ticker Exchange: DTN (254) => 254 Flags: x00000000: Description: NASDAQ 3RD MKT TICKS RATIO Last: 119, frac: 2, pure7 val: 1.19 Open: 106, frac: 2, pure7 val: 1.06 High: 124, frac: 2, pure7 val: 1.24 Low: 75, frac: 2, pure7 val: 0.75 Close: 119, frac: 2, pure7 val: 1.19 Change: 0, frac: 2, pure7 val: --- Market Status: x01 Last Trade Time: 16:05:00.000 Last Trade Date: 10/4/2012 Open Time: 9:31:00.000 Close Time: 16:05:00.000 MTR_MKTSTAT, Length: 104, Timestamp: 3:30:01.000 MsgKey: statId: @ | type: I | basis: P | exchgId: T | modifier: T Symbol: @XXX, hash 54504915 QS Symbol: @XXXX, market 236 Security Type: MKTSTATS (8) Symbol Type: Statistic.Share.Single.None Session: US_Day (3) Ticker Exchange: DTN (254) => 254 Flags: x00000010: LAST Description: Last: 1396, frac: 0, pure7 val: 1396 Open: 5, frac: 0, pure7 val: 5 High: 1396, frac: 0, pure7 val: 1396 Low: 3, frac: 0, pure7 val: 3 Close: 0, frac: 0, pure7 val: --- Change: 1396, frac: 0, pure7 val: 1396 Market Status: x02 Last Trade Time: 3:30:01.000 Last Trade Date: 10/5/2012 Open Time: 0:00:00.000 Close Time: 23:59:59.000 If you look at the 9th line of each message, there is Flags with a code like x00000000. But the first 2 messages have the same code so, I want to saperate only those messages that have a unique code in a text file. Last edited by Scrutinizer; 11-20-2012 at 02:37 PM.. Reason: Wanted to put 3 example messages but accidently put only 2. Therefore added third message.; mod: code tags |
| Sponsored Links | |
|
|
#6
|
|||
|
|||
|
If you just want to filter out one type, and if messages are separated by at least one blank line: Code:
awk 'index($9, "code_to_match")' RS= Regards, Alister |
| Sponsored Links | |
|
|
#7
|
||||
|
||||
|
The below code creates the file with flagname (with the appropriate message block) Code:
$ nawk '/MTR/,/Close Time/{a[i++]=$0;if($0~/Flags:/){sub(":","",$2);fname=$2}if($0~/Close Time/){for(j=0;j<=i;j++)print a[j] >> fname;i=0}}' input.txtIn Solaris, use nawk , otherwise you can use awk Above code will create two files ( x00000000, x00000010 ) - for the sample text you posted. Last edited by itkamaraj; 11-20-2012 at 11:36 PM.. |
| Sponsored Links | ||
|
![]() |
| Thread Tools | Search this Thread |
| Display Modes | |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Problem passing arguments to a Python script | kristinu | Shell Programming and Scripting | 0 | 01-18-2011 04:58 PM |
| Passing variable from shell script to python script | shashi792 | Shell Programming and Scripting | 4 | 12-02-2010 05:20 AM |
| problem with python on mac | thefloydpink | UNIX for Dummies Questions & Answers | 0 | 02-11-2009 11:19 AM |
| Python tkinter grid problem? | SRCoder! | Shell Programming and Scripting | 0 | 06-19-2004 11:49 AM |
|
|