Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Problem with python script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Problem with python script
# 1  
Old 11-19-2012
Problem with python script
I have a txt file of 8GB with same type of messages in it. for eg:
Code:
MTQ_BQUOTE, Length: 40, Timestamp: 4:00:52.064
MsgKey: symbol: XXX | reportingExchange: 11
Symbol: XXX, hash 004C5746
QS Symbol: XXX, market 1
Security Type: EQUAL (1)
Symbol Type: Equity.Share.Single.None
Session: XX_XXX (0)
Ticker Exchange: ABCD (11) => 11
Flags: x00000000
Bid: 2475, frac: 2, pure7 val: 24.75
Bid Size: 50
Ask: 0, frac: 0, pure7 val: ---
Ask Size: 0
Quote Condition: x00

There are tons of same messages with different values in the file. Can you please help me with a python script that can saperate the whole message with unique Flags value in an output file.
Last edited by jim mcnamara; 11-19-2012 at 06:20 PM..
# 2  
Old 11-19-2012
Does it have to be python? There are other suitable tools.
# 3  
Old 11-20-2012
UNIX?
It does not necessarily be python. I use a tool called Cygwin (UNIX like). Can it be UNIX command?
# 4  
Old 11-20-2012
can you explain more about " can saperate the whole message with unique Flags value in an output file."
# 5  
Old 11-20-2012
Unique Flags value
Quote:
Originally Posted by itkamaraj
can you explain more about " can saperate the whole message with unique Flags value in an output file."
I have a text file that has several messages, here I put 3 examples:

Code:
MTR_MKTSTAT, Length: 104, Timestamp: 3:30:00.000
MsgKey: statId: @ | type: T | basis: I | exchgId: O | modifier: T
Symbol: @XXX, hash 4F495415
QS Symbol: @XXXX, market 236
Security Type: MKTSTATS (8)
Symbol Type: Statistic.Share.Single.None
Session: US_Day (3)
Ticker Exchange: DTN (254) => 254
Flags: x00000000: 
Description: NASDAQ 3RD MKT ISSUES TICKS TOTAL 
Last: 446, frac: 0, pure7 val: 446
Open: 146, frac: 0, pure7 val: 146
High: 446, frac: 0, pure7 val: 446
Low: 146, frac: 0, pure7 val: 146
Close: 446, frac: 0, pure7 val: 446
Change: 0, frac: 0, pure7 val: --- 
Market Status: x01
Last Trade Time: 16:05:00.000
Last Trade Date: 10/4/2012
Open Time: 9:31:00.000
Close Time: 16:05:00.000
 
MTR_MKTSTAT, Length: 104, Timestamp: 3:30:00.000
MsgKey: statId: @ | type: T | basis: R | exchgId: O | modifier: T
Symbol: @TROT, hash 4F525415
QS Symbol: @TROT, market 236
Security Type: MKTSTATS (8)
Symbol Type: Statistic.Share.Single.None
Session: US_Day (3)
Ticker Exchange: DTN (254) => 254
Flags: x00000000: 
Description: NASDAQ 3RD MKT TICKS RATIO              
Last: 119, frac: 2, pure7 val: 1.19
Open: 106, frac: 2, pure7 val: 1.06
High: 124, frac: 2, pure7 val: 1.24
Low: 75, frac: 2, pure7 val: 0.75
Close: 119, frac: 2, pure7 val: 1.19
Change: 0, frac: 2, pure7 val:  --- 
Market Status: x01
Last Trade Time: 16:05:00.000
Last Trade Date: 10/4/2012
Open Time: 9:31:00.000
Close Time: 16:05:00.000
 
MTR_MKTSTAT, Length: 104, Timestamp: 3:30:01.000
MsgKey: statId: @ | type: I | basis: P | exchgId: T | modifier: T
Symbol: @XXX, hash 54504915
QS Symbol: @XXXX, market 236
Security Type: MKTSTATS (8)
Symbol Type: Statistic.Share.Single.None
Session: US_Day (3)
Ticker Exchange: DTN (254) => 254
Flags: x00000010: LAST 
Description: 
Last: 1396, frac: 0, pure7 val: 1396
Open: 5, frac: 0, pure7 val: 5
High: 1396, frac: 0, pure7 val: 1396
Low: 3, frac: 0, pure7 val: 3
Close: 0, frac: 0, pure7 val: --- 
Change: 1396, frac: 0, pure7 val: 1396
Market Status: x02
Last Trade Time: 3:30:01.000
Last Trade Date: 10/5/2012
Open Time: 0:00:00.000
Close Time: 23:59:59.000


If you look at the 9th line of each message, there is Flags with a code like x00000000. But the first 2 messages have the same code so, I want to saperate only those messages that have a unique code in a text file.
Last edited by Scrutinizer; 11-20-2012 at 04:37 PM.. Reason: Wanted to put 3 example messages but accidently put only 2. Therefore added third message.; mod: code tags
# 6  
Old 11-20-2012
If you just want to filter out one type, and if messages are separated by at least one blank line:
Code:
awk 'index($9, "code_to_match")' RS=

Regards,
Alister
# 7  
Old 11-21-2012
The below code creates the file with flagname (with the appropriate message block)

Code:
 
 $ nawk '/MTR/,/Close Time/{a[i++]=$0;if($0~/Flags:/){sub(":","",$2);fname=$2}if($0~/Close Time/){for(j=0;j<=i;j++)print a[j] >> fname;i=0}}' input.txt

In Solaris, use nawk, otherwise you can use awk

Above code will create two files ( x00000000, x00000010 ) - for the sample text you posted.
Last edited by itkamaraj; 11-21-2012 at 01:36 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Windows & DOS: Issues & Discussions

How to execute python script on remote with python way..?

Hi all, I am trying to run below python code for connecting remote windows machine from unix to run an python file exist on that remote windows machine.. Below is the code I am trying: #!/usr/bin/env python import wmi c = wmi.WMI("xxxxx", user="xxxx", password="xxxxxxx")... (1 Reply)
Discussion started by: onenessboy
1 Replies

2. Shell Programming and Scripting

Python problem with input

Hello everyone, I have just started python, and there is something that escapes me. I don't understand why my little script doesn't work. When I do, it work: my_string = "bonjour" if len(my_string) < "6": print(my_string + " < 6") else: print(my_string + " > 6") ... (2 Replies)
Discussion started by: Arnaudh78
2 Replies

3. Shell Programming and Scripting

Capture run time of python script executed inside shell script

I have bash shell script which is internally calling python script.I would like to know how long python is taking to execute.I am not allowed to do changes in python script.Please note i need to know execution time of python script which is getting executed inside shell .I need to store execution... (2 Replies)
Discussion started by: Adfire
2 Replies

4. Shell Programming and Scripting

Python SSH problem

Hello geeks, Am trying to write a custom plugin using python but it seems python is not handling the SSH part well, find below for the code: #!/usr/bin/python import os, sys host = sys.argv pdpactgsm=os.popen("ssh -l GbeAdi $host "pdc_kpi.pl" | grep -A 4 sgsn_g | awk 'NR == 5 ' |... (1 Reply)
Discussion started by: infinitydon
1 Replies

5. AIX

Strange problem running python program from within ant

Hello, I'm currently trying to port an ant based build to AIX 6.1. The build queries the underlying version control system (Mercurial) for some data, which works fine on other platforms (Linux, Solaris). However, on AIX the hg command fails to load python's md5 module when used in the build,... (1 Reply)
Discussion started by: dhs
1 Replies

6. Shell Programming and Scripting

Python script called by a shell script

experts, i wrote a python script to do a certain job, i tried it and it is working fine, i want this script to be executed automatically after a ksh script, the problem is when i execute the ksh script my python script runes perfectly after the ksh script as I have include it at the end of the ksh... (1 Reply)
Discussion started by: q8devilish
1 Replies

7. Shell Programming and Scripting

Problem passing arguments to a Python script

I have part of the script below and I tried calling the script using ./tsimplex.py --fstmod=chris.cmod --nxz=8x6 --varp=0.25 but am getting the error option --fstmod must not have an argument Any idea on how to fix this would be highly appreciated #! /usr/bin/python import... (0 Replies)
Discussion started by: kristinu
0 Replies

8. Shell Programming and Scripting

Passing variable from shell script to python script

I have a shell script main.sh which inturn call the python script ofdm.py, I want to pass two variables from shell script to python script for its execution. How do i achieve this ????? Eg: main.sh a=3 b=3; c= a+b exec python ofdm.py ofdm.py d=c+a Thanks in Anticipation (4 Replies)
Discussion started by: shashi792
4 Replies

9. UNIX for Dummies Questions & Answers

problem with python on mac

Hi, I'm sure there's a very easy way to fix this but i'm a newbie so I'm posting this. The problem is that whenever I run python on the terminal I can no longer use the left and right arrows to move left or right, or the up/down arrows to call commands used before. Whenever I click an arrow I get... (0 Replies)
Discussion started by: thefloydpink
0 Replies

10. Shell Programming and Scripting

Python tkinter grid problem?

There seems to be some problem with the grid geometry manager, or maybe its just my syntax thats incorrect. Im using Python 2.3.4. My problem is that even when I use the sticky='news' option in any grid method, it just doesnt work period. For example, the following code gives be a window with a... (0 Replies)
Discussion started by: SRCoder!
0 Replies
Login or Register to Ask a Question