Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Python Cat

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Python Cat
# 1  
Old 10-10-2014
Python Cat
Hi,

Below is the Python Script am trying to execute:

Code:
#!/usr/bin/python

import os
import sys
import subprocess
import time

def mpg(node, sitename):
	os.system('/usr/local/bin/expect /gsn/pdp/pdp_expect ' + node + ' > /tmp/' + node + '.temp')
	date = os.popen("cat /tmp/" + node + ".temp | grep time-sampled: | awk '{print substr($0, index($0,$2))}' | tr -d '\r''").read().strip()
	totalpdp = os.popen("cat /tmp/" + node + ".temp | grep pdp-active: | head -n 1 | awk '{print $2}' | sed 's/[^0-9]*//g'").read().strip()
	total_uplink = os.popen("cat /tmp/" + node + ".temp | head -n 95 | grep bytes | grep -v ipv6 | head -n 1 | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	total_downlink = os.popen("cat /tmp/" + node + ".temp | head -n 95 | grep bytes | grep -v ipv6 | tail -1 | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	total_uplink_packet = os.popen("cat /tmp/" + node + ".temp | head -n 95 | /usr/sfw/bin/ggrep -A 2 uplink: | grep packets: | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	total_downlink_packet = os.popen("cat /tmp/" + node + ".temp | head -n 95 | /usr/sfw/bin/ggrep -A 2 downlink: | grep packets: | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	total_uplink_packet_drop = os.popen("cat /tmp/" + node + ".temp | head -n 95 | /usr/sfw/bin/ggrep -A 3 uplink: | grep dropped-packets: | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	total_downlink_packet_drop = os.popen("cat /tmp/" + node + ".temp | head -n 95 | /usr/sfw/bin/ggrep -A 3 downlink: | grep dropped-packets: | awk '{ print $2 }' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	attempted_bb_act = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 20 blackberry.net | grep pdp-attempted-activation: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	completed_bb_act = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 20 blackberry.net | grep pdp-completed-activation: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	attempted_web_act = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 20 web.gprs.mtnnigeria.net | grep pdp-attempted-activation: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	completed_web_act = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 20 web.gprs.mtnnigeria.net | grep pdp-completed-activation: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	active_pdp_web = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 3 web.gprs.mtnnigeria.net | grep pdp-active: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	active_pdp_bb = os.popen("cat /tmp/" + node + ".temp | /usr/sfw/bin/ggrep -A 3 blackberry.net | grep pdp-active: | awk '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//' | sed 's/[^0-9]*//g'").read().strip()
	
	print "Active PDP WEB IS " + str(active_pdp_web)
	print "Active PDP BB IS " + str(active_pdp_bb)
	
	#os.system('echo ' + str(date) + " Total PDP " + str(totalpdp) + ' > /tmp/' + str(sitename) + '.txt')
	
mpg("mpg", "slos")


But when I run this code,it gives the following error:
Code:
cat: write error: Broken pipe
cat: write error: Broken pipe
cat: write error: Broken pipe
cat: write error: Broken pipe
cat: write error: Broken pipe
cat: write error: Broken pipe

Script works fine with BASH and I have found some information via google that CAT is not easy to use in python, is this true?

Or is there any way I can get rid of the errors?

I still have a lot of lines with CAT that I want to implement and I can't risk going on without solving this.
Last edited by vbe; 10-10-2014 at 12:08 PM..
# 2  
Old 10-10-2014
Hi,
Why use cat file ?
grep,head,... not need to read only one file.

Regards.
This User Gave Thanks to disedorgue For This Post:
infinitydon
# 3  
Old 10-10-2014
Your code isn't Python -- just a transparently-thin shrinkwrapping of Python around 99% pure Bourne shell. I'm not sure why you bothered.

Even for shell code this is atrocious, though. You could do this in one awk call instead of umpteen cat's, sed's, tr's, grep's, and kitchen sinks.

I tried to figure out how many extra processes you're wasting, but I lost count. This is going to be extraordinarily, noticeably slow and wasteful, even on a modern computer. Please, seriously consider rewriting this properly, either using native Python features or properly in the shell.
This User Gave Thanks to Corona688 For This Post:
infinitydon
# 4  
Old 10-10-2014
Quote:
Originally Posted by Corona688
Your code isn't Python -- just a transparently-thin shrinkwrapping of Python around 99% pure Bourne shell. I'm not sure why you bothered.

Even for shell code this is atrocious, though. You could do this in one awk call instead of umpteen cat's, sed's, tr's, grep's, and kitchen sinks.

I tried to figure out how many extra processes you're wasting, but I lost count. This is going to be extraordinarily, noticeably slow and wasteful, even on a modern computer. Please, seriously consider rewriting this properly, either using native Python features or properly in the shell.
Hi Corona688,

Thanks for your candid opinion, much appreciated. Am still a beginner, I have no programming experience, am just learning on my own via tutorials on the internet.

Be it as it may, my challenge remains "How do I optimize this code seeing that the input file alone from EXPECT statement is over 800 lines and I need to extract about 20 to 30 variables (KPI values) from it".

And I discovered that writing equivalent code for the CAT command in python will use more lines of code?

Or if anyone else has idea, I will appreciate it.

@disedorgue --- I will also check your opinion, thanks for taking time out to suggest an idea.
# 5  
Old 10-10-2014
Many tools including python would happily go through the file, once, and write a sort or one or many output files with the desired information.Properly formatted, sorted output can then support the desired reporting from sequential access. It's very old school.
# 6  
Old 10-10-2014
Quote:
Originally Posted by infinitydon
Be it as it may, my challenge remains "How do I optimize this code seeing that the input file alone from EXPECT statement is over 800 lines and I need to extract about 20 to 30 variables (KPI values) from it".
Optimize? Not really feasible. You've written inside-out code crossing three languages. Optimizing it would leave nothing of your original code and possibly strip out the python, the shell, or the awk.

Show the input you have, and the output you want, and we will show you a better way. As DGPickett says, read the file once, do all your processing once. awk or perl would be fairly powerful languages for this.

Quote:
And I discovered that writing equivalent code for the CAT command in python will use more lines of code?
Your cat does nothing in python or shell -- see useless use of cat.
# 7  
Old 10-14-2014
Python Cat
Hi all,

Find attached for the input file. (test_expect.txt)

I want to extract the following fields (some names are identical which needs to be mapped to unique variables):

All the "pdp-active" statistics

All the "ip-address-pool-statistics"

All the "ccas-statistics"

All the "uplink and downlink statistics"

All the "apn-radius-acct-server-statistics"
test_expect.txt (53.9 KB) 
This User Gave Thanks to infinitydon For This Post:
Corona688
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Programming

Create a C source and compile inside Python 1.4.0 to 3.7.0 in Python for ALL? platforms...

Hi all... As you know I like making code backwards compatible for as many platforms as possible. This Python script was in fact dedicated for the AMIGA A1200 using Pythons 1.4.0, 1.5.2, 1.6.0, 2.0.1, and 2.4.6 as that is all we have for varying levels of upgrades from a HDD and 4MB FastRam... (1 Reply)
Discussion started by: wisecracker
1 Replies

2. Windows & DOS: Issues & Discussions

How to execute python script on remote with python way..?

Hi all, I am trying to run below python code for connecting remote windows machine from unix to run an python file exist on that remote windows machine.. Below is the code I am trying: #!/usr/bin/env python import wmi c = wmi.WMI("xxxxx", user="xxxx", password="xxxxxxx")... (1 Reply)
Discussion started by: onenessboy
1 Replies

3. Shell Programming and Scripting

**python** unable to read the background color in python

I am working on requirement on spreadsheet in python scripting. I have a spreadsheet containing cell values and with background color. I am able to read the value value but unable to get the background color of that particular cell. Actually my requirement is to read the cell value along... (1 Reply)
Discussion started by: giridhar276
1 Replies

4. SuSE

"ssh suse-server 'python -V' > python-version.out" not redirecting

Okay, so I have had this problem on openSUSE, and Debian systems now and I am hoping for a little help. I think it has something to do with Python but I couldn't find a proper Python area here. I am trying to redirect the output of "ssh suse-server 'python -V'" to a file. It seems that no matter... (3 Replies)
Discussion started by: Druonysus
3 Replies

5. Shell Programming and Scripting

CAT equivalent in python

cat is such a simple and useful command in UNIX. I was wonder if python had any equivalent. More specifically, I have a .txt file that I would like displayed from a python script. Is there a similar command besides "print?" When I try to use the print command, the text is formatted... (2 Replies)
Discussion started by: jl487
2 Replies

6. Shell Programming and Scripting

cat in the command line doesn't match cat in the script

Hello, So I sorted my file as I was supposed to: sort -n -r -k 2 -k 1 file1 | uniq > file2 and when I wrote > cat file2 in the command line, I got what I was expecting, but in the script itself ... sort -n -r -k 2 -k 1 averages | uniq > temp cat file2 It wrote a whole... (21 Replies)
Discussion started by: shira
21 Replies

7. Shell Programming and Scripting

for i in `cat myname.txt` && for y in `cat yourname.txt`

cat myname.txt John Doe I John Doe II John Doe III ----------------------------------------------------------------------- for i in `cat myname.txt` do echo This is my name: $i >> thi.is.my.name.txt done ----------------------------------------------------------------------- cat... (1 Reply)
Discussion started by: danimad
1 Replies

8. UNIX for Dummies Questions & Answers

Difference between cat , cat > , cat >> and touch !!!

Hi Can anybody tell the difference between Difference between cat , cat > , cat >> and touch command in UNIX? Thanks (6 Replies)
Discussion started by: skyineyes
6 Replies
Login or Register to Ask a Question