Python Paramiko multi threading to capture all output for command applied in loop


 
Thread Tools Search this Thread
Top Forums Programming Python Paramiko multi threading to capture all output for command applied in loop
# 1  
Old 08-04-2019
Python Paramiko multi threading to capture all output for command applied in loop

My issue :
I am getting only last command output data in ouput file. Though comamnd "print(output)" displays data for all 3rd column values but the data saved in file is not what required it hs to be the same which is being printed by command"print(output)".
Could you please help me to fix this,
i think multi threading is required in this, but i dnt know how to apply in this scenario

My code :

Code:
import os
import csv
import sys
import telnetlib
import time
import datetime
import re
import os
import subprocess
#import pandas as pd
import paramiko
import csv
import os
import logging
import socket
import threading

#outlock = threading.Lock()
#directory = '/tmp/'
directory1 = '/tmp/input/'

f = open(directory1+str('universal_file.txt'))
reader = csv.reader(f, delimiter=",")
second_col = []

for row in reader:

    str1_1 = ''.join(str(e) for e in row)
    print str1_1

    for filename in os.listdir(directory1):
        if filename==str1_1:
           # print str1
            f = open(directory1+str(filename))
            reader = csv.reader(f, delimiter=",")
            second_col = []
            x=[]
            for row in reader:
                str1 = ''.join(str(e) for e in row[2])
                print row[2]
                second_col.append(row[2])
                if "c" in str(second_col):
                       second_col.remove("c")
                for data in second_col:
                    ssh = paramiko.SSHClient()
                    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
                    ssh.connect(str1_1, username='abc', password='abc')
                    stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet %s\r\n' %data)
                    stdin.write('xy\n')
                    stdin.flush()

#                    with outlock:
                    output = stdout.readlines()
                    output = "". join(output)
                    print(output)
#                       saved_ip.write(output)

#            print second_col

#mylist = list(dict.fromkeys(second_col))
            file_name = str1_1.rsplit('.', 1)[0]
            with open(directory1+'output/'+str(file_name)+'.csv', 'a') as f:
#                print second_col
                f.write(output)
#                for item in second_col:
#                    f.write("\n%s" % item)
        continue

Output File(10.106.222.96):

Code:
Building configuration...

Current configuration : 107 bytes
!
interface GigabitEthernet2
 no ip address
 shutdown
 negotiation auto
 no mop enabled
 no mop sysid
end



Suppose we have 3 input files with data with filename(out_10.106.222.96.csv, out_10.106.222.97.csv, out_10.106.222.98.csv) at path /tmp/input
In these input files there can be any number of rows


out_10.106.222.96.csv
Code:
a,b,c
43,17104773,1
45,17104234,2


out_10.106.222.97.csv
Code:
a,b,c
43,17104773,3
45,17104234,4


out_10.106.222.99.csv
Code:
a,b,c
43,17104773,5
45,17104234,6

Now have to login to node one by one with IP(present in testin.csv file)
These IP matches with the above filenames(out_10.106.222.96.csv, out_10.106.222.97.csv, out_10.106.222.98.csv)
There can be case when file name is not present that matches with below IP, so we need to ignore or continue with other files.

Ip File(testin.csv)
Code:
10.106.222.96
10.106.222.97
10.106.222.99

Now have to login to node one by one.
suppose we login to node 10.106.222.96, then have to search file at path(/tmp/input) that has name with this IP(10.106.222.96)
Want to pick column 3rd values from file and have to pass in below command present in above code on line number
Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet %s\r\n' %data)

Like

Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet 1' %data)

Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet 2' %data)

...so on.

generate one more output file at different path say(/tmp/output) with the data gathered from above command(show run interface gigabitEthernet %s\r\n' %data)
with the name(f_10.106.222.96) or any other name but should have IP in it.


Similarly,
have to login to another node say(10.106.222.97)
suppose we login to node 10.106.222.97, then have to search file at path(/tmp/input) that has name with this IP(10.106.222.97)
Want to pick column 3rd values from file and have to pass in below command present in above code on line number
Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet %s\r\n' %data)

Same step have to repeat until we have logged into every node and collected the data for each file column 3 value by passing into command

Like

Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet 3' %data)

Code:
stdin, stdout, stderr = ssh.exec_command('show run interface gigabitEthernet 4' %data).

..so on.

generate one more output file at different path say(/tmp/output) with the data gathered from above command(show run interface gigabitEthernet %s\r\n' %data)
with the name(f_10.106.222.97) or any other name but should have IP in it.
This User Gave Thanks to as7951 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Multi threading in UNIX

Hi, Can we apply multi threading in Unix. I am using bash shell. We have a generic script to load the data to table based on file input. For each file there is an individual table to load. For each file found in directory I want to load the data in parallel to target table using ... (3 Replies)
Discussion started by: vedanta
3 Replies

2. Shell Programming and Scripting

sed command inside paramiko python

Hi I am trying to execute a sed command inside paramiko which finds and deletes the particular string from a file But sed command doesnt work inside paramiko python machine=qwe dssh = paramiko.SSHClient() dssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) try: ... (4 Replies)
Discussion started by: Priya Amaresh
4 Replies

3. UNIX for Dummies Questions & Answers

Confusion over Multi Threading

Hi, I am trying to get my head round Multi Threading and I have a few queries to try and clear up my confusion Q1. Is multi threading a hardware / chip level concept, an OS level or an application level concept ? I am trying to work out where SMT architecture fits in. Q2. What's the multi... (3 Replies)
Discussion started by: jimthompson
3 Replies

4. Shell Programming and Scripting

[Python] Search a file with paramiko

I need to compare the output files in a directory for sftp, looking through a mask. Return the full file name. Eg. I have a file named locally: test.txt I must check through sftp, if a file with the following name: test_F060514_H173148.TXT My idea is for the filename to a... (0 Replies)
Discussion started by: Jomeaide
0 Replies

5. Programming

Multi-threading

In this piece i implemented the gossip method. The first thread is invoked from inside the (msg is first sent from node -1 to 0 from main()) and the other threads are invoked from inside of the thread function itself. I used two mutexes and a condition variable to control the synchronization. ... (4 Replies)
Discussion started by: saman_glorious
4 Replies

6. Programming

Multi-threading

Hi, If we create 10 threads to invoke runQuery method at same time, Will queryProcessor will be overriden sometime or 10 different copies will be created? We are not using any sunchronzation mechnism in runQuery(). so there is not gurantee on QueryProcessor class variables right OR each 10... (1 Reply)
Discussion started by: jramesh1
1 Replies

7. Programming

Multi threading?

I am not sure if multi threading is the correct term, but here is what I am trying to do. I have a while loop that displays the number 1, pauses, displays the number 2, pauses , displays the number 3 ad infinitum. It just keeps counting. While the screen displays the sequence of numbers counting... (4 Replies)
Discussion started by: enuenu
4 Replies

8. Programming

Regarding Multi-Threading

Hi All, Here's my question I have a 385 MB file containing 5,000,000 records. I need to read from the file and load into a table. Initially i thought of doing it in a single thread (execution of a single program) but when calculated accounted 16 hours of time on a standard benchmark. Hence... (5 Replies)
Discussion started by: matrixmadhan
5 Replies

9. Programming

Multi threading using fork

Hi, I have written a code which will run a set of process using fork. I want to know from You how can i start another job when one of my job in my loop is completed My code is #include<stdio.h> #include<ctype.h> main() { int pid,cid; ChildProcess(); ... (1 Reply)
Discussion started by: sureshraju_ma
1 Replies
Login or Register to Ask a Question