Merge the data from two servers into a single file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge the data from two servers into a single file
# 1  
Old 01-29-2013
Merge the data from two servers into a single file

Hi All,

Need your inputs for the below.

I have 2 different servers 611 & 610, where i would be running two scripts. And would would be running one script from 611 at every 4 hours to merge the data from the 2 servers into 2 files and send a mail.

so below is the code snippet for 611:
Code:
#!/bin/bash
var1=`netstat | grep 16001 | wc -l`
var2=`date`
var3=`psme | grep -i DM_EAI | wc -l`
 
while true; do
    echo $var2 $var1
    sleep 1800
done  > eai_js_611.csv
 
while true; do
    echo $var2 $var3
    sleep 1800
done  > dm_eai_611.csv

code snippet for 610:
Code:
#!/bin/bash
var1=`netstat | grep 16001 | wc -l`
var2=`date`
var3=`psme | grep -i DM_EAI | wc -l`
 
while true; do
    echo $var2 $var1
    sleep 1800
done  > eai_js_610.csv
 
while true; do
    echo $var2 $var3
    sleep 1800
done  > dm_eai_610.csv

Now my question is how to merge the data from 611 & 610 in the below format:
Code:
eai_js_counts.csv
 
Timestamp                            BRM Servers 
                                          611       610
1/28/2013 2:00 AM PST          176       99
1/28/2013 6:00 AM PST          150       115

Code:
dm_eai_counts.csv
 
 
Timestamp                             BRM Servers 
                                           611       610
1/28/2013 2:00 AM PST           4           5
1/28/2013 6:00 AM PST           4           5

Please share your inputs.

Thanks!!

---------- Post updated at 11:32 PM ---------- Previous update was at 09:44 PM ----------

Can combine the data from 2 servers by using below:

final script on 611:
Code:
#!/bin/bash
paste -d '\n' eai_js_611.csv eai_js_610.csv > eai_js_counts.csv
paste -d '\n' dm_eai_611.csv dm_eai_610.csv > dm_eai_counts.csv

Also, from one server can read other server file using ftp, please correct me if I am wrong.

Now the only thing is left how to add the column names to the final output.

Thanks
# 2  
Old 01-29-2013
ftp works, but scp and ssh with access keys are easier to script; rcp and rsh are earlier, less secure versions of scp/ssh -- not encrypted, no keyed access, not well thought of. Easiest of all is to mount one system's dir on the other with NFS.
# 3  
Old 01-30-2013
Hi DGPickett,

Also, could you please me tell how can i use sudo user after logging to remote server.

Thanks

---------- Post updated at 07:04 AM ---------- Previous update was at 01:19 AM ----------

Hi All,

Could you please help me with the join command.

I am having 2 files as:

Code:
c1.csv
1/29/2013 01:00 AM PST  129     93
1/29/2013 01:30 AM PST  142     79
1/29/2013 02:00 AM PST  130     72

Code:
c2.csv
1/29/2012 09:00 PM PST  105     51
1/29/2012 11:00 PM PST  96      64
1/29/2012 11:30 PM PST  92      66

Using below join command:
Code:
join -a1 -a2 c1.csv c2.csv

And getting below output:
Code:
1/29/2012 09:00 PM PST 105 51
1/29/2012 11:00 PM PST 96 64
1/29/2012 11:30 PM PST 92 66
1/29/2013 01:00 AM PST 129 93
1/29/2013 01:30 AM PST 142 79
1/29/2013 02:00 AM PST 130 72

So wanted to check how can i modify the join command to add the column headings & proper spacing as below:
Code:
Timestamp                      610    611
1/29/2012 09:00 PM PST   105    51
1/29/2012 11:00 PM PST   96      64
1/29/2012 11:30 PM PST   92      66
1/29/2013 01:00 AM PST   129    93
1/29/2013 01:30 AM PST   142    79
1/29/2013 02:00 AM PST   130    72

Thanks.
# 4  
Old 01-30-2013
Three questions should be three chains! We're roaming!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge data in lines from same file

Need help figuring out how to merge data from a file. I have a large txt file with some data that needs to be merged from separate lines into one line. Doug.G|3/12/2011|817-555-5555|Portland Doug.G|3/12/2011|817-555-5522|Portland Steve.F|1/11/2007|817-555-5111|Portland... (5 Replies)
Discussion started by: cdubu2
5 Replies

2. Shell Programming and Scripting

How to merge fields in a single file using awk ?

Hi, From a file, using: awk -F" " '{ if (NF == 6) print $1, $2, $3, $4, $5, $6; if (NF == 5) print $1, $2, $3, $4, $5; }' i printed out the required output. But i'm trying to merge the columns. Please look at the desired output. Any suggestions? Thanks Output: 00015 PSA1 ... (5 Replies)
Discussion started by: web2moha
5 Replies

3. Shell Programming and Scripting

How to merge variable data from another file into specific place?

Hello, I'm trying to create multiple commands using a variable input from another file but am not getting any successful results. Basically, file1.txt contains multiple lines with single words: <file1.txt> yellow blue black white I want to create multiple echo commands with these... (8 Replies)
Discussion started by: demmel
8 Replies

4. Shell Programming and Scripting

How to merge the multiple data files as a single file?

Hi Experts, I have created multiple scripts and send the output to new file, getting this output to my mailbox on daily basis. I would like to send the all outputs to a single file, need to merge all file outputs on a single file. For example, Created script for df -h > df.doc grep... (7 Replies)
Discussion started by: seenuvasan1985
7 Replies

5. Shell Programming and Scripting

How to merge two .doc files from different servers.?

Folks, I would like to know how to merge two .doc files from different servers. For example, df -h > host1.doc --> from host1 df -h > host2.doc --> from host2 Now i want to merge these different server details into single .doc file. Pls let me know, if any questions... Regards,... (2 Replies)
Discussion started by: seenuvasan1985
2 Replies

6. Shell Programming and Scripting

Perl script to Merge contents of 2 different excel files in a single excel file

All, I have an excel sheet Excel1.xls that has some entries. I have one more excel sheet Excel2.xls that has entries only in those cells which are blank in Excel1.xls These may be in different workbooks. They are totally independent made by 2 different users. I have placed them in a... (1 Reply)
Discussion started by: Anamika08
1 Replies

7. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

8. UNIX for Dummies Questions & Answers

Merge rows in bid data file

Dear all, Please help me ,,,, if I have input file like this A_AA960715 leucine-rich repeat-containing protein GO:0006952 defense response P A_AA960715 leucine-rich repeat-containing protein GO:0008152 metabolic process P A_AA960715 leucine-rich... (5 Replies)
Discussion started by: AAWT
5 Replies

9. Shell Programming and Scripting

Merge two file data together based on specific pattern match

My input: File_1: 2000_t g1110.b1 abb.1 2001_t g1111.b1 abb.2 abb.2 g1112.b1 abb.3 2002_t . . File_2: 2000_t Ali england 135 abb.1 Zoe british 150 2001_t Ali england 305 g1111.b1 Lucy russia 126 (6 Replies)
Discussion started by: patrick87
6 Replies

10. UNIX for Dummies Questions & Answers

get the data in a file into single string

Hello everyone ! I need to get the data in a file into a string. file1 1 2 3 4 5 I need to write a script where the file1 info is stored in a string (say variable s). So the output should be 1,2,3,4,5 or (1,2,3,4,5) If i say echo $s or print $s output should be 1,2,3,4,5 or ... (7 Replies)
Discussion started by: i.scientist
7 Replies
Login or Register to Ask a Question