Write file over network


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Write file over network
# 1  
Old 09-03-2018
Write file over network

Hello Forum,

I have an embedded Busybox system with ash shell. On this system is a service with logging output. I want to have this logfile on another computer. This service creates the log file, but I can specify the filename via parameter. The memory is very fast filled up. What could be the easiest way to transfer this file on the fly to another linux machine? I cant install much additional software. I want to run the service for hours, without starting&stopping the service when the space is used.

I thought something like starting the service and writing the log into the file, but transferring the data with netcat to another machine. Where a second netcat instance writes it to hd. If I specify - as filename (for writing the log to stdout), the service only creates a file called --01.log in the current directory. I only need a hint that directs me into the correct direction.


Thanks a lot
Peter

Last edited by peterfarge; 09-03-2018 at 06:03 PM..
# 2  
Old 09-03-2018
Your problem is not unique and it has been solved long time ago. Configure busybox syslogd to send message logs to a remote logging service (a remote server).

If you want to rediscover the wheel on you own. Setup a log rotation policy in busybox and then send the rotated archive to another server using rsync or scp.
This User Gave Thanks to Aia For This Post:
# 3  
Old 09-04-2018
Its not this kind of log. Its a plain file created by a program. There is nothing in the syslog.
The system has only 2MB free flash drive and 60MB ram. mounting nfs is not supported.

Last edited by peterfarge; 09-04-2018 at 06:05 AM..
# 4  
Old 09-04-2018
Quote:
Originally Posted by peterfarge
Its not this kind of log. Its a plain file created by a program. There is nothing in the syslog.
If you have the logger program installed you could write a small script reading the log file, creating syslog messages out of it and then truncate the log file accordingly. If the program is creating the logfile by redirectio you could even use a pipe and forego the logfile completely:

Code:
/my/app > /some/log    # turn that into:
/my/app | /some/script_with_logger

logger is part of many Linux distributions and is used to create syslog messages from the commandline.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 5  
Old 09-04-2018
I have found a netcat/named pipe solution:

// On my linux laptop, write the logfile to disk:
Code:
netcat -l -p 16674 > log.txt

// On the busybox machine
// Create named pipe:
Code:
mknod /mnt/np.log p

// Read from the named pipe and write it to the latop:
Code:
cat < /mnt/np.log | netcat 192.168.1.2 16674

// Start the service
Code:
theservice -write-log /mnt/np.log

------ Post updated at 08:25 AM ------

Close Thread
Moderator's Comments:
Mod Comment edit by bakunin: Please use CODE-tags the same way i inserted them for you. Thank you.

Last edited by bakunin; 09-04-2018 at 07:33 PM..
# 6  
Old 09-04-2018
Be vary tho.
Named pipes have limited buffer size, which depend on operating system and its implementation.

If you are sure those limits won't be hit and remote host will always work....
Otherwise you risk losing logs.

This is reinventing the wheel so to say, a real crude way to ship logs.

Also, using single redirection ... > log.txt , will if a linux laptop process is restarted overwrite log.txt, causing again past log loss.

As bakunin mentioned, a logger facility will be much more robust, configurable and is in the standard.

Regards
Peasant.
This User Gave Thanks to Peasant For This Post:
# 7  
Old 09-04-2018
Its a unreliable solution. But currently I only want to see what could be logged.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ksh: How to write the log file simultaneously when .sql file executes by UNIX process

Hello Team- we would like to implement an approach which has to write the log file simultaneously when .sql file is executing by Unix process. At present,it is writing the log file once the process is completed. I've tested the current process with the below approaches and none of them... (1 Reply)
Discussion started by: Hima_B
1 Replies

2. Shell Programming and Scripting

Need help to write a shell script to convert text file to excel file.

Hi Everyone, I want your help to write a script which will take text file as input and on the basis of delimiter ":"script will create excel sheet. Example input: IpAdress:InstanceName:Port:ServerName 10.255.255.1:abc:2232:xyz_abc Output should be an excel sheet like below: Column... (8 Replies)
Discussion started by: akabhinav18
8 Replies

3. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

4. Shell Programming and Scripting

Need to search a particular String form a file a write to another file using perl script

I have file which contains a huge amount of data. I need to search the pattern Message id. When that pattern is matched I need to get abcdeff0-1g6g-91g3-1z2z-2mm605m90000 to another file. Kindly provide your input. File is like below Jan 11 04:05:10 linux100 |NOTICE... (2 Replies)
Discussion started by: Raysf
2 Replies

5. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

6. OS X (Apple)

Write message to another user on same network

On our home network, with 2 Macbook pros running OS X 10.6.*, I would like be able to popup a message to the user of the other computer. Is there a way I can "call" from one computer to another not using skype, or ichat or any application that the user has to have running? I guess first I... (8 Replies)
Discussion started by: allelopath
8 Replies

7. Shell Programming and Scripting

Extract data from an XML file & write into a CSV file

Hi All, I am having an XML tag like: <detail sim_ser_no_1="898407109001000090" imsi_1="452070001000090"> <security>ADM1=????</security> <security>PIN1=????</security> <security>PIN2=????</security> ... (2 Replies)
Discussion started by: ss_ss
2 Replies

8. Programming

Some how the open(file,flag, acc) returns 0 and write to the screen, instead of the file ???

I am out of idea what to do to resolve the problem! I need to use the open(file, for.., access) function to write a file. Never have the situation like that: it is return 0 - zero. As a result all write(..) going to the screen! What the problem it could be? I do not even know... (2 Replies)
Discussion started by: alex_5161
2 Replies

9. IP Networking

read/write,write/write lock with smbclient fails

Hi, We have smb client running on two of the linux boxes and smb server on another linux system. During a backup operation which uses smb, read of a file was allowed while write to the same file was going on.Also simultaneous writes to the same file were allowed.Following are the settings in the... (1 Reply)
Discussion started by: swatidas11
1 Replies

10. Cybersecurity

file permission/acl: 2 users with write access on 1 file...

Hello, i need some help/advice on how to solve a particular problem. these are the users: |name | group | ---------- --------------- |boss | department1 | |assistant | department1 | |employee | department1 | |spy | department2 | this is the... (0 Replies)
Discussion started by: elzalem
0 Replies
Login or Register to Ask a Question