04-05-2019
Automate splitting of files , scp files as each split completes and combine files on target server
i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B , once all the ten files have been received, i wll like to combine (cat them). How can i automate the whole process.
I want to save time and not wait for the completion of the splitting of all the ten 1000 GB files before I scp, I want to scp as each file completes.
Can anyone help with how to automate this process ? Thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
:mad: I have to write a script in informatica, which is used to combine two remote flat files (3 Replies)
Discussion started by: srinivasaraju.m
3 Replies
2. Shell Programming and Scripting
Hi,
I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below
for file in *.txt
do
grep -i -h "value1|value2" $file > $file;
done.
My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies
3. Shell Programming and Scripting
I need to write a shell script to transfer the files every hour from source - target server.
The cron job should be running every hour and shouldn't copy already copied files to the remote server ?
I was able to write intial script but not able to get the logic for (in the next run it should... (12 Replies)
Discussion started by: radhirk
12 Replies
4. Shell Programming and Scripting
Hi All,
what is the comand to log off the remote server?
I have 2 servers A, B. I need to find all files older than 7 days on server B and copy over to server A. My logic is:
login the remote server:
=================
ssh hostB
cd /data/test
find . -mtime -7 -ls | awk '{print... (4 Replies)
Discussion started by: Beginer0705
4 Replies
5. Shell Programming and Scripting
Hi all,
I am new to this forum,and it has been really a great pleasure to be a part of this forum:)
well,i m trying to write a shell script where i m transfering a set of files from other server to my own server.
for this i m using scp... (1 Reply)
Discussion started by: prasson_ibm
1 Replies
6. Shell Programming and Scripting
Hi Scripting experts,
I am new to the unix scripting. Please help me out for solving the condition given below
I am trying to develop a script for Copying files which are getting generated in server A to server B using scp. In serverA files are generating as for eg abc1.txt, abc2.txt,... (5 Replies)
Discussion started by: rohithji
5 Replies
7. Shell Programming and Scripting
I have multiple files; each file contains a certain data in a column view
simply i want to combine all those files into one file in columns
example
file1:
a
b
c
d
file 2:
1
2
3
4
file 3:
G (4 Replies)
Discussion started by: ahmedamro
4 Replies
8. Shell Programming and Scripting
hello,
i am trying to get a list of files to be scped from the remote server
by running the below in my local unix server ( note - there is a passwordless connectivity setup made between the local and remote server) and, we use KSH.
---
ssh $scp_host "find /a/b/c/*/ -iname "$remote_file"" >... (4 Replies)
Discussion started by: billpeter3010
4 Replies
9. Shell Programming and Scripting
Hello,
I need help in writing the shell script for below mentioned case.
There are 2 servers(server A, server B).
A cronjob syncs files between these 2 servers.
Existing script is copying files from A to B.
This is done using the command rsync.
However, the files are not deleted... (2 Replies)
Discussion started by: SravaniVedam11
2 Replies
10. UNIX for Beginners Questions & Answers
Hello,
I would like to scp multiple files to a remote server that requires a password for the process to be completed.
I have 30 folders (x_1, x_2 ... x_30), each containing 25 files. What I want to do is scp 1 out of the 25 files (file called bvals) for all my folders to a remote server and... (3 Replies)
Discussion started by: nasia.m
3 Replies
zipnote(1) General Commands Manual zipnote(1)
NAME
zipsplit - split a zipfile into smaller zipfiles
SYNOPSIS
zipsplit [-t] [-i] [-p] [-s] [-n size] [-r room] [-b path] [-h] [-v] [-L] zipfile
ARGUMENTS
zipfile Zipfile to split.
OPTIONS
-t Report how many files it will take, but don't make them.
-i Make index (zipsplit.idx) and count its size against first zip file.
-n size
Make zip files no larger than "size" (default = 36000).
-r room
Leave room for "room" bytes on the first disk (default = 0).
-b path
Use path for the output zip files.
-p Pause between output zip files.
-s Do a sequential split even if it takes more zip files.
-h Show a short help.
-v Show version information.
-L Show software license.
DESCRIPTION
zipsplit reads a zipfile and splits it into smaller zipfiles.
EXAMPLES
To be filled in.
BUGS
Does not yet support large (> 2 GB) or split archives.
SEE ALSO
zip(1), unzip(1)
AUTHOR
Info-ZIP
v3.0 of 8 May 2008 zipnote(1)