Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Automate splitting of files , scp files as each split completes and combine files on target server Post 303033449 by rbatte1 on Friday 5th of April 2019 12:59:55 PM
Old 04-05-2019
Hello malaika,

I would like to bring up a few things to consider:-
  • How do you know that the file is complete before you try to split it? Perhaps it gets transferred as filename.partial then renamed to filename when complete or there is a flag file such as filename.OKAY created in the next step
  • Why break up the file to send it and reassemble? Have you considered rsync? If there is an interruption it can restart to catch up. You might be able to set it up to transfer then remove the original. Trial this with small files though Smilie

Does this help/hinder/miss the point?




Kind regards,
Robin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to combine to flat files which are there in remote server ?

:mad: I have to write a script in informatica, which is used to combine two remote flat files (3 Replies)
Discussion started by: srinivasaraju.m
3 Replies

2. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies

3. Shell Programming and Scripting

Shell script to transfer the files from source to target server.

I need to write a shell script to transfer the files every hour from source - target server. The cron job should be running every hour and shouldn't copy already copied files to the remote server ? I was able to write intial script but not able to get the logic for (in the next run it should... (12 Replies)
Discussion started by: radhirk
12 Replies

4. Shell Programming and Scripting

copy files from remote server (B) to target server (A)?

Hi All, what is the comand to log off the remote server? I have 2 servers A, B. I need to find all files older than 7 days on server B and copy over to server A. My logic is: login the remote server: ================= ssh hostB cd /data/test find . -mtime -7 -ls | awk '{print... (4 Replies)
Discussion started by: Beginer0705
4 Replies

5. Shell Programming and Scripting

scp files from other server with check

Hi all, I am new to this forum,and it has been really a great pleasure to be a part of this forum:) well,i m trying to write a shell script where i m transfering a set of files from other server to my own server. for this i m using scp... (1 Reply)
Discussion started by: prasson_ibm
1 Replies

6. Shell Programming and Scripting

script for Copying files from one server to another using scp

Hi Scripting experts, I am new to the unix scripting. Please help me out for solving the condition given below I am trying to develop a script for Copying files which are getting generated in server A to server B using scp. In serverA files are generating as for eg abc1.txt, abc2.txt,... (5 Replies)
Discussion started by: rohithji
5 Replies

7. Shell Programming and Scripting

combine multiple files by column into one files already sorted!

I have multiple files; each file contains a certain data in a column view simply i want to combine all those files into one file in columns example file1: a b c d file 2: 1 2 3 4 file 3: G (4 Replies)
Discussion started by: ahmedamro
4 Replies

8. Shell Programming and Scripting

scp files that are 3 days older from remote server-

hello, i am trying to get a list of files to be scped from the remote server by running the below in my local unix server ( note - there is a passwordless connectivity setup made between the local and remote server) and, we use KSH. --- ssh $scp_host "find /a/b/c/*/ -iname "$remote_file"" >... (4 Replies)
Discussion started by: billpeter3010
4 Replies

9. Shell Programming and Scripting

Need shell script to compare directories and delete files on target server

Hello, I need help in writing the shell script for below mentioned case. There are 2 servers(server A, server B). A cronjob syncs files between these 2 servers. Existing script is copying files from A to B. This is done using the command rsync. However, the files are not deleted... (2 Replies)
Discussion started by: SravaniVedam11
2 Replies

10. UNIX for Beginners Questions & Answers

scp of multiple files to remote server

Hello, I would like to scp multiple files to a remote server that requires a password for the process to be completed. I have 30 folders (x_1, x_2 ... x_30), each containing 25 files. What I want to do is scp 1 out of the 25 files (file called bvals) for all my folders to a remote server and... (3 Replies)
Discussion started by: nasia.m
3 Replies
dpkg-split(1)							  dpkg utilities						     dpkg-split(1)

NAME
dpkg-split - Debian package archive split/join tool SYNOPSIS
dpkg-split [options] command DESCRIPTION
dpkg-split splits Debian binary package files into smaller parts and reassembles them again, to support the storage of large package files on small media such as floppy disks. It can be operated manually using the --split, --join and --info options. It also has an automatic mode, invoked using the --auto option, where it maintains a queue of parts seen but not yet reassembled and reassembles a package file when it has seen all of its parts. The --listq and --discard options allow the management of the queue. All splitting, joining and queueing operations produce informative messages on standard output; these may safely be ignored. COMMANDS
-s, --split complete-archive [prefix] Splits a single Debian binary package into several parts. The parts are named prefix.NofM.deb where N is the part number, starting at 1, and M is the total number of parts (both in decimal). If no prefix is supplied then the complete-archive filename is taken, including directory, with any trailing .deb removed. -j, --join part... Joins the parts of a package file together, reassembling the original file as it was before it was split. The part files given as arguments must be all the parts of exactly the same original binary file. Each part must occur exactly once in the argument list, though the parts to not need to be listed in order. The parts must of course all have been generated with the same part size specified at split time, which means that they must usually have been generated by the same invocation of dpkg-split --split. The parts' filenames are not significant for the reassembly process. By default the output file is called package-version.deb. -I, --info part... Prints information, in a human-readable format, about the part file(s) specified. Arguments which are not binary package parts pro- duce a message saying so instead (but still on standard output). -a, --auto -o complete-output part Automatically queue parts and reassemble a package if possible. The part specified is examined, and compared with other parts of the same package (if any) in the queue of packages file parts. If all parts of the package file of which part is a part are available then the package is reassembled and written to complete-out- put (which should not usually already exist, though this is not an error). If not then the part is copied into the queue and complete-output is not created. If part is not a split binary package part then dpkg-split will exit with status 1; if some other trouble occurs then it will exit with status 2. The --output or -o option must be supplied when using --auto. (If this were not mandatory the calling program would not know what output file to expect.) -l, --listq Lists the contents of the queue of packages to be reassembled. For each package file of which parts are in the queue the output gives the name of the package, the parts in the queue, and the total number of bytes stored in the queue. -d, --discard [package...] This discards parts from the queue of those waiting for the remaining parts of their packages. If no package is specified then the queue is cleared completely; if any are specified then only parts of the relevant package(s) are deleted. -h, --help Show the usage message and exit. --version Show the version and exit. OPTIONS
--depotdir directory Specifies an alternative directory for the queue of parts awaiting automatic reassembly. The default is /var/lib/dpkg. -S, --partsize kibibytes Specifies the maximum part size when splitting, in kibibytes (1024 bytes). The default is 450 KiB. -o, --output complete-output Specifies the output file name for a reassembly. This overrides the default for a manual reassembly (--join) and is mandatory for an automatic queue-or-reassemble (--auto). -Q, --npquiet When doing automatic queue-or-reassembly dpkg-split usually prints a message if it is given a part that is not a binary package part. This option suppresses this message, to allow programs such as dpkg to cope with both split and unsplit packages without pro- ducing spurious messages. --msdos Forces the output filenames generated by --split to be msdos-compatible. This mangles the prefix - either the default derived from the input filename or the one supplied as an argument: alphanumerics are lowercased, plus signs are replaced by x's and all other characters are discarded. The result is then truncated as much as is necessary, and filenames of the form prefixNofM.deb are generated. EXIT STATUS
An exit status of 0 indicates that the requested split, merge, or other command succeeded. --info commands count as successful even if the files are not binary package parts. An exit status of 1 occurs only with --auto and indicates that the part file was not a binary package part. An exit status of 2 indicates some kind of trouble, such as a system call failure, a file that looked like a package part file but was cor- rupted, a usage error or some other problem. BUGS
dpkg-split uses some rather out-of-date conventions for the the filenames of Debian packages. Full details of the packages in the queue are impossible to get without digging into the queue directory yourself. There is no easy way to test whether a file that may be a binary package part is one. The architecture is not represented in the part files' header, only in the control information of the contained binary package file, and it is not present in the filenames generated. FILES
/var/lib/dpkg/parts The default queue directory for part files awaiting automatic reassembly. The filenames used in this directory are in a format internal to dpkg-split and are unlikely to be useful to other programs, and in any case the filename format should not be relied upon. SEE ALSO
deb(5), deb-control(5), dpkg-deb(1), dpkg(1). AUTHOR
Copyright (C) 1995-1996 Ian Jackson This is free software; see the GNU General Public Licence version 2 or later for copying conditions. There is NO WARRANTY. Debian Project 2009-06-26 dpkg-split(1)
All times are GMT -4. The time now is 09:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy