Sponsored Content
Full Discussion: FTP a huge Size file
Top Forums Shell Programming and Scripting FTP a huge Size file Post 302548240 by Chubler_XL on Wednesday 17th of August 2011 11:30:31 PM
Old 08-18-2011
Quote:
Originally Posted by Corona688
If there was a special 'go faster' data transfer method, most things would be using it already.
Agreed, also if this is for backup purposes why not do what most over people do in this situation - write the data to tape/disk and store off-site but close by. This also has the advantage that recovery time is much quicker as you don't have another 8 hour delay to transfer the data over the network again.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

ftp file size

how to compare file size which has been received through ftp get from a remote location with local copy available any clue regards (5 Replies)
Discussion started by: sathiya
5 Replies

2. UNIX for Dummies Questions & Answers

ftp hangs on file size = 0

I have an ftp process which runs every 10 minutes between Unix and an NT box. Normally it works, but when the script tries to get a file from NT that has a length of 0, the ftp process hangs (as if it is still waiting for the end of the file). This is the script... (3 Replies)
Discussion started by: mheinrich
3 Replies

3. Shell Programming and Scripting

How to compare file size after ftp?

Is possible if I want to campare file size on source and destination after ftp transfer? If anybody know, please explain to me. (1 Reply)
Discussion started by: icemania
1 Replies

4. UNIX for Advanced & Expert Users

Need to synchronize filesystems with huge size

Hello, I am using IXOS 3rd party utility to maintain invoices (images) in my client's SAP system. We have a DR (Disaster Recovery) setup & for that I need to synchronize my /ixos filesystem (with size more than 50 GB) between two servers say x182 & x050. Initially I thought to tar the... (4 Replies)
Discussion started by: vishal_ranjan
4 Replies

5. Shell Programming and Scripting

Checking the size of a file after FTP

Hi I am doing a FTP process through which I am copying a file from my local server to Remote server. After this I want to check the size of the file Below is my program: LOCALDIR=/batch/ediprocess REMOTESERVER=test.appl.com REMOTEPATH=batch/ftpTest LOGIN=px PASSWORD=abcd ftp -n... (3 Replies)
Discussion started by: shanth_chandra
3 Replies

6. Shell Programming and Scripting

Implement in one line sed or awk having no delimiter and file size is huge

I have file which contains around 5000 lines. The lines are fixed legth but having no delimiter.Each line line contains nearly 3000 characters. I want to delete the lines a> if it starts with 1 and if 576th postion is a digit i,e 0-9 or b> if it starts with 0 or 9(i,e header and footer) ... (4 Replies)
Discussion started by: millan
4 Replies

7. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

8. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

9. AIX

FTP huge file transfer

Hi, I need to transfer 2000 files from one host to another.. I modified /etc/security/limits to -1 and ulimit -f, ulimit -s, ulimit -a.. Even then only 700 files are transferred. Could You please help me to sort out this issue.. I think some configuration related to memory is... (3 Replies)
Discussion started by: Priya Amaresh
3 Replies

10. UNIX for Advanced & Expert Users

FTP zero size file

Hi AM using unix Aix Ksh I need to clarify regarding sending the zero size file to another server. VAL=ftp.sh -c put souce_file Dest_file $1 $2 $3 $4 $5 $VAL 2 > $ERR When am Sending the Zero Size File getting alerts netout write returned Zero. But i would like to know $VAL... (4 Replies)
Discussion started by: Venkatesh1
4 Replies
bup-fsck(1)						      General Commands Manual						       bup-fsck(1)

NAME
bup-fsck - verify or repair a bup repository SYNOPSIS
bup fsck [-r] [-g] [-v] [--quick] [-j jobs] [--par2-ok] [--disable-par2] [filenames...] DESCRIPTION
bup fsck is a tool for validating bup repositories in the same way that git fsck validates git repositories. It can also generate and/or use "recovery blocks" using the par2(1) tool (if you have it installed). This allows you to recover from dam- aged blocks covering up to 5% of your .pack files. In a normal backup system, damaged blocks are less important, because there tends to be enough data duplicated between backup sets that a single damaged backup set is non-critical. In a deduplicating backup system like bup, however, no block is ever stored more than once, even if it is used in every single backup. If that block were to be unrecoverable, all your backup sets would be damaged at once. Thus, it's important to be able to verify the integrity of your backups and recover from disk errors if they occur. WARNING: bup fsck's recovery features are not available unless you have the free par2(1) package installed on your bup server. WARNING: bup fsck obviously cannot recover from a complete disk failure. If your backups are important, you need to carefully consider redundancy (such as using RAID for multi-disk redundancy, or making off-site backups for site redundancy). OPTIONS
-r, --repair attempt to repair any damaged packs using existing recovery blocks. (Requires par2(1).) -g, --generate generate recovery blocks for any packs that don't already have them. (Requires par2(1).) -v, --verbose increase verbosity (can be used more than once). --quick don't run a full git verify-pack on each pack file; instead just check the final checksum. This can cause a significant speedup with no obvious decrease in reliability. However, you may want to avoid this option if you're paranoid. Has no effect on packs that already have recovery information. -j, --jobs=numjobs maximum number of pack verifications to run at a time. The optimal value for this option depends how fast your CPU can verify packs vs. your disk throughput. If you run too many jobs at once, your disk will get saturated by seeking back and forth between files and performance will actually decrease, even if numjobs is less than the number of CPU cores on your system. You can experiment with this option to find the optimal value. --par2-ok immediately return 0 if par2(1) is installed and working, or 1 otherwise. Do not actually check anything. --disable-par2 pretend that par2(1) is not installed, and ignore all recovery blocks. EXAMPLE
# generate recovery blocks for all packs that don't # have them bup fsck -g # generate recovery blocks for a particular pack bup fsck -g ~/.bup/objects/pack/153a1420cb1c8*.pack # check all packs for correctness (can be very slow!) bup fsck # check all packs for correctness and recover any # damaged ones bup fsck -r # check a particular pack for correctness and recover # it if damaged bup fsck -r ~/.bup/objects/pack/153a1420cb1c8*.pack # check if recovery blocks are available on this system if bup fsck --par2-ok; then echo "par2 is ok" fi SEE ALSO
bup-damage(1), fsck(1), git-fsck(1) BUP
Part of the bup(1) suite. AUTHORS
Avery Pennarun <apenwarr@gmail.com>. Bup unknown- bup-fsck(1)
All times are GMT -4. The time now is 01:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy