Sponsored Content
Full Discussion: Compress files >2GB
Operating Systems Linux Compress files >2GB Post 302303683 by xavix on Friday 3rd of April 2009 08:59:53 AM
Old 04-03-2009
Compress files >2GB

Hi folks,

I'm trying to compress a certain number of files from a cifs mount to a xfs mount, but cannot do it when the total size of the files is bigger than 2GB.
Is there any limitation for above 2GB?

OS is SLES 64bit
The files are maximum 1MB, so there are aprox. 2000 files to compress
The cifs is in a Windows 2003 R2 Enterprise server
The xfs is a 1TB logical volume from a local raid with enough free space

I'm trying the following:

Code:
myfunction () {
find $ORIG -type f -mtime +30 -print > /tmp/tempfile.$$
tar -cvj `cat /tmp/tempfile.$$` | split -b 40m -d - $DEST/$TARFILE
rm `cat /tmp/tempfile.$$`
rm /tmp/tempfile.$$
}

I tried using tar, cpio, pax, gnutar and always I get the same output error:
tar: List of arguments is too large
pax: List of arguments is too large

I also tried with gzip, without compression, and without split...

When the number or size of the files to be compressed is below 2GB or below 2000 files, it works perfectly.

Any clues on that? Smilie
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

Use of unzip with content files > 2Gb

I am zipping and downloading zip files from an AS400 using the unzip utility. The files are being downloaded onto a Solaris box. Some of the content files in the zip are larger than 2GB. When using the unzip utility (version 5.32), it complains of 'disk full'. The disk is not full, I still have... (2 Replies)
Discussion started by: tcarlson
2 Replies

2. UNIX for Dummies Questions & Answers

How to Compress log files?

Hi, I have my log files in /home/user1/temp2/logs i want to archive *.log and *.txt files and to store in my /home/user1/temp2/archved/ with *.log with Time stamp ,Please let me know how to do this? (1 Reply)
Discussion started by: redlotus72
1 Replies

3. Shell Programming and Scripting

cpio - files > 2gb

Hi, Currently a backup script copies compressed files to tape using cpio command (on AIX 5.2). Recently we've had a compressed file which has gone over 2 GB in size resulting in an error while copying this file onto the tape using cpio. Any suggestions on relevant workarounds would be much... (2 Replies)
Discussion started by: dnicky
2 Replies

4. UNIX for Advanced & Expert Users

Problem creating files greater than 2GB

With the C code I am able to create files greater than 2GB if I use the 64 bit compile option -D_FILE_OFFSET_BITS=64. There I am using the function fprintf to write into the file. But when I use C++ and ofstream the file is getting truncated when the size grows beyond 2GB. Is there any special... (1 Reply)
Discussion started by: bobbyjohnz
1 Replies

5. UNIX for Dummies Questions & Answers

Compress files

Hi All, I would like to archive some of the scripts below(USFINUM042006_01.CSV USFINUM042006_02.CSV and USFINUM042006_03.CSV )and also use a wildcard e.g. <command> USFINUM*.CSV. Also there are a lot of similar files but I want only the three latest files to be compressed. Which is the best... (3 Replies)
Discussion started by: indira
3 Replies

6. UNIX for Dummies Questions & Answers

how to compress html files

Hello, On a Centos 5.0 server, Apache 2.2 delivers static html page. How could I compress those html pages to gain speed and save bandwidth? is there a utility that would be effective and save? Thanks (2 Replies)
Discussion started by: JCR
2 Replies

7. Filesystems, Disks and Memory

Compress files on NAS

Hello, I am having difficulty compressing the files using compress or GZIP utility on NAS share NFS mounted on my linux server. Any one have idea on how to do this ? I get the followign error but the trying to compress the files STRP2> compress STR_OUTBOUND_CDM_LOG_LOB.PRT2008_26.txt... (0 Replies)
Discussion started by: kamathg
0 Replies

8. Shell Programming and Scripting

compress files

Could someone give me an idea how to compress all files from a given directory that are not of type .z (compressed). Please help. (2 Replies)
Discussion started by: lesstjm
2 Replies

9. UNIX for Dummies Questions & Answers

Issue: Compress in unix server and FTP to windows and open the compress file using Winzip

Hi All ! We have to compress a big data file in unix server and transfer it to windows and uncompress it using winzip in windows. I have used the utility ZIP like the below. zip -e <newfilename> df2_test_extract.dat but when I compress files greater than 4 gb using zip utility, it... (4 Replies)
Discussion started by: sakthifire
4 Replies

10. AIX

Tar files larger than 2GB

Hi, Does anyone know if it is possible to tar files larger than 2GB? The reason being is they want me to dump a single file (which is around 20GB) to a tape drive and they will restore it on a Solaris box. I know the tar have a limitation of 2GB so I am thinking of a way how to overcome this.... (11 Replies)
Discussion started by: depam
11 Replies
CIFS.UPCALL(8)						    System Administration tools 					    CIFS.UPCALL(8)

NAME
cifs.upcall - Userspace upcall helper for Common Internet File System (CIFS) SYNOPSIS
cifs.upcall [--trust-dns|-t] [--version|-v] [--legacy-uid|-l] [--krb5conf=/path/to/krb5.conf|-k /path/to/...] {keyid} DESCRIPTION
This tool is part of the cifs-utils suite. cifs.upcall is a userspace helper program for the linux CIFS client filesystem. There are a number of activities that the kernel cannot easily do itself. This program is a callout program that does these things for the kernel and then returns the result. cifs.upcall is generally intended to be run when the kernel calls request-key(8) for a particular key type. While it can be run directly from the command-line, it's not generally intended to be run that way. OPTIONS
-c This option is deprecated and is currently ignored. --krb5conf=/path/to/krb5.conf|-k /path/to/krb5.conf This option allows administrators to set an alternate location for the krb5.conf file that cifs.upcall will use. --trust-dns|-t With krb5 upcalls, the name used as the host portion of the service principal defaults to the hostname portion of the UNC. This option allows the upcall program to reverse resolve the network address of the server in order to get the hostname. This is less secure than not trusting DNS. When using this option, it's possible that an attacker could get control of DNS and trick the client into mounting a different server altogether. It's preferable to instead add server principals to the KDC for every possible hostname, but this option exists for cases where that isn't possible. The default is to not trust reverse hostname lookups in this fashion. --legacy-uid|-l Traditionally, the kernel has sent only a single uid= parameter to the upcall for the SPNEGO upcall that's used to determine what user's credential cache to use. This parameter is affected by the uid= mount option, which also governs the ownership of files on the mount. Newer kernels send a creduid= option as well, which contains what uid it thinks actually owns the credentials that it's looking for. At mount time, this is generally set to the real uid of the user doing the mount. For multisession mounts, it's set to the fsuid of the mount user. Set this option if you want cifs.upcall to use the older uid= parameter instead of the creduid= parameter. --version|-v Print version number and exit. CONFIGURATION FOR KEYCTL
cifs.upcall is designed to be called from the kernel via the request-key callout program. This requires that request-key be told where and how to call this program. The current cifs.upcall program handles two different key types: cifs.spnego This keytype is for retrieving kerberos session keys dns_resolver This key type is for resolving hostnames into IP addresses. Support for this key type may eventually be deprecated (see below). To make this program useful for CIFS, you'll need to set up entries for them in request-key.conf(5). Here's an example of an entry for each key type: #OPERATION TYPE D C PROGRAM ARG1 ARG2... #========= ============= = = ================================ create cifs.spnego * * /usr/sbin/cifs.upcall %k create dns_resolver * * /usr/sbin/cifs.upcall %k See request-key.conf(5) for more info on each field. The keyutils package has also started including a dns_resolver handling program as well that is preferred over the one in cifs.upcall. If you are using a keyutils version equal to or greater than 1.5, you should use key.dns_resolver to handle the dns_resolver keytype instead of cifs.upcall. See key.dns_resolver(8) for more info. SEE ALSO
request-key.conf(5), mount.cifs(8), key.dns_resolver(8) AUTHOR
Igor Mammedov wrote the cifs.upcall program. Jeff Layton authored this manpage. The maintainer of the Linux CIFS VFS is Steve French. The Linux CIFS Mailing list is the preferred place to ask questions regarding these programs. cifs-utils 02/07/2010 CIFS.UPCALL(8)
All times are GMT -4. The time now is 09:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy