Sponsored Content
Top Forums Shell Programming and Scripting How to tar large amount of files? Post 302472821 by pludi on Thursday 18th of November 2010 07:55:08 AM
Old 11-18-2010
What cabrao said, but you'll have to use single quotes ('), not double quotes ("), otherwise parameter expansion will still happen, and you'll get the same error again.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Using TAR on a large file

I am runnign Solaris 8. I am trying to TAR a large file (13 gig) and it is complaining about the size. Any suggestions? (4 Replies)
Discussion started by: hshapiro
4 Replies

2. Linux

shmat() Failure While Using a Large Amount of Shared Memory

Hi, I'm developing a data processing pipeline with multiple stages, with data being moved between the stages using shared memory segments. The size of the data is typically of the order of hundreds of megabytes, and there are typically a few tens of main shared memory segments each of size... (2 Replies)
Discussion started by: theicarusagenda
2 Replies

3. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies

4. AIX

amount of memory allocated to large page

We just set up a system to use large pages. I want to know if there is a command to see how much of the memory is being used for large pages. For example if we have a system with 8GB of RAm assigned and it has been set to use 4GB for large pages is there a command to show that 4GB of the *GB is... (1 Reply)
Discussion started by: daveisme
1 Replies

5. Solaris

Tar too large to archive. Use E function modifier.

hey all, i am trying to tar up a folder with sub folders the over all size will be about 70gb but when i use the normal command tar -cvf tar -cvf CLPSI_PRU_Escrow_31994.tar CLPSI_PRU_Escrow_31994 i get an error tar: CLPSI_PRU_Escrow_31994/dump1/PROD_SAE_jria3_dump.5 too large to archive. ... (9 Replies)
Discussion started by: dshakey
9 Replies

6. UNIX for Dummies Questions & Answers

questing regarding tar large number of files

I want to tar large number of files about 150k. i am using the find command as below to create a file with all file names. & then trying to use the tar -I command as below. # find . -type f -name "gpi*" > include-file # tar -I include-file -cvf newfile.tar This i got from one of the posts... (2 Replies)
Discussion started by: crux123
2 Replies

7. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

8. UNIX for Dummies Questions & Answers

Compress 1st 20 Large different File using tar

Sorry guys.. dont know where to put this.. currently I am cleaning up system dump on our aix machine and I wanted to set zero the 1st 20 large file but before doing that I wanted to create some backup. is there any command that can compress all these file same time? tar syntax file? ... (2 Replies)
Discussion started by: thermometer
2 Replies

9. Shell Programming and Scripting

Perl : Large amount of data put into an array

This basic code works. I have a very long list, almost 10000 lines that I am building into the array. Each line has either 2 or 3 fields as shown in the code snippit. The array elements are static (for a few reasons that out of scope of this question) the list has to be "built in". It... (5 Replies)
Discussion started by: sumguy
5 Replies

10. Shell Programming and Scripting

How to make awk command faster for large amount of data?

I have nginx web server logs with all requests that were made and I'm filtering them by date and time. Each line has the following structure: 127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br) These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies
SHQUOTE(3)						   BSD Library Functions Manual 						SHQUOTE(3)

NAME
shquote, shquotev -- quote argument strings for use with the shell LIBRARY
Standard C Library (libc, -lc) SYNOPSIS
#include <stdlib.h> size_t shquote(const char *arg, char *buf, size_t bufsize); size_t shquotev(int argc, char * const *argv, char *buf, size_t bufsize); DESCRIPTION
The shquote() and shquotev() functions copy strings and transform the copies by adding shell escape and quoting characters. They are used to encapsulate arguments to be included in command strings passed to the system() and popen() functions, so that the arguments will have the correct values after being evaluated by the shell. The exact method of quoting and escaping may vary, and is intended to match the conventions of the shell used by system() and popen(). It may not match the conventions used by other shells. In this implementation, the following transformation is applied to each input string: o it is surrounded by single quotes ('), o any single quotes in the input are escaped by replacing them with the four-character sequence: ''', and o extraneous pairs of single quotes (caused by multiple adjacent single quotes in the input string, or by single quotes at the begin- ning or end of the input string) are elided. The shquote() function transforms the string specified by its arg argument, and places the result into the memory pointed to by buf. The shquotev() function transforms each of the argc strings specified by the array argv independently. The transformed strings are placed in the memory pointed to by buf, separated by spaces. It does not modify the pointer array specified by argv or the strings pointed to by the pointers in the array. Both functions write up to bufsize - 1 characters of output into the buffer pointed to by buf, then add a NUL character to terminate the out- put string. If bufsize is given as zero, the buf parameter is ignored and no output is written. RETURN VALUES
The shquote() and shquotev() functions return the number of characters necessary to hold the result from operating on their input strings, not including the terminating NUL. That is, they return the length of the string that would have been written to the output buffer, if it were large enough. If an error occurs during processing, the value ((size_t)-1) is returned and errno is set appropriately. EXAMPLES
The following code fragment demonstrates how you might use shquotev() to construct a command string to be used with system(). The command uses an environment variable (which will be expanded by the shell) to determine the actual program to run. Note that the environment vari- able may be expanded by the shell into multiple words. The first word of the expansion will be used by the shell as the name of the program to run, and the rest will be passed as arguments to the program. char **argv, c, *cmd; size_t cmdlen, len, qlen; int argc; ... /* * Size buffer to hold the command string, and allocate it. * Buffer of length one given to snprintf() for portability. */ cmdlen = snprintf(&c, 1, "${PROG-%s} ", PROG_DEFAULT); qlen = shquotev(argc, argv, NULL, 0); if (qlen == (size_t)-1) { ... } cmdlen += qlen + 1; cmd = malloc(cmdlen); if (cmd == NULL) { ... } /* Create the command string. */ len = snprintf(cmd, cmdlen, "${PROG-%s} ", PROG_DEFAULT); qlen = shquotev(argc, argv, cmd + len, cmdlen - len); if (qlen == (size_t)-1) { /* Should not ever happen. */ ... } len += qlen; /* "cmd" can now be passed to system(). */ The following example shows how you would implement the same functionality using the shquote() function directly. char **argv, c, *cmd; size_t cmdlen, len, qlen; int argc, i; ... /* * Size buffer to hold the command string, and allocate it. * Buffer of length one given to snprintf() for portability. */ cmdlen = snprintf(&c, 1, "${PROG-%s} ", PROG_DEFAULT); for (i = 0; i < argc; i++) { qlen = shquote(argv[i], NULL, 0); if (qlen == (size_t)-1) { ... } cmdlen += qlen + 1; } cmd = malloc(cmdlen); if (cmd == NULL) { ... } /* Start the command string with the env var reference. */ len = snprintf(cmd, cmdlen, "${PROG-%s} ", PROG_DEFAULT); /* Quote all of the arguments when copying them. */ for (i = 0; i < argc; i++) { qlen = shquote(argv[i], cmd + len, cmdlen - len); if (qlen == (size_t)-1) { /* Should not ever happen. */ ... } len += qlen; cmd[len++] = ' '; } cmd[--len] = ''; /* "cmd" can now be passed to system(). */ SEE ALSO
sh(1), popen(3), system(3) BUGS
This implementation does not currently handle strings containing multibyte characters properly. To address this issue, /bin/sh (the shell used by system() and popen()) must first be fixed to handle multibyte characters. When that has been done, these functions can have multi- byte character support enabled. BSD
September 7, 2008 BSD
All times are GMT -4. The time now is 02:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy