Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Argument list too long w/ sed
Top Forums UNIX for Advanced & Expert Users Argument list too long w/ sed Post 302850589 by jim mcnamara on Thursday 5th of September 2013 05:44:43 AM
Old 09-05-2013
78000 file names is a lot of text when taken together. ARG_MAX is the limit to the total size of the arguments in bytes given to a command, including all env variables (like PATH, etc.)

one way around the problem, if I get what you are doing with sed:
Code:
ls | while read fname 
do 
 md5sum $fname | gsed -e s/\([^ ]*\) \(.*\(\..*\)\)$/mv -v \2 \1\3/e'
done [... more code]

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Argument list too long - Shell error

Trying to tar specific files from a directory causes problems when the number of files is too large. ls ~/logs | wc -l 5928 In the logs directory - I have 5928 files If I want to include all files with today's date - I run the following command tar cf ~/archive/LoadLogs_20060302.tar... (8 Replies)
Discussion started by: dad5119
8 Replies

2. UNIX for Dummies Questions & Answers

Argument list too long - SSH

Hi I executed the code for file in `ls pdb*.ent` do new_name=`echo $file | sed 's/^pdb//;s/.ent/.txt/'` mv $file $new_name done Its giving error at ' ls pdb*.ent' argument list too long i have around 150000 entries please help Thank you (6 Replies)
Discussion started by: empyrean
6 Replies

3. Shell Programming and Scripting

Argument too long list error

I have a wrote a script which consits of the below line.. Below of this script I'm getting this error "ksh: /usr/bin/ls: arg list too long" The line is log_file_time=`ssh -i $HOME/.ssh/id_rsa -q $i ls -lrt /bp/karthik/test/data/log/$abc*|tail -1|awk '{print $8}'` And $abc alias is as "p |... (1 Reply)
Discussion started by: 22karthikreddy
1 Replies

4. UNIX for Dummies Questions & Answers

Argument list too long for Sed command

Hi guys Following command results in sed -i 's/#/\\#/g' /home/test/sqlstents* -bash: /bin/sed: Argument list too long Please help me solve it.. is there any other way i can do this?.. thanks (4 Replies)
Discussion started by: depakjan
4 Replies

5. Shell Programming and Scripting

Argument list too long problem

I have a huge set of files (with extension .common) in my directory around 2 million. When I run this script on my Linux with BASH, I get /bin/awk: Argument list too long awk -F'\t' ' NR == FNR { a=NR } NR != FNR { sub(".common", "", FILENAME) print a, FILENAME, $1 } '... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

6. Shell Programming and Scripting

Argument list too long!!

Dear Experts, I have a list of 10K files in a directory. I am not able to execute any commands lile ls -lrt, awk, sed, mv, etc........ I wanna execute below command and get the output. How can I achieve it?? Pls help. root# awk -F'|' '$1 == 1' file_20120710* | wc -l /bin/awk: Argument list... (2 Replies)
Discussion started by: Naga06
2 Replies

7. Shell Programming and Scripting

grep : Argument list too long

Hi, i am having some trouble with the below command, can some one suggest me the better way to do it. grep -l 'ReturnCode=1' `find $Log -newer /tmp/Failed.tmp -print | xargs ls -ld | egrep SUB | egrep -ve 'MTP' -ve 'ABC' -ve 'DEF' -ve 'JKL' -ve 'XYZ' | awk '{print $9}'` > $Home1 Its... (2 Replies)
Discussion started by: Prateek007
2 Replies

8. Shell Programming and Scripting

mv : Argument list too long

Hi I am using find command -- find "directory1" -type f | xargs -i mv {} "directory2" to avoid above argument list too long problem. But, issue i am facing is directory1 is having subdirectories due to this i am facing directory traversal problem as i dont want to traverse subdirectories... (9 Replies)
Discussion started by: VSom007
9 Replies

9. Shell Programming and Scripting

sed cannot execute [Argument list too long]

Hi All, This question has been asked many times, but my problem is slightly different. In my shell script i am connecting to oracle database and loading the results to .dat file. This .dat file is later used to create to .xls file Some times the size of .dat file becomes more than 120000... (8 Replies)
Discussion started by: galaxy_rocky
8 Replies

10. Shell Programming and Scripting

Argument list too long

Hi Team, Here's the situation. I have approximately 300000 to 500000 jpg files in /appl/abcd/work_dir mv /appl/abcd/work_dir /appl/abcd/process_dir The above move command will work if the jpg files count is close to 50000 (not sure). If the count is less this mv command holds good. But if... (14 Replies)
Discussion started by: kmanivan82
14 Replies

LEARN ABOUT OPENSOLARIS

env

env(1)								   User Commands							    env(1)

NAME

       env - set environment for command invocation

SYNOPSIS

       /usr/bin/env [-i | -] [name=value]... [utility [arg... ]]

       /usr/xpg4/bin/env [-i | -] [name=value]...
	    [utility [arg... ]]

DESCRIPTION

       The  env utility obtains the current environment, modifies it according to its arguments, then invokes the utility named by the utility op-
       erand with the modified environment.

       Optional arguments are passed to utility. If no utility operand is specified, the resulting environment is written to the standard  output,
       with one name=value pair per line.

   /usr/bin
       If env executes commands with arguments, it uses the default shell /usr/bin/sh (see sh(1)).

   /usr/xpg4/bin
       If env executes commands with arguments, it uses /usr/xpg4/bin/sh (see ksh(1)).

OPTIONS

       The following options are supported:

       -i | -	     Ignores  the  environment that would otherwise be inherited from the current shell.  Restricts the environment for utility to
		     that specified by the arguments.

OPERANDS

       The following operands are supported:

       name=value     Arguments of the form name=value modify the execution environment, and are placed  into  the  inherited  environment  before
		      utility is invoked.

       utility	      The  name of the utility to be invoked.  If utility names any of the special shell built-in utilities, the results are unde-
		      fined.

       arg	      A string to pass as an argument for the invoked utility.

EXAMPLES

       Example 1 Invoking utilities with new PATH values

       The following utility:

	 example% env -i PATH=/mybin mygrep xyz myfile

       invokes the utility mygrep with a new PATH value as the only entry in its environment. In this case, PATH is used to locate  mygrep,  which
       then must reside in /mybin.

ENVIRONMENT VARIABLES

       See  environ(5)	for  descriptions of the following environment variables that affect the execution of env: LANG, LC_ALL, LC_CTYPE, LC_MES-
       SAGES, and NLSPATH.

       PATH    Determine the location of the utility. If PATH is specified as a name=value operand to env, the value given shall be  used  in  the
	       search for utility.

EXIT STATUS

       If  utility  is invoked, the exit status of env is the exit status of utility. Otherwise, the env utility returns one of the following exit
       values:

       0	 Successful completion.

       1-125	 An error occurred.

       126	 utility was found but could not be invoked.

       127	 utility could not be found.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

   /usr/bin
       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+
       |CSI			     |enabled			   |
       +-----------------------------+-----------------------------+

   /usr/xpg4/bin
       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWxcu4			   |
       +-----------------------------+-----------------------------+
       |CSI			     |enabled			   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Standard			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       ksh(1), sh(1), exec(2), profile(4), attributes(5), environ(5), standards(5)

SunOS 5.11							    2 Jan 2002								    env(1)
All times are GMT -4. The time now is 04:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy