Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: sed cannot execute [Argument list too long]
Top Forums Shell Programming and Scripting sed cannot execute [Argument list too long] Post 302897639 by ygemici on Tuesday 15th of April 2014 08:28:52 AM
Old 04-15-2014
Because of the kernel ARG_MAX value is limited and asterisk character expands the all matching files,directory,links or any other forms.
Therefore your bash(shell) command argument buffer's ( commandline length ) is fill and and then overflow.

You can use
* for loop
* while loop
* or any scriptable methods

* split the arguments to meaning values
Code:
ls -1 tgt_type_${tmp_file_*}.dat

* xargs (to split to the arguments with find is one good method )

* ls (use the folder hierarchy with asteriks )
Code:
ls -1 yourfolder/

* find (use the folder hierarchy without asteriks )
Code:
find /yourfolder -name '*'


For example :
Code:
while read myfile ; do sed '/^$/d' $myfile ; done | find /yourdolfer -name '*.dat'|xargs -n1

# (use the while loop and read the arguments one by one (-n1) )
Code:
find /yourdolfer -name '*.dat' -print0 | xargs -0 -n5 sed '/^$/d'

# (use the spaces instead of newlines with xargs and use getdents to get the file entries with find)


and consider the @CarloM 's notice..
your sed command's default output is your stdout (usually your screen tty,pts,console,...) ,
therefore you must redirect to another file for save the changes or use the '-i' parameter , if you have gnu sed for save to your file same time.


regards
ygemici
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Argument list too long - SSH

Hi I executed the code for file in `ls pdb*.ent` do new_name=`echo $file | sed 's/^pdb//;s/.ent/.txt/'` mv $file $new_name done Its giving error at ' ls pdb*.ent' argument list too long i have around 150000 entries please help Thank you (6 Replies)
Discussion started by: empyrean
6 Replies

2. Shell Programming and Scripting

Argument too long list error

I have a wrote a script which consits of the below line.. Below of this script I'm getting this error "ksh: /usr/bin/ls: arg list too long" The line is log_file_time=`ssh -i $HOME/.ssh/id_rsa -q $i ls -lrt /bp/karthik/test/data/log/$abc*|tail -1|awk '{print $8}'` And $abc alias is as "p |... (1 Reply)
Discussion started by: 22karthikreddy
1 Replies

3. UNIX for Dummies Questions & Answers

Argument list too long for Sed command

Hi guys Following command results in sed -i 's/#/\\#/g' /home/test/sqlstents* -bash: /bin/sed: Argument list too long Please help me solve it.. is there any other way i can do this?.. thanks (4 Replies)
Discussion started by: depakjan
4 Replies

4. Shell Programming and Scripting

Argument list too long problem

I have a huge set of files (with extension .common) in my directory around 2 million. When I run this script on my Linux with BASH, I get /bin/awk: Argument list too long awk -F'\t' ' NR == FNR { a=NR } NR != FNR { sub(".common", "", FILENAME) print a, FILENAME, $1 } '... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

5. Shell Programming and Scripting

Argument list too long for date command

Dear Friends, The following script processes a 14508 lines log file. #!/bin/sh while read line do d=`sed 's/* - * \*\/*\/* *\)\] .*/\1/' | tr '/' ' ' | sed 's/\(*\):\(*\)/\1 \2/'` y=`date -d "${d}" "+%Y%m%d%H%M%S"` echo "${y}" done While running the above script, I am... (4 Replies)
Discussion started by: tamil.pamaran
4 Replies

6. Shell Programming and Scripting

Argument list too long!!

Dear Experts, I have a list of 10K files in a directory. I am not able to execute any commands lile ls -lrt, awk, sed, mv, etc........ I wanna execute below command and get the output. How can I achieve it?? Pls help. root# awk -F'|' '$1 == 1' file_20120710* | wc -l /bin/awk: Argument list... (2 Replies)
Discussion started by: Naga06
2 Replies

7. Shell Programming and Scripting

grep : Argument list too long

Hi, i am having some trouble with the below command, can some one suggest me the better way to do it. grep -l 'ReturnCode=1' `find $Log -newer /tmp/Failed.tmp -print | xargs ls -ld | egrep SUB | egrep -ve 'MTP' -ve 'ABC' -ve 'DEF' -ve 'JKL' -ve 'XYZ' | awk '{print $9}'` > $Home1 Its... (2 Replies)
Discussion started by: Prateek007
2 Replies

8. Shell Programming and Scripting

mv : Argument list too long

Hi I am using find command -- find "directory1" -type f | xargs -i mv {} "directory2" to avoid above argument list too long problem. But, issue i am facing is directory1 is having subdirectories due to this i am facing directory traversal problem as i dont want to traverse subdirectories... (9 Replies)
Discussion started by: VSom007
9 Replies

9. UNIX for Advanced & Expert Users

Argument list too long w/ sed

Hi all, I am using GNU sed (named gsed under macports) in OSX. I have a directory with a series of files named pool_01.jpg through pool_78802.jpg. I am trying to use this command to rename the files to their checksum + extension. md5sum * | gsed -e 's/\(*\) \(.*\(\..*\)\)$/mv -v \2 \1\3/e' ... (3 Replies)
Discussion started by: openthomas
3 Replies

10. Shell Programming and Scripting

Argument list too long

Hi Team, Here's the situation. I have approximately 300000 to 500000 jpg files in /appl/abcd/work_dir mv /appl/abcd/work_dir /appl/abcd/process_dir The above move command will work if the jpg files count is close to 50000 (not sure). If the count is less this mv command holds good. But if... (14 Replies)
Discussion started by: kmanivan82
14 Replies

LEARN ABOUT DEBIAN

lat

lat(1)								Linux User's Manual							    lat(1)

NAME

       lat - LDAP Administration Tool

SYNOPSIS

       lat [options]

DESCRIPTION

       lat,  stands  for  LDAP	Administration	Tool.  The  tool allows you to browse LDAP-based directories and add/edit/delete entries contained
       within.	It can store profiles for quick access to different servers. There are also different views available such as  Users,  Groups  and
       Hosts which allow you to easily manage objects without having to deal with the intricacies of LDAP.

OPTIONS

       lat has few options, they are listed below.

       -d, --debug
	      Run the program with debugging code turned on.

       -h, --help
	      Print the program usage and exit.

       -v, --version
	      Print the program version and exit.

FILES

       $HOME/.lat/profiles.xml

       Contains the users saved connection profiles.  There is no password
	      information held in the files;  Passwords are stored in the users GNOME keyring database.

       $HOME/.lat/plugins.state

       Contains the user configuration for LAT plugins

       $HOME/.lat/profiles.xml

       Contains information on user connection profiles

       $HOME/.lat/templates.dat

       Contains information on the users custom templates.

AUTHOR

       Loren Bandiera <lorenb@mmgsecurity.com>

REPORTING BUGS

       Report bugs to http://dev.mmgsecurity.com/bugs/

COPYRIGHT

       Copyright (C) 2005 MMG Security, Inc.

       This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU-
       LAR PURPOSE.

lat								     May 2005								    lat(1)
All times are GMT -4. The time now is 12:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy