Sponsored Content
Top Forums Shell Programming and Scripting cp files containing \ from a script Post 302244730 by Keith Daniels on Wednesday 8th of October 2008 01:32:06 PM
Old 10-08-2008
Problem's solution

I never did truly understand "exactly" what it was about how I was using variables that was the problem but I found a work around, so to speak.

First if you want to use a variable to hold a "bad" file name, or one with unusual characters in it, to be used with the cp command -- then process the filename the minimum amount possible. I was processing the file name to create a new name, then I had to process it some more to undo the first processing so I could get the original name back. Bad idea, something happened to the filename while doing all that processing and I could never get it back to exactly what it was before... So I set out to minimize the processing of the filename to be used in the variable used to copy the old filename to the new filename.

Second, bash does its own processing of special characters and spaces when the cp command is run. Do not try to help it by adding escape characters (backslashes) to the variable. You have to deliver the name to the variable "exactly" as it appears when you use ls to display it. Then if the variable is quoted with double quotes, then bash can properly process it -- no matter what characters are in the file name.

Even though you can type the filename in by hand and add backslashes to the name and when you run the cp command from the script, it will work -- that ability will not transfer to a variable. Putting that same text with backslashes into a variable does not work because the variable does some processing "before or as" it expands the variable when the cp command is run. That "extra" processing produces something that is different than what was put in the variable, usually it adds backslashes and when it does, it does not match the "real" filename and can not copy it to the new filename.

Here is how I was originally processing the filenames. I was creating a list of filenames and storing them as a variable. Because the variable "fed" a for loop, I had to remove any spaces before I used the variable (for loops will not work with spaces in the filenames). I created the "filename list" variable and converted the spaces to % at the same time. I then separated the filenames into the basename and extension and stored them in variables. When I finished I reassembled this to get the original filename's basename and extension and removed the %s that I had inserted previously

This was too much processing. I ignored the KISS rule. The final cure was use the entire original filename with the % added -- as provided by the for loop -- instead of breaking it into basename and extension and having to reassemble all that back into the original file name. This allowed me to use the original filename with no processing other than removing the % symbol which I did at the same time I reassigned it to the final variable used by the cp command. Then all I had to do was quote the variable and everything worked.

My filename repair script works perfectly now and can repair filenames with leading and trailing spaces, backslashes, single quotes and any other characters that I have tried so far except the % symbol. There are two problems with % in the filename. When they are assigned to the variable storing the filenames bash turned % into %%, then the script turned all spaces into % and things went downhill from there. I decided to leave things alone and not try to process files with % in them, (files with % in them are very rare anyway).

I hope this makes sense, and can help someone in the future.

Thanks for everyone's help.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to retrieve all the linked script files/ctl files/sql files?

Hi I am going to migrate our datawarehouse system from HP Tru 64 Unix to the Red Hat Linux. Inside the box, it is running around 40 cron jobs; inside each cron job, it is calling other shell script files, and the shell script files may again call other shell script files or ctl files(for... (1 Reply)
Discussion started by: franksubramania
1 Replies

2. Shell Programming and Scripting

Help - Bug: A script to compile two types of data files into two temporary files

Dear other forum members, I'm writing a script for my homework, but I'm scratching all over my head and still can't figure out what I did wrong. Please help me. I just started to learn about bash scripting, and I appreciate if anyone of you can point out my errors. I thank you in advance. ... (3 Replies)
Discussion started by: ilove2smoke
3 Replies

3. Shell Programming and Scripting

Need help creating a script to FTP files to a server and then delete the files that were transfered.

I am trying to FTP files to a Windows server through my Linux machine. I have setup the file transfer with no problems but am having problem deleting those files from the Linux box. My current non-working solution is below. Any ideas, anyone?? :wall: Please be gentle, I'm fairly new to this... (4 Replies)
Discussion started by: jmalfhs
4 Replies

4. Shell Programming and Scripting

need a shell script to extract the files from source file and check whether those files existonserve

Hi, I am new to shell scripting.Please help me on this.I am using solaris 10 OS and shell i am using is # echo $0 -sh My requirement is i have source file say makefile.I need to extract files with extensions (.c |.cxx |.h |.hxx |.sc) from the makefile.after doing so i need to check whether... (13 Replies)
Discussion started by: muraliinfy04
13 Replies

5. Shell Programming and Scripting

Need to exclude .NFSxxx files in clear old files batch script

I am new to Shell Scripting and need some help. The following batch job has been failing for me due to the .nfsxxx files in use. I need to know how to modify the following script to exclude the .nfsxxx files so this batch job will not fail on me. I have done lots of googling and keep coming back... (2 Replies)
Discussion started by: kimberlyg2007
2 Replies

6. Shell Programming and Scripting

Read files in shell script code and run a C program on those files

HI, I am trying to implement a simple shell script program that does not make use of ls or find commands as they are quite expensive on very large sets of files. So, I am trying to generate the file list myself. What I am trying to do is this: 1. Generate a file name using shell script, for... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

7. Shell Programming and Scripting

Bash script deleting my files, and editing files in subdirectories question

#!/bin/bash # name=$1 type=$2 number=1 for file in ./** do if then filenumber=00$number elif then filenumber=0$number fi tempname="$name""$filenumber"."$type" if (4 Replies)
Discussion started by: TheGreatGizmo
4 Replies

8. Shell Programming and Scripting

How to create or convert to pdf files from csv files using shell script?

Hi, Can anyone help me how to convert a .csv file to a .pdf file using shell script Thanks (2 Replies)
Discussion started by: ssk250
2 Replies

9. Shell Programming and Scripting

Renaming multiple files in sftp server in a get files script

Hi, In sftp script to get files, I have to rename all the files which I am picking. Rename command does not work here. Is there any way to do this? I am using #!/bin/ksh For eg: sftp user@host <<EOF cd /path get *.txt rename *.txt *.txt.done ... (7 Replies)
Discussion started by: jhilmil
7 Replies

10. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies
ENV(1)							    BSD General Commands Manual 						    ENV(1)

NAME
env -- set environment and execute command, or print environment SYNOPSIS
env [-iv] [-P altpath] [-S string] [-u name] [name=value ...] [utility [argument ...]] DESCRIPTION
The env utility executes another utility after modifying the environment as specified on the command line. Each name=value option specifies the setting of an environment variable, name, with a value of value. All such environment variables are set before the utility is executed. The options are as follows: -i Execute the utility with only those environment variables specified by name=value options. The environment inherited by env is ignored completely. -P altpath Search the set of directories as specified by altpath to locate the specified utility program, instead of using the value of the PATH environment variable. -S string Split apart the given string into multiple strings, and process each of the resulting strings as separate arguments to the env util- ity. The -S option recognizes some special character escape sequences and also supports environment-variable substitution, as described below. -u name If the environment variable name is in the environment, then remove it before processing the remaining options. This is similar to the unset command in sh(1). The value for name must not include the '=' character. -v Print verbose information for each step of processing done by the env utility. Additional information will be printed if -v is spec- ified multiple times. The above options are only recognized when they are specified before any name=value options. If no utility is specified, env prints out the names and values of the variables in the environment, with one name/value pair per line. Details of -S (split-string) processing The processing of the -S option will split the given string into separate arguments based on any space or <tab> characters found in the string. Each of those new arguments will then be treated as if it had been specified as a separate argument on the original env command. Spaces and tabs may be embedded in one of those new arguments by using single (``''') or double ('"') quotes, or backslashes (''). Single quotes will escape all non-single quote characters, up to the matching single quote. Double quotes will escape all non-double quote charac- ters, up to the matching double quote. It is an error if the end of the string is reached before the matching quote character. If -S would create a new argument that starts with the '#' character, then that argument and the remainder of the string will be ignored. The '#' sequence can be used when you want a new argument to start with a '#' character, without causing the remainder of the string to be skipped. While processing the string value, -S processing will treat certain character combinations as escape sequences which represent some action to take. The character escape sequences are in backslash notation. The characters and their meanings are as follows: c Ignore the remaining characters in the string. This must not appear inside a double-quoted string. f Replace with a <form-feed> character. Replace with a <new-line> character. Replace with a <carriage return> character. Replace with a <tab> character. v Replace with a <vertical tab> character. # Replace with a '#' character. This would be useful when you need a '#' as the first character in one of the arguments created by splitting apart the given string. $ Replace with a '$' character. \_ If this is found inside of a double-quoted string, then replace it with a single blank. If this is found outside of a quoted string, then treat this as the separator character between new arguments in the original string. " Replace with a <double quote> character. ' Replace with a <single quote> character. \ Replace with a backslash character. The sequences for <single-quote> and backslash are the only sequences which are recognized inside of a single-quoted string. The other sequences have no special meaning inside a single-quoted string. All escape sequences are recognized inside of a double-quoted string. It is an error if a single '' character is followed by a character other than the ones listed above. The processing of -S also supports substitution of values from environment variables. To do this, the name of the environment variable must be inside of '${}', such as: ${SOMEVAR}. The common shell syntax of $SOMEVAR is not supported. All values substituted will be the values of the environment variables as they were when the env utility was originally invoked. Those values will not be checked for any of the escape sequences as described above. And any settings of name=value will not effect the values used for substitution in -S processing. Also, -S processing can not reference the value of the special parameters which are defined by most shells. For instance, -S can not recog- nize special parameters such as: '$*', '$@', '$#', '$?' or '$$' if they appear inside the given string. Use in shell-scripts The env utility is often used as the interpreter on the first line of interpreted scripts, as described in execve(2). Note that the way the kernel parses the '#!' (first line) of an interpreted script has changed as of FreeBSD 6.0. Prior to that, the FreeBSD kernel would split that first line into separate arguments based on any whitespace (space or <tab> characters) found in the line. So, if a script named /usr/local/bin/someport had a first line of: #!/usr/local/bin/php -n -q -dsafe_mode=0 then the /usr/local/bin/php program would have been started with the arguments of: arg[0] = '/usr/local/bin/php' arg[1] = '-n' arg[2] = '-q' arg[3] = '-dsafe_mode=0' arg[4] = '/usr/local/bin/someport' plus any arguments the user specified when executing someport. However, this processing of multiple options on the '#!' line is not the way any other operating system parses the first line of an interpreted script. So after a change which was made for FreeBSD 6.0 release, that script will result in /usr/local/bin/php being started with the arguments of: arg[0] = '/usr/local/bin/php' arg[1] = '-n -q -dsafe_mode=0' arg[2] = '/usr/local/bin/someport' plus any arguments the user specified. This caused a significant change in the behavior of a few scripts. In the case of above script, to have it behave the same way under FreeBSD 6.0 as it did under earlier releases, the first line should be changed to: #!/usr/bin/env -S /usr/local/bin/php -n -q -dsafe_mode=0 The env utility will be started with the entire line as a single argument: arg[1] = '-S /usr/local/bin/php -n -q -dsafe_mode=0' and then -S processing will split that line into separate arguments before executing /usr/local/bin/php. ENVIRONMENT
The env utility uses the PATH environment variable to locate the requested utility if the name contains no '/' characters, unless the -P option has been specified. EXIT STATUS
The env utility exits 0 on success, and >0 if an error occurs. An exit status of 126 indicates that utility was found, but could not be exe- cuted. An exit status of 127 indicates that utility could not be found. EXAMPLES
Since the env utility is often used as part of the first line of an interpreted script, the following examples show a number of ways that the env utility can be useful in scripts. The kernel processing of an interpreted script does not allow a script to directly reference some other script as its own interpreter. As a way around this, the main difference between #!/usr/local/bin/foo and #!/usr/bin/env /usr/local/bin/foo is that the latter works even if /usr/local/bin/foo is itself an interpreted script. Probably the most common use of env is to find the correct interpreter for a script, when the interpreter may be in different directories on different systems. The following example will find the 'perl' interpreter by searching through the directories specified by PATH. #!/usr/bin/env perl One limitation of that example is that it assumes the user's value for PATH is set to a value which will find the interpreter you want to execute. The -P option can be used to make sure a specific list of directories is used in the search for utility. Note that the -S option is also required for this example to work correctly. #!/usr/bin/env -S -P/usr/local/bin:/usr/bin perl The above finds 'perl' only if it is in /usr/local/bin or /usr/bin. That could be combined with the present value of PATH, to provide more flexibility. Note that spaces are not required between the -S and -P options: #!/usr/bin/env -S-P/usr/local/bin:/usr/bin:${PATH} perl COMPATIBILITY
The env utility accepts the - option as a synonym for -i. SEE ALSO
printenv(1), sh(1), execvp(3), environ(7) STANDARDS
The env utility conforms to IEEE Std 1003.1-2001 (``POSIX.1''). The -P, -S, -u and -v options are non-standard extensions supported by FreeBSD, but which may not be available on other operating systems. HISTORY
The env command appeared in 4.4BSD. The -P, -S and -v options were added in FreeBSD 6.0. BUGS
The env utility does not handle values of utility which have an equals sign ('=') in their name, for obvious reasons. The env utility does not take multibyte characters into account when processing the -S option, which may lead to incorrect results in some locales. BSD
April 17, 2008 BSD
All times are GMT -4. The time now is 07:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy