OK, I see where you were going with your initial post Corona. Thanks for teaching me that, I never knew about those string features in bash.
I've run into a new problem with my code that I can't figure out. The while loop below successfully iterates through the array and fills the temporary files with the values I need. However, when I attempt to use sed to process these files after the loop completes, sed reports that there is "No such file or directory".
After the program exits I can open the two files I created and filled in the loop and they have the appropriate values, so why is sed telling me they don't exist? I even tried to use cat after the loop and I get the same error message. Any ideas? I'm at a loss and so close to completing this script.
Code:
# parse out, de-dupe and clean up failed source UNC paths.
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path
# create array of failed source UNC paths.
mapfile -t PATH <path
# parse out and save the source folder path and filenames from PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
echo "${PATH["$x"]##*\\}" >>file
echo "${PATH["$x"]%\\*}" >>folder
x=$(( $x + 1 ))
done
# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed
Last edited by bytesnoop; 05-28-2012 at 04:49 PM..
Reason: corrected code
$ ./NEW-rc_logparser-denied.sh GS_Data3.txt
./NEW-rc_logparser-denied.sh: line 26: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 27: sed: No such file or directory
---------- Post updated at 04:36 PM ---------- Previous update was at 04:19 PM ----------
I know I'll eventually get this sorted out so I started looking at my code and seeing what I could do to make it more efficient. Since I have to duplicate the folder path from my source location to my destination location, I have decided to try out 'tee'. I am getting the same issues as I previously stated, take a look at code and STDERR:
Code:
# parse out and save the source folder path and filenames from
# PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
echo "${PATH["$x"]##*\\}" >>file
echo "${PATH["$x"]%\\*}" |tee -a folder >dest
x=$(( $x + 1 ))
done
# wrap each line from temp files in double quotes and assign temporary drive letter to destination path.
sed 's/^/"/; s/$/"/' file >file.sed
sed 's/^/"/; s/$/"/' folder >folder.sed
sed -r 's/^(.{1,2})/"B:\\/; s/$/"/' dest >dest.sed
Code:
$ ./NEW-rc_logparser-denied.sh GS_Data3.txt
./NEW-rc_logparser-denied.sh: line 22: tee: No such file or directory
./NEW-rc_logparser-denied.sh: line 27: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 28: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 29: sed: No such file or directory
The temporary drive letter will be replaced by the computer forensics data collection engineer after the batch file is created using WordPad or something similar. That is, if I can ever get this damn script to work.
Last edited by Scrutinizer; 05-28-2012 at 05:26 PM..
Reason: code tags
# parse out, de-dupe and clean up failed source UNC paths.
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path
# create array of failed source UNC paths.
mapfile -t PATH <path
# parse out and save the source folder path and filenames from PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
echo "${PATH["$x"]##*\\}" >>file
echo "${PATH["$x"]%\\*}" >>folder
x=$(( $x + 1 ))
done
# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed
I have no idea what does go wrong with your script because i can't see a (syntactical) error and a first test worked for me. I can tell you, though, that you are going in circles.
You "cook up" a file <path> in your first line, which should be a single sed (awk, text-filter-of-your-choice) statement, while it is a whole pipeline of different commands. That could and should be optimized, but let us put that aside for the moment.
After creating file <path> you parse it into an array PATH (which is at least dangerous, because PATH is a special variable to the shell) and then circle through this array writing two new files. What is the array for when you already have the same information in your file?
Code:
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path
typeset pathname=""
while read pathname ; do
echo "${pathname##*\\}" >>file
echo "${pathname%\\*}" >>folder
done < path
# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed
Then you use two sed-invocations to add double quotes. Why don't you add them already in the echo-statements? It is possible to escape characters and use them literally instead of their special meaning to the shell:
I won't even mention using full-qualified pathnames ("/path/to/file" instead of "file", etc.) inside of scripts because otherwise the result files are going to go wherever the script is called from - ever thought about putting this script in cron? You're guaranteed to have fun.
Once you overwrite it with this garbage, your shell will no longer be able to find external commands like sed.
Wow, that was the culprit?! Sorry to have waisted your time. I appreciate the help and thanks for helping me learn.
---------- Post updated at 06:54 PM ---------- Previous update was at 06:42 PM ----------
Quote:
Originally Posted by bakunin
You "cook up" a file <path> in your first line, which should be a single sed (awk, text-filter-of-your-choice) statement, while it is a whole pipeline of different commands. That could and should be optimized, but let us put that aside for the moment.
Definitely the least of my worries but I will definetly look into sed for this. I know I can make sed "grep" out the string and since if I don't use the global option I take care of de-duplication. My only issue with employing sed would be, how do I delete the first 54 characters of each line it finds? I got it to work before in another part of my code by haven't been able to do it since. I'll figure it out.
Quote:
Originally Posted by bakunin
After creating file <path> you parse it into an array PATH (which is at least dangerous, because PATH is a special variable to the shell) and then circle through this array writing two new files. What is the array for when you already have the same information in your file?
My initial program didn't use array's but I couldn't get a while loop to read through the <path> file and then process the strings with the string operations for folder and filename. I found that by passing the elements of the array in to the while loop the string operations worked and achieved my goal.
Quote:
Originally Posted by bakunin
Code:
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path
typeset pathname=""
while read pathname ; do
echo "${pathname##*\\}" >>file
echo "${pathname%\\*}" >>folder
done < path
# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed
I love this idea, and it was something I was trying to do and failed at (see previous excuse). While this looks sound, it does not populate the files correctly. The file <file> contains the entire string from <path> and <folder> contains just <"">. For now I am sticking with my array.
Quote:
Originally Posted by bakunin
Then you use two sed-invocations to add double quotes. Why don't you add them already in the echo-statements? It is possible to escape characters and use them literally instead of their special meaning to the shell:
I guess at some point I got too carried away with sed and didn't even thing to escape the quotes. Thanks, this makes it much cleaner.
Thanks a million for all the help. You made my day, I hope I can repay the favor to someone on this forum some day.
Hello,
I am creating a file with all the source folders included in my git branch, when i grep for the used source, i found source included as relative path instead of absolute path, how can convert relative path to absolute path without changing directory to that folder and using readlink -f ? ... (4 Replies)
Hi All,
Need some help in parsing a directory listing .. output into 2 files
Input file
level1,/level2/level3/level4/ora001,10,IBB23
level1,/level2/level3/level4/ora001/blu1,,IBB23
level1,/level2/level3/level4/ora001/clu1,,IBB23
level1,/level2/level3/level4/ora002,,IBB24... (10 Replies)
Hi all,
Using sh/csh, unfortunately shell scripts are not my strong suit.
Trying to write a script that gets called from a program for pre-processing.
The program passes individual components of a UNC (//server/path1/path2/filename).
Thus the unc path of: //server/path1/path2/filename, is... (7 Replies)
I am looking to parse a text file output and set variables based on what is cropped from the parsing.
Below is my script I am looking to add this feature too.
All it does is scan a certain area of users directories for anyone using up more than X amount of disk space. It then writes to the... (4 Replies)
I'm trying to use Perl on Windows (Doh!) to connect to a folder on a Domain Controller via UNC.
Right now, I have
perl -e "`runas /user:DOMAIN\\Username dir \\\\SERVER\\d\$\\Path`"
This does not seem to connect nor does it prompt for password. Should I try throwing it into a script and... (0 Replies)
Hi,
I have a file abcd.txt which has contents in the form of full path file names i.e.
$home> vi abcd.txt
/a/b/c/r1.txt
/q/w/e/r2.txt
/z/x/c/r3.txt
Now I want to retrieve only the directory path name for each row
i.e
/a/b/c/
/q/w/e/
How to get the same through shell script?... (7 Replies)
Hi,
I am trying to connect to a remote server using Plink tool. Both my local and remote machines are Windows. On remote server, I have OpenSSH server installed. I am able to run commands on remote machine but there is some problem with long UNC path, which I noticed today.
For... (3 Replies)
Hi all,
Hope all the expert can help me in this situation.
Let say I have one file with multiple record like below:
NAME=FRAGMENT
LANGUAGE=1
DIALECT=0
GENDER=NONE
FILE=TEST1
DIRECTORY=D:/DETAILS/1/0/test1.txt
END
NAME=FRAGMENT
LANGUAGE=1
DIALECT=0
GENDER=NONE (13 Replies)
Hi,
I have a file with about 60 lines of path:
app-defaults/boxXYZ.......
I want to change this to /my/path/goes/here/app-defaults/boxXYZ, but of course vi doesn't like the regualr :s/old/new/ command.
Is there any other quick way to do this?
Thanks ;) (2 Replies)
05-28-2012
Wow, that was the culprit?! Sorry to have waisted your time. I appreciate the help and thanks for helping me learn.