Sponsored Content
Top Forums UNIX for Advanced & Expert Users Recursive directory search using ls instead of find Post 302537491 by bearvarine on Friday 8th of July 2011 10:34:27 AM
Old 07-08-2011
Use a program rather than a shell script

One of the phenomena I have noticed over my years of being involved in Unix/Linux is that people tend to over-use shell scripting.

The problem with shell scripting is, simply, performance. Its one thing to accomplish small to medium tasks with a shell script. But once you begin doing serious processing work, involving tight loops of text processing, you will very quickly run into trouble. The reason is because most things done in a shell script are done by small programs - cut, sed, awk, head, tail, etc. When you combine dozens of these in a loop that will be running heavily - the computer has to launch THOUSANDS of tiny programs to accomplish the overall task.

I have seen large powerful Unix systems brought to their knees by simple DB loader scripts done in ksh for this very reason.

The solution is to use a more appropriate software tool to solve the problem. If you really want to do it in shell scripting style, why not try it in perl or python? These programs will allow you to write a single program to accomplish the task, no spawning of child programs required. This will vastly speed up the program. I know this to be a fact, because I've had a simple perl file and text search program in my toolbox since 1994.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

find file with date and recursive search for a text

Hey Guyz I have a requirement something like this.. a part of file name, date of modification of that file and a text is entered as input. like Date : 080206 (MMDDYY format.) filename : hotel_rates text : Jim now the file hotel_rates.ZZZ.123 (creation date is Aug 02 2006) should be... (10 Replies)
Discussion started by: rosh0623
10 Replies

2. UNIX for Dummies Questions & Answers

Unix find command to print directory and search string

Hi i need to print pathname in which the string present using 'find' command sample output like this Pathname String to be searched ---------- -------------------- /usr/test/myfile get /opt/test/somefile get Thanks in... (4 Replies)
Discussion started by: princein
4 Replies

3. Shell Programming and Scripting

non recursive search in the current directory only

Hi, Am trying for a script which should delete more than 15 days older files in my current directory.Am using the below piece of code: "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/ls -altr {} \;" "find /tmp -type f -name "pattern" -mtime +15 -exec /usr/bin/rm -f {} \;" ... (9 Replies)
Discussion started by: puppala
9 Replies

4. Shell Programming and Scripting

search directory-find files-append at end of line

Hi, I have a command "get_data" with some parameters in few *.text files of a directory. I want to first find those files that contain this command and then append the following parameter to the end of the command. example of an entry in the file :- get_data -x -m50 /etc/web/getid this... (1 Reply)
Discussion started by: PrasannaKS
1 Replies

5. UNIX for Dummies Questions & Answers

Restricting a Find search to the current directory only

Hi All, I am trying to delete file (with a mtime older than 2 days) from the current directory ONLY using: find . -daystart -maxdepth 1 -mtime 2 -exec rm {} \; but this doesn't seem to work it is still find files in subdirectories which I don't want to delete. Please can anyone offer... (2 Replies)
Discussion started by: daveu7
2 Replies

6. Shell Programming and Scripting

Find command to search files in a directory excluding subdirectories

Hi Forum, I am using the below command to find files older than x days in a directory excluding subdirectories. From the previous forums I got to know that prune command helps us not to descend in subdirectories. Though I am using it here, not getting the desired result. cd $dir... (8 Replies)
Discussion started by: jhilmil
8 Replies

7. Shell Programming and Scripting

How to restrict Find only search the current directory?

hello, all I have googled internet, read the man page of Find, searched this forum, but still could not figure out how. My current directory is: little@wenwen:~$ pwd /home/little little@wenwen:~$ I want to use find command to list the files in my current directory, how should i write... (3 Replies)
Discussion started by: littlewenwen
3 Replies

8. UNIX for Dummies Questions & Answers

Help needed - find command for recursive search

Hi All I have a requirement to find the file that are most latest to be modified in each directory. Can somebody help with the command please? E.g of the problem. The directory A is having sub directory which are having subdirectory an so on. I need a command which will find the... (2 Replies)
Discussion started by: sudeep.id
2 Replies

9. UNIX for Dummies Questions & Answers

How to search in specific directory using find?

Hi, Is there any way to use find command and search only specific subdirectories in a directory. for example /home/d1 /home/d2 /home/d3 i want to search in the following directories /home /home/d1 /home/d2 i do not want the find command to search the /home/d3 directory. (6 Replies)
Discussion started by: Little
6 Replies

10. Shell Programming and Scripting

Recursive folder search faster than find?

I'm trying to find folders created by a propritary data aquisition software with the .aps ending--yes, I have never encountered folder with a suffix before (some files also end in .aps) and sort them by date. I need the whole path ls -dt "$dataDir"*".aps"does exactly what I want except for the... (2 Replies)
Discussion started by: Michael Stora
2 Replies
AUTOEXPECT(1)						      General Commands Manual						     AUTOEXPECT(1)

NAME
autoexpect - generate an Expect script from watching a session SYNOPSIS
autoexpect [ args ] [ program args... ] INTRODUCTION
autoexpect watches you interacting with another program and creates an Expect script that reproduces your interactions. For straightline scripts, autoexpect saves substantial time over writing scripts by hand. Even if you are an Expect expert, you will find it convenient to use autoexpect to automate the more mindless parts of interactions. It is much easier to cut/paste hunks of autoexpect scripts together than to write them from scratch. And if you are a beginner, you may be able to get away with learning nothing more about Expect than how to call autoexpect. The simplest way to use autoexpect is to call it from the command line with no arguments. For example: % autoexpect By default, autoexpect spawns a shell for you. Given a program name and arguments, autoexpect spawns that program. For example: % autoexpect ftp ftp.cme.nist.gov Once your spawned program is running, interact normally. When you have exited the shell (or program that you specified), autoexpect will create a new script for you. By default, autoexpect writes the new script to "script.exp". You can override this with the -f flag fol- lowed by a new script name. The following example runs "ftp ftp.cme.nist.gov" and stores the resulting Expect script in the file "nist". % autoexpect -f nist ftp ftp.cme.nist.gov It is important to understand that autoexpect does not guarantee a working script because it necessarily has to guess about certain things - and occasionally it guesses wrong. However, it is usually very easy to identify and fix these problems. The typical problems are: o Timing. A surprisingly large number of programs (rn, ksh, zsh, telnet, etc.) and devices (e.g., modems) ignore keystrokes that arrive "too quickly" after prompts. If you find your new script hanging up at one spot, try adding a short sleep just before the previous send. You can force this behavior throughout by overriding the variable "force_conservative" near the beginning of the generated script. This "conservative" mode makes autoexpect automatically pause briefly (one tenth of a second) before sending each char- acter. This pacifies every program I know of. This conservative mode is useful if you just want to quickly reassure yourself that the problem is a timing one (or if you really don't care about how fast the script runs). This same mode can be forced before script generation by using the -c flag. Fortunately, these timing spots are rare. For example, telnet ignores characters only after entering its escape sequence. Modems only ignore characters immediately after connecting to them for the first time. A few programs exhibit this behavior all the time but typically have a switch to disable it. For example, rn's -T flag disables this behavior. The following example starts autoexpect in conservative mode. autoexpect -c The -C flag defines a key to toggle conservative mode. The following example starts autoexpect (in non-conservative mode) with ^L as the toggle. (Note that the ^L is entered literally - i.e., enter a real control-L). autoexpect -C ^L The following example starts autoexpect in conservative mode with ^L as the toggle. autoexpect -c -C ^L o Echoing. Many program echo characters. For example, if you type "more" to a shell, what autoexpect actually sees is: you typed 'm', computer typed 'm', you typed 'o', computer typed 'o', you typed 'r', computer typed 'r', ... Without specific knowledge of the program, it is impossible to know if you are waiting to see each character echoed before typ- ing the next. If autoexpect sees characters being echoed, it assumes that it can send them all as a group rather than inter- leaving them the way they originally appeared. This makes the script more pleasant to read. However, it could conceivably be incorrect if you really had to wait to see each character echoed. o Change. Autoexpect records every character from the interaction in the script. This is desirable because it gives you the ability to make judgements about what is important and what can be replaced with a pattern match. On the other hand, if you use commands whose output differs from run to run, the generated scripts are not going to be correct. For example, the "date" command always produces different output. So using the date command while running autoexpect is a sure way to produce a script that will require editing in order for it to work. The -p flag puts autoexpect into "prompt mode". In this mode, autoexpect will only look for the the last line of program output - which is usually the prompt. This handles the date problem (see above) and most others. The following example starts autoexpect in prompt mode. autoexpect -p The -P flag defines a key to toggle prompt mode. The following example starts autoexpect (in non-prompt mode) with ^P as the toggle. Note that the ^P is entered literally - i.e., enter a real control-P. autoexpect -P ^P The following example starts autoexpect in prompt mode with ^P as the toggle. autoexpect -p -P ^P OTHER FLAGS
The -quiet flag disables informational messages produced by autoexpect. The -Q flag names a quote character which can be used to enter characters that autoexpect would otherwise consume because they are used as toggles. The following example shows a number of flags with quote used to provide a way of entering the toggles literally. autoexpect -P ^P -C ^L -Q ^Q STYLE
I don't know if there is a "style" for Expect programs but autoexpect should definitely not be held up as any model of style. For example, autoexpect uses features of Expect that are intended specifically for computer-generated scripting. So don't try to faithfully write scripts that appear as if they were generated by autoexpect. This is not useful. On the other hand, autoexpect scripts do show some worthwhile things. For example, you can see how any string must be quoted in order to use it in a Tcl script simply by running the strings through autoexpect. SEE ALSO
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, O'Reilly and Associates, January 1995. AUTHOR
Don Libes, National Institute of Standards and Technology expect and autoexpect are in the public domain. NIST and I would appreciate credit if these programs or parts of them are used. 30 June 1995 AUTOEXPECT(1)
All times are GMT -4. The time now is 07:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy