UNIX for Dummies Questions & Answers

HA! You're cool, man. I didn't know how to do all this at one point... Let me answer your second question first, then I'll break the script down.

You can actually do short scripting at a command line without putting it into a text file. Generally, if I come up with a script that's 5-10 lines and I just need to run it once, I'll do it at a command line. If I know I'm going to use it several times, I'll put it into a script. I'll give you a quick tutorial on doing both, though you might go through the "Shell Programming and Scripting Q & A" Forum for more information.

< Putting the script in a file >

I assume that you know vi or some unix text editor. With that assumtion in mind, just makeup a name for your script and type in the example script that I gave you. For example:

# vi myScript
for filename in `cat somefile`
do
ls -lc $filename >> somefile2
done
mv somefile2 somefile
~
~
~
Then write/quit out of the file. You now have a file in your current directory named myScript. Before we can run it, though, we need to make it executable. That is easy enough.
# chmod u+x myScript <-- This makes the file executable by just you. If you wanted anyone to be able to execute it:
# chmod a+x myScript <-- a=all and x=execute.
Now, type:
# ./myScript <-- The dot slash tells Unix to run the program myScript which is located in the directory where you are right now (called your present working directory).

After your script runs, you'll have an output file called "somefile". That should have a list of the files that match your search pattern and information about those files.

< Running the script at a command line >
As I said earlier, you can write 1-time use quick and dirty scripts at the command line. Like so:
# for filename in `cat somefile` <-- when you press return, you drop to
what is called a sub-prompt, or
secondary prompt. On most
systems, it is a greater than sign by
default. So now you have:
# for filename in `cat somefile`
> do
> ls -lc $filename >> somefile2
> done
# mv somefile2 somefile <-- Note that the line after "done"
returns to a regular # prompt. This
indicates you have returned to the
primary shell prompt.
And again, you should have an output file called "somefile" in your present working directory.

Now, to explain what the file is doing. This is an example of a loop. In a nutshell, there are generally three types of loops: while, until, and for.
A while loop generally begins with a value that is true and runs while that condition is true.
An until loop generally begins with a condition that is false and runs until that condition is not false anymore.
A for loop is usually used when you know exactly how many times you want to repeat an action. This example uses a for loop.
For the example, we'll pretend that the contents of somefile are:
john.dbf
paul.dbf
george.dbf
ringo.dbf

So, the first line reads:
for filename in `cat somefile`
for: says "I want to start a for loop"
filename: here, we are creating a temporary variable called "filename". You can use whatever you want right here, though.
in: In a for loop, the format is always for variableName in expression. So always put "in" right here.
`cat somefile`: OK, couple of important things to note here. First, I used backticks, as opposed to single-quotes. When you script things in the shell, the backtick, single-quote and double-quote all perform different tasks, so it is very important that you use the right type of quote. If you are using a standard PC keyboard, the backtick is generally located on the same key as the tilde (~), just to the left of the number 1.
So what are the backticks doing? In this case, the backticks are saying "Run the command cat somefile and return the output right here." So the first time the for loop runs, it takes the first line of somefile and stores it in the variable filename that we made. Then the rest of the loop executes.

Next line:
do: This is just more syntax of a for loop. So do always goes here.

Next line:
ls -lc $filename >> somefile2: This runs the ls -lc command and takes as it's argument whatever the value of our temporary variable filename is. The $ in front of filename tells Unix "I want the value stored in the variable filename (or whatever you decided to call it), instead of literlly using the word 'filename'". Now since this is our first run through the loop, the value of $filename is john.dbf. So, Unix reads this line as:
ls -lc john.dbf >> somefile2

That last bit (>> somefile2) says "Run the output of ls -lc $filename and append the output to a file called somefile2". A couple of notes on this.

I assume you know the difference between using > and >> to place data in a file, if not:
command > someFilename will write the output of command to someFilename. If someFilename already exists, the contents are overwritten.
-- however --
command >> someFilename will append the output of command into someFilename, meaning that if someFilename already existed an had information in it, the new contents would be added to the bottom, instead of overwriting the original.

It is important in loops to use >>. If you used >, then everytime the loop executes you are overwriting your output file and when the loop is finished, all you would have in your output is information on the file ringo.dbf.

Next line:
done: More for loop syntax. The last line of a for loop will always be "done".

Next line:
mv somefile2 somefile: Technically this line is optional. When reading a file into a shell script, you N-E-V-E-R save the output back to the original file. This holds true for using utilities like sed, awk, perl, etc... All kinds of weird and interesting things happen when you try and write the output right back to the file from which you are still reading. So, as a general practice, when you read in from fileA, write the output to fileB. When you are finished, you have 2 choices. fileA has your raw, unproccessed data and fileB has the information you are actually looking for. If you no longer need the raw data, then use the move command to rename fileB back to fileA like so:
# mv fileB fileA

Note that doing this erases the raw data file and replaces it with the final results. If, however you wanted to keep the raw data and have the output in a separate file, then strike the last line of my suggested script. This would leave you with a file called "somefile" which has your raw data and a file called "somefile2" that has your proccessed information.

< Putting it all together >
When you finish typing it all in the the loop runs, here is what happens:
Unix sets up a for loop with a temporary variable called filename. It runs the command "cat somefile" and receives the first line of that output (john.dbf) and stores it to our temporary variable "filename".
The loop then makes it's first pass. The ls -lc command is run on john.dbf and the output is appended to somefile2. Since somefile2 does not exist at this point, it is created, and then the output of ls -lc john.dbf is stored in that file.
Then the loop finishes its first pass and runs again. It reads the next line of somefile and gets paul.dbf, which it stores in our temporary variable "filename". The loop then executes "ls -lc paul.dbf" and stores the information at the bottom of somefile2 under the information for john.dbf. Then the second pass finishes.
The third pass begins. The for loop reads in the next line of somefile, which is now geroge.dbf, and stores it to our temporary variable somefile. The ls command executes against george.dbf and stores the output at the bottom of somefile2, after the output of john.dbf and paul.dbf. The third run finishes.
The fourth run begins. The for loop reads in the next line of somefile, which is ringo.dbf, stores it to the temporary variable "filename". The ls command is run against the ringo.dbf and the output is appended to the bottom of the file somefile2. The fourth run finishes.
The loop executes a fifth time. The for loop gets an "end of file" message and proceeds straight to done, and the loop closes.
If you choose to use the mv command here, it runs.

I hope this helps and I also apologize for being verbose. When I was learning shell scripting I got irritated when people would put some weird looking code up and then give me an even more confusing answer when I asked how to read it, so I tried to explain this code as I went along. If you really want to learn shell scripting, I recommend the following book.

Unix Shell Programming by Stephen G. Kochan and Patrick Wood (3rd Edition)
Sams Publishing
ISBN: 0-672-32490-3

Also, O'Reilly and Associates (www.ora.com) have dozens of books on shell scripting, vi, sed and awk, etc... I own almost all of those and find them extremely helpful.

Thanks.

Thanks for an extremely helpful answer, I just have an additional question

the line:

# for filename in `cat somefile`

Does not open a sub-promt when I try to do the commands without a script, it just returns

for: Command not found

Do you think it is my UNIX version that gives this respons or am I doing something wrong?

Another question; should I define filename in any way before I start?

Yet again, many thanks for all help,

Slire

Hmm... the for command is usually built into your shell. Type this:

# echo $SHELL

You should get something back like:


/usr/bin/ksh or
/bin/sh

something like that. If your shell doesn't know the for command, you can easily change shells. I generally use the Korn shell, the Unix command for which is ksh. So, let's pretend you are running something other than Korn. At the prompt, type:

# ksh
# (<-- now you're running in Korn shell)

You can verify you're running in Korn shell by issuing the 'echo $SHELL' command again and seeing if the output changes. Also, the # I've been using in the examples is the default prompt for root in Korn. If your prompt is something different like a $ or > or ), you'll notice you're prompt change when you type ksh.


As far as declaring filename before hand, there is no need. You could use anything there for a variable. It will just be used for the loop and then Unix discards it. You cold write our pretend script this way:

for bubba in `cat somefile`
do
ls -lc $bubba >> somefile2
done
mv somefile2 somefile

The variable name can be absolutley anything. Most of my scripts I just use the letter "i". Just make sure that when you call that variable in your script that you use whatever name you gave it in the opening line.

Let me know about that shell. I'm really suprised that yours didn't have for built into it.

Hi,

echo $SHELL --> /local/bin/tcsh

If I write ksh things start to work, I get the sub-script but the script responds:

Cannot access cat: No such file or directory

I dont understand why this happends, what am I doing wrong???

Another question: how do I quit the sub-promt?

Even if it doesn't seems like it, I have started to get a clue about how this work, so thanks again,

Slire

I've never used tcsh, so that explains the inability to find for.

As far as "cannot access cat", again, make sure it is in your path. If you had a text file and did

# cat file

does it display any output or tell you that it can't open or find cat?

If cat does work by itself at the command line, make sure you are putting backticks around the cat command in the first line.

for variable-name in `cat filename`

If your using a single quote or a double quote it won't work.

As far as exiting the sub-prompt, if you've typed in the syntax correctly, you should exit the sub-prompt after you type done. Again, the basic syntax for a for loop is:

for some-variable in some-command-or-condition
do <-- literally type do here.
commands
done <-- literally type done here

If all of that is done correctly, what should happen is that after you type done, you won't get a cursor for a bit while your script runs. When your program is finished, it will return to whatever your original prompt was.

Let me know how things go.

and it works........

I had gone lost in the world of backticks and single quotes

H_K thanks for all help....... you have made my life a little more easier

Slire,

Can you post us the final script which is working for you?

Thanks,
Ravi