UNIX for Dummies Questions & Answers

Hello,

I am practicing creating loops in bash scripting. I am trying to understand the different uses of the for loop. I have a script I made below with some help.

Code:

#/bin/bash

#This script will take the extension of a file in ls command and echo a certain statement based on what type of file it is (.sh , .txt etc)

for filename in $(ls)

	do
	ext=${filename#*.}
		case "$ext" in
		sh) echo "This a script file";;
		txt) echo "This is a text file";;
		*) echo "Never mind, it is none of these 2" ;;
		esac
	done

The objective was to take the output of the "ls" command and classify the files found as either script or text file, based on the extracted characters after the "." in the name. I am wanting to know what does "filename" mean in the for loop. I understand it takes the filenames from the output of "ls" and runs the action, but conceptually what does "filename" represent? I want to find other things I can put instead of "filename" and learn more so I could search the topic, but I don't know what it was called. Thanks!

filename represents each element in the list returned by $(ls). I will say now that for is not the right way to do this task.

Each word output by ls is assigned into filename variable. That is the name you gave on the for line. The reason for is bad is because a filename with a space will actually be treated as two words because of "word splitting", but let's continue anyway...

filename can be any variable name. $(ls) can be replaced by any list. for example:

Code:

for number in 3 5 8 21; do
  echo "the next number is $number"
  echo "makes the total: $((sum+=number))"
done

This is a proper list. If you want a properly split list of files, use a glob:

Code:

for file in *; do
  case $file in
  *.txt) echo "$file is a text file" ;;
  *.sh) echo "$file is a shell script" ;;
  *) echo "i'm not sure what $file is" ;;
  esac
done

We can leave out the parameter expansion to get the extension and just do that logic in the case patterns.

I apologize as I am new to Linux/Unix scripting. I understand what you mean by "filename" can be any variable and "ls" can be any list. That makes sense. However, what do you mean by properly split list of files? Also what is a glob? Also you used a wildcard (*) in the "for" statement, so how does the variable $file receive input? In my example, I gave the output of the ls command (truncated to the characters after the (.)) for the variable $ext.

Thank you in advance.

There seem to be a few misconceptions at work here:

First, about $(ls). This is a device called "process substitution" and it works like this: the command inside the brackets is executed, its output is taken and put on the commandline. Then the resulting commandline is executed. In your case the command is "ls" but it could be any simple or cmplicated command:

Code:

echo "foo $(ls | grep '^a') bar"

This will first execute "ls", then feed its output to "grep '^a'" (which will filter out all filenames starting with "a"). Suppose there are two files, "afile1" and "afile2". These strings will be put on the command line replacing the original command, resulting in:

Code:

echo "foo afile1 afile2 bar"

and this command will finally be executed. You can use this always instead of a normal string.

The for-loop in shell languages works quite differently than in other languages. Normally there are 4 elements to a for-loop: a variable, a starting value, an increment and an endcondition. The variable is loaded with the starting value, then the loop is executed each time incrementing the loop variable, until the endcondition is finally reached, upon which the loop is terminated:

Code:

for( i=1; i<=10; i++ )

this means: the loop variable is "i" and is loaded initially with the value 1 ("i=1"). Each iteration of the loop the value is incremented by 1 ("i++", which is an abbreviation for "i=i+1") and each iteration the end condition ("i<=10") is checked. Once i becomes bigger than 10, the loop terminates.

A shell for-loop works similar, but it is given a list of values to set the variable to:

Code:

for X in a b c d e f g

means: in the first iteration set variable "X" to value "a", in the second to "b", and so on, until the last, "g", is reached, upon which the loop terminates.

Combine this with the above and you see:

Code:

for filename in $(ls)

"filename" is the name of the variable to be used and the list of values is the output of "ls", which produces a list of filenames.

Finally:
The "type" of a file is - unlike in DOS or Windows - not determined by its extension. In Windows every file with the extension ".EXE" is executable, everything else is not. In Unix there are the rights (issue "ls -l" to see them on the leftmost column), which determine if a file is executable or not. If the "x"-flag is set the OS will try to execute it, if not, then not.

It is true that most name files conventionally according to their respective types: ".sh" for shell scripts, ".txt" for text files and so on. But this is just convention! There is no hard rule to do so and if you name a shell script "script.txt" it will still get executed if you have set the execute-flag (via the "chmod"-command). This may not be a god idea, because it will be confusing, but technically there is nothing that keeps you from doing it.

The "type" of a file is a different concept in UNIX and it is controlled via the file "/etc/magic". This file contians rule definitions what constitutes a certain file type based on the respective files contents. For instance, there could be a rule that a file starting with "#! /bin/sh" in the first line is a shell script. execute

Code:

file /path/to/file.in.question

and the file command will figure out the file type based on consulting the rules in "/etc/magic". You can even extend the ruleset with your own rules and file types.

I hope this helps.

bakunin

See the following shell session example:

Code:

mute@tiny:~$ mkdir shah9250
mute@tiny:~$ cd shah9250/
mute@tiny:~/shah9250$ touch 'file with spaces'
mute@tiny:~/shah9250$ for f in $(ls); do echo "work on [$f]"; done
work on [file]
work on [with]
work on [spaces]
mute@tiny:~/shah9250$ for f in *; do echo "work on [$f]"; done
work on [file with spaces]

We know there is only one file and the loop should only be called once, but with that use of $(ls) it's not. Quoting and word splitting are the toughest parts of learning the shell, in my opinion. Once that clicks things are much easier.

Glob is filename expansion. * is a wildcard that matches anything. You can simply do echo * or echo *.txt to show all files or all files ending in .txt. My meaning wasn't to distract from the for loop, just for down the road you'll want to use best practices. Filenames can contain any character except the NUL character and the path delimiter / and a great deal of care must be taken to make robust scripts.

I cannot recommend Greg's wiki enough. The bash guide on the site is excellent value for the price: BashGuide - Greg's Wiki

Thank you both for the help. Your responses has cleared up a few of my misconceptions. Last question though , is how does (*) cause the system to ignore spaces in file names? Also how does (*) know to refer to the output of (ls) and then perform the action? If (*) is a wildcard and refers to anything, how does using (*) in the for loop cause the system to read the output of ls (ignoring spaces)? Thanks!

It doesn't "ignore". * gives valid filenames, split where they should be.

But cramming the result of * into one string flattens it out. It forgets where the filenames begin and end. You have it completely backwards. * does not need ls's help to do anything. You can plug * into anything that takes multiple arguments, and the shell will convert it into filenames for you, before the command is even run.

It doesn't. ls isn't needed at all.