UNIX for Dummies Questions & Answers

Hi!!
I am trying to write a program which allows me to count how many times I used the same word in a text:

Code:

 {$0 = tolower ($0)
 gsub (/[^[:alnum:]_[:blank:]]/, "", $0)
  for (i = 1; i <= NF; i++)
  freq[$1]++
  
  }
 
 END {
  for (word in freq)
  printf "%s\t%d\n", word, freq[word]

It seems work but I would like to exclude the articles (in Italiana: la, le, gli, lo), Do I have to use "gsub" another time? Like:

Code:

gsub (/la/, "", $0)

Then, I would like to substitute the overused words with synonyms which are in another file, how can I do? I have no idea.
Thank you

What you call "word" might be a problematic concept:
Lasciate ogni speranza voi ch' entrate. (Dante)

How would your algorithm split that into words? Is "ch" a word as per your definition? And shouldn't it be counted as the same as "che"?

I hope this helps.

bakunin

not much

Yes, use a second (or further) gsub. I think you can use the "alternate" (= or) regex operator | . Might carry some difficulties, e.g. upper case at the begin of a sentence, or trailing punctuation. Here I did this for the German words Der, der, die, das, dem, den, des (all with optional leading capital letter):

Code:

(^[dD]|[[:space:]][dD])(ie|e[mnrs]|as)[[:punct:][:space:]]