Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Count unique words
Top Forums UNIX for Beginners Questions & Answers Count unique words Post 302991765 by wisecracker on Thursday 16th of February 2017 05:51:33 AM
Old 02-16-2017
(Apologies for any typos.)
RudiC has already given you a starter with this, assume 'FN' is pointing to an Entertainment text file:-
Code:
FN='31_October_2012_Entertainment1094.txt'
TMP=${FN##*_}
TMP=${TMP%%[0-9]*}

This would give you a result inside the TMP variable, Entertainment .

So your logic would require a count for each file containing 'Entertainment'.
Similarly for the others.

So what would your logic be to obtain your count(s) per category?

You are here to learn how to do it for yourself and the best way is to attempt something no matter how bad your code looks. We are not here to ridicule your attempts but to correct your logic so that you understand what is going on and become capable of doing it again if need be.
If it is JUST the filenames you want then this will _perhaps_ help:-
ls *.txt > /your/path/to/filenames which will create a single text file with your thousands of filenames ONLY inside it.
grep is your friend here.

However if you intend to read EACH individual file to count these words also, then this is a totally different _animal_.
Last edited by wisecracker; 02-16-2017 at 07:12 AM.. Reason: Added the 'grep' line.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to read all the unique words in a text file

How can i read all the unique words in a file, i used - cat comment_file.txt | /usr/xpg6/bin/tr -sc 'A-Za-z' '/012' and cat comment_file.txt | /usr/xpg6/bin/tr -sdc 'A-Za-z' '/012' but they didnt worked..... (5 Replies)
Discussion started by: aditya.ece1985
5 Replies

2. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

3. Shell Programming and Scripting

Shell script to find out words, replace them and count words

hello, i 'd like your help about a bash script which: 1. finds inside the html file (it is attached with my post) the code number of the Latest Stable Kernel, 2.finds the link which leads to the download location of the Latest Stable Kernel version, (the right link should lead to the file... (3 Replies)
Discussion started by: alex83
3 Replies

4. Homework & Coursework Questions

unique words in files of folder and its subfolders

Hello, I tried to count all unique words of all files in one folder and its subfolders. Can anybody say me, why this doesnt work: ls| find -d | cat | tr "\ " "\n"| uniq -u | wc -l ??? Cat writes only the names of those files, but not the wors, which should be in them. Thanks for any advice. ... (9 Replies)
Discussion started by: Dworza
9 Replies

5. Shell Programming and Scripting

display unique words.

I am having a file with duplicate words how can I eliminate them ant,bat bat,cat cat a.txt | grep -bat | awk '{print $1}' expecting o/p as ant,bat,cat How can I display the output as ant,bat,cat in a single line and no duplicates exists. (2 Replies)
Discussion started by: shikshavarma
2 Replies

6. Shell Programming and Scripting

Unique words in each line

In each row there could be repetition of a word. I want to delete all repetitions and keep unique occurrences. Example: a+b+c ab+c ab+c abbb+c ab+bbc a+bbbc aaa aaa aaa Output: a+b+c ab+c abbb+c ab+bbc a+bbbc aaa (6 Replies)
Discussion started by: Viernes
6 Replies

7. Shell Programming and Scripting

awk to count using each unique value

Im looking for an awk script that will take the unique values in column 5, then print and count the unique values in column 6. CA001011500 11111 11111 -9999 201301 AAA CA001012040 11111 11111 -9999 201301 AAA CA001012573 11111 11111 -9999 201301 BBB CA001012710 11111 11111 -9999 201301... (4 Replies)
Discussion started by: ncwxpanther
4 Replies

8. Shell Programming and Scripting

How count the number of two words associated with the two words occurring in the file?

Hi , I need to count the number of errors associated with the two words occurring in the file. It's about counting the occurrences of the word "error" for where is the word "index.js". As such the command should look like. Please kindly help. I was trying: grep "error" log.txt | wc -l (1 Reply)
Discussion started by: jmarx
1 Replies

9. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

10. Shell Programming and Scripting

Regex to identify unique words in a dictionary database

Hello, I have a dictionary which I am building for the Open Source Community. The data structure is as under HEADWORD=PARTOFSPEECH=ENGLISH MEANING as shown in the example below अ=m=Prefix signifying negation. अँहँ=ind=Interjection expressing disapprobation. अं=int=An interjection... (2 Replies)
Discussion started by: gimley
2 Replies

LEARN ABOUT DEBIAN

mbtg

mbtg(1) 						      General Commands Manual							   mbtg(1)

NAME

       MBTG - Memory Based Tagger generator

SYNOPSYS

       mbtg -T <filename> -s <setting filename>

       or

       mbtg [options]

DESCRIPTION

       This programs generates, based on a tagged corpus, all the files needed to be able to tag a text with mbt.

OPTIONS

       -h or --help
	      show help

       -T <tagged training corpus file>

       or

       -E <enriched tagged training corpus file>

       All further options have reasonable defaults, so using them is only needed for the experienced user. See the mbt manual for more details.

       -s settingsfile
	      mbtg creates this file, which can be used to run mbt with minimal effort. (like mbt -s settings -T somefile)

       -p pattern
	      the pattern for known words (default ddfa)

       -P pattern
	      the pattern for unknown words (default dFapsss)

       -% <number>
	      filter threshold for ambitag construction (default 5%)

       -l <lexiconfile>

       -L <file with list of frequent words>

       -r <ambitagfile>

       -k <known words case base>

       -u <unknown words case base>

       -K <known words instances file>

       -U <unknown words instances file>

       -V or --version
	      show version info

       -e <sentence delimiter> (default '<utt>')

       -X
	      keep the intermediate files

       -Otimbl options
	       (Note: there is NO SPACE between O and the options)
		<options>   classifier options for both known and unknown words instances bases
		K: <options>   classifier options for known words instance base
		U: <options>   classifier options for unknown words case base
		valid timbl options are: a d k m q v w x -

BUGS

       possibly

AUTHORS

       Ko van der Sloot Timbl@uvt.nl

       Antal van den Bosch Timbl@uvt.nl

SEE ALSO

       timbl(1) mbt(1) mbtserver(1)

								   2011 march 21							   mbtg(1)
All times are GMT -4. The time now is 06:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy