Visit Our UNIX and Linux User Community

how to read all the unique words in a text file

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting how to read all the unique words in a text file
# 1  
Old 11-30-2007
how to read all the unique words in a text file

How can i read all the unique words in a file, i used -

cat comment_file.txt | /usr/xpg6/bin/tr -sc 'A-Za-z' '/012'
cat comment_file.txt | /usr/xpg6/bin/tr -sdc 'A-Za-z' '/012'
but they didnt worked.....
# 2  
Old 11-30-2007
Does uniq somefile help?
# 3  
Old 11-30-2007
yes u will be using uniq only but the first part is not working i used -
cat comment_file.txt | /usr/xpg6/bin/tr -sdc 'A-Za-z' '/012' | sort | uniq -c
# 4  
Old 11-30-2007
This one from the archives may help. Although it shows non-unique non-unique words you would need to tweak it.
# 5  
Old 11-30-2007
thankx anyways
# 6  
Old 11-30-2007
this is the solution ---

cat comment_file.txt | tr " " "\n" | sort | uniq -c | wc -l

Previous Thread | Next Thread
Test Your Knowledge in Computers #470
Difficulty: Medium
As of December 2018 the NTP pool consisted of around 7,700 active time servers on IPv4.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Count unique words

Dear all, I would like to know how to list and count unique words in thousands number of text files. Please help me out thanks in advance (9 Replies)
Discussion started by: imranrasheedamu
9 Replies

2. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies

3. Shell Programming and Scripting

Unique words in each line

In each row there could be repetition of a word. I want to delete all repetitions and keep unique occurrences. Example: a+b+c ab+c ab+c abbb+c ab+bbc a+bbbc aaa aaa aaa Output: a+b+c ab+c abbb+c ab+bbc a+bbbc aaa (6 Replies)
Discussion started by: Viernes
6 Replies

4. Shell Programming and Scripting

display unique words.

I am having a file with duplicate words how can I eliminate them ant,bat bat,cat cat a.txt | grep -bat | awk '{print $1}' expecting o/p as ant,bat,cat How can I display the output as ant,bat,cat in a single line and no duplicates exists. (2 Replies)
Discussion started by: shikshavarma
2 Replies

5. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

6. Shell Programming and Scripting

extract unique pattern from large text file

Hi All, I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like 12345- , i tried with egrep ,eg : egrep "+" text.txt but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies

7. Programming

fscanf: read words from file

Hi I have a file like that: 1 2 3 4 5 6 7 8 and I want print on stdout: 1 3 8 in other words i want choose what print out. I was thinking to use fscanf as: fscanf(file_in,"%d %d %d",&a, &b,&c); but in this way i get: 1 2 3 Is there a solution using fscanf to obtain my... (2 Replies)
Discussion started by: Dedalus
2 Replies

8. Shell Programming and Scripting

extracting unique lines from text file

I have a file with 14million lines and I would like to extract all the unique lines from the file into another text file. For example: Contents of file1 happy sad smile happy funny sad I want to run a command against file one that only returns the unique lines (ie 1 line for happy... (3 Replies)
Discussion started by: soliberus
3 Replies

9. Shell Programming and Scripting

To read and separate number and words in file and store to two new file using shell

hi, I am a begginer in unix and i want to know how to open a file and read it and separate the numbers & words and storing it in separate files, Using shell scripting. Please help me out for this. Regards S.Kamakshi (2 Replies)
Discussion started by: kamakshi s
2 Replies

10. Shell Programming and Scripting

Read words from file and create new file using K-shell.

Hi All, Please help me in creating files through K-shell scripts. I am having one file in this format. OWNER.TABLE_NAME OWNER.TABLE_NAME1 OWNER1.TABLE_NAME OWNER1.TABLE_NAME1 I want to read the above file and create new file through k shell script. The new file should looks like this.... (4 Replies)
Discussion started by: bsrajirs
4 Replies

Featured Tech Videos