Sponsored Content
Top Forums Shell Programming and Scripting how to read all the unique words in a text file Post 302148129 by aditya.ece1985 on Friday 30th of November 2007 12:59:19 AM
Old 11-30-2007
yes u will be using uniq only but the first part is not working i used -
cat comment_file.txt | /usr/xpg6/bin/tr -sdc 'A-Za-z' '/012' | sort | uniq -c
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read words from file and create new file using K-shell.

Hi All, Please help me in creating files through K-shell scripts. I am having one file in this format. OWNER.TABLE_NAME OWNER.TABLE_NAME1 OWNER1.TABLE_NAME OWNER1.TABLE_NAME1 I want to read the above file and create new file through k shell script. The new file should looks like this.... (4 Replies)
Discussion started by: bsrajirs
4 Replies

2. Shell Programming and Scripting

To read and separate number and words in file and store to two new file using shell

hi, I am a begginer in unix and i want to know how to open a file and read it and separate the numbers & words and storing it in separate files, Using shell scripting. Please help me out for this. Regards S.Kamakshi (2 Replies)
Discussion started by: kamakshi s
2 Replies

3. Shell Programming and Scripting

extracting unique lines from text file

I have a file with 14million lines and I would like to extract all the unique lines from the file into another text file. For example: Contents of file1 happy sad smile happy funny sad I want to run a command against file one that only returns the unique lines (ie 1 line for happy... (3 Replies)
Discussion started by: soliberus
3 Replies

4. Programming

fscanf: read words from file

Hi I have a file like that: 1 2 3 4 5 6 7 8 and I want print on stdout: 1 3 8 in other words i want choose what print out. I was thinking to use fscanf as: fscanf(file_in,"%d %d %d",&a, &b,&c); but in this way i get: 1 2 3 Is there a solution using fscanf to obtain my... (2 Replies)
Discussion started by: Dedalus
2 Replies

5. Shell Programming and Scripting

extract unique pattern from large text file

Hi All, I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like 12345- , i tried with egrep ,eg : egrep "+" text.txt but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies

6. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

7. Shell Programming and Scripting

display unique words.

I am having a file with duplicate words how can I eliminate them ant,bat bat,cat cat a.txt | grep -bat | awk '{print $1}' expecting o/p as ant,bat,cat How can I display the output as ant,bat,cat in a single line and no duplicates exists. (2 Replies)
Discussion started by: shikshavarma
2 Replies

8. Shell Programming and Scripting

Unique words in each line

In each row there could be repetition of a word. I want to delete all repetitions and keep unique occurrences. Example: a+b+c ab+c ab+c abbb+c ab+bbc a+bbbc aaa aaa aaa Output: a+b+c ab+c abbb+c ab+bbc a+bbbc aaa (6 Replies)
Discussion started by: Viernes
6 Replies

9. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies

10. UNIX for Beginners Questions & Answers

Count unique words

Dear all, I would like to know how to list and count unique words in thousands number of text files. Please help me out thanks in advance (9 Replies)
Discussion started by: imranrasheedamu
9 Replies
uniq(1) 						      General Commands Manual							   uniq(1)

Name
       uniq - report repeated lines in a file

Syntax
       uniq [-udc[+n][-n]] [input[output]]

Description
       The  command  reads  the  input	file comparing adjacent lines.	In the normal case, the second and succeeding copies of repeated lines are
       removed; the remainder is written on the output file.  Note that repeated lines must be adjacent in order to be found.  For further  infor-
       mation, see

Options
       The n arguments specify skipping an initial portion of each line in the comparison:

       -n Skips specified number of fields.  A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
	  neighbors.

       +n Skips specified number of characters in addition to fields.  Fields are skipped before characters.

       -c Displays number of repetitions, if any, for each line.

       -d Displays only lines that were repeated.

       -u Displays only unique (nonrepeated) lines.

       If the -u flag is used, just the lines that are not repeated in the original file are output.  The -d option specifies  that  one  copy	of
       just the repeated lines is to be written.  The normal mode output is the union of the -u and -d mode outputs.

       The  -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
       times it occurred.

See Also
       comm(1), sort(1)

																	   uniq(1)
All times are GMT -4. The time now is 04:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy