Sponsored Content
Top Forums Shell Programming and Scripting how to read all the unique words in a text file Post 302148124 by encrypted on Friday 30th of November 2007 12:51:21 AM
Old 11-30-2007
Does uniq somefile help?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read words from file and create new file using K-shell.

Hi All, Please help me in creating files through K-shell scripts. I am having one file in this format. OWNER.TABLE_NAME OWNER.TABLE_NAME1 OWNER1.TABLE_NAME OWNER1.TABLE_NAME1 I want to read the above file and create new file through k shell script. The new file should looks like this.... (4 Replies)
Discussion started by: bsrajirs
4 Replies

2. Shell Programming and Scripting

To read and separate number and words in file and store to two new file using shell

hi, I am a begginer in unix and i want to know how to open a file and read it and separate the numbers & words and storing it in separate files, Using shell scripting. Please help me out for this. Regards S.Kamakshi (2 Replies)
Discussion started by: kamakshi s
2 Replies

3. Shell Programming and Scripting

extracting unique lines from text file

I have a file with 14million lines and I would like to extract all the unique lines from the file into another text file. For example: Contents of file1 happy sad smile happy funny sad I want to run a command against file one that only returns the unique lines (ie 1 line for happy... (3 Replies)
Discussion started by: soliberus
3 Replies

4. Programming

fscanf: read words from file

Hi I have a file like that: 1 2 3 4 5 6 7 8 and I want print on stdout: 1 3 8 in other words i want choose what print out. I was thinking to use fscanf as: fscanf(file_in,"%d %d %d",&a, &b,&c); but in this way i get: 1 2 3 Is there a solution using fscanf to obtain my... (2 Replies)
Discussion started by: Dedalus
2 Replies

5. Shell Programming and Scripting

extract unique pattern from large text file

Hi All, I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like 12345- , i tried with egrep ,eg : egrep "+" text.txt but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies

6. Shell Programming and Scripting

Finding the number of unique words in a file

find the number of unique words in a file using sort com- mand. (7 Replies)
Discussion started by: abhikamune
7 Replies

7. Shell Programming and Scripting

display unique words.

I am having a file with duplicate words how can I eliminate them ant,bat bat,cat cat a.txt | grep -bat | awk '{print $1}' expecting o/p as ant,bat,cat How can I display the output as ant,bat,cat in a single line and no duplicates exists. (2 Replies)
Discussion started by: shikshavarma
2 Replies

8. Shell Programming and Scripting

Unique words in each line

In each row there could be repetition of a word. I want to delete all repetitions and keep unique occurrences. Example: a+b+c ab+c ab+c abbb+c ab+bbc a+bbbc aaa aaa aaa Output: a+b+c ab+c abbb+c ab+bbc a+bbbc aaa (6 Replies)
Discussion started by: Viernes
6 Replies

9. UNIX for Beginners Questions & Answers

Shell - Read a text file with two words and extract data

hi I made this simple script to extract data and pretty much is a list and would like to extract data of two words separated by commas and I would like to make a new text file that would list these extracted data into a list and each in a new line. Example that worked for me with text file... (5 Replies)
Discussion started by: dandaryll
5 Replies

10. UNIX for Beginners Questions & Answers

Count unique words

Dear all, I would like to know how to list and count unique words in thousands number of text files. Please help me out thanks in advance (9 Replies)
Discussion started by: imranrasheedamu
9 Replies
UNIQ(1) 								FSF								   UNIQ(1)

NAME
uniq - remove duplicate lines from a sorted file SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]] DESCRIPTION
Discard all but one of successive identical lines from INPUT (or standard input), writing to OUTPUT (or standard output). Mandatory arguments to long options are mandatory for short options too. -c, --count prefix lines by the number of occurrences -d, --repeated only print duplicate lines -D, --all-repeated[=delimit-method] print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines. -f, --skip-fields=N avoid comparing the first N fields -i, --ignore-case ignore differences in case when comparing -s, --skip-chars=N avoid comparing the first N characters -u, --unique only print unique lines -w, --check-chars=N compare no more than N characters in lines --help display this help and exit --version output version information and exit A field is a run of whitespace, then non-whitespace characters. Fields are skipped before chars. AUTHOR
Written by Richard Stallman and David MacKenzie. REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>. COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the command info uniq should give you access to the complete manual. uniq (coreutils) 4.5.3 February 2003 UNIQ(1)
All times are GMT -4. The time now is 12:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy