08-14-2010
Extracting only words from a log file
hello:
i have a file and i am trying to extract only unique words from that file.
i used the command: cat messages.1 | tr " " "\n" | sort | uniq -c
but using this command outputs everything unique in the file be it words, numbers, like all the characters..i need a command which will only words like starting with a-z, A-Z
can you please help me with this. thanx
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
i run a command that submits a word to WordNET which stores the search results in a document which looks like this... i searched "car" in this instance
and id like to extract auto, automobile, machine, and store it in a file with the , , stripped away just the words. WordNET's results' template... (2 Replies)
Discussion started by: mark_nsx
2 Replies
2. Shell Programming and Scripting
Hi all! Im trying to extract a portion of text from a KML and put it into a new file. Im trying to get all of the points out of it, ignoring everything else so I need only the text between <Placement> and </Placement>. Is there a way to make it extract all instances of these points and not just... (2 Replies)
Discussion started by: Grizzly
2 Replies
3. Shell Programming and Scripting
Hi,
I have a file few hundred MB's with text like one below in single line.
20091117 abc xyg 20091117 def ghi 20091118 ppp ttt 20091118 zzz zzz xxx
I need to extract part of line from 1st occurence of pattern 20091117
till first occurence of another pattern 20091118.
I tried... (3 Replies)
Discussion started by: artistic94555
3 Replies
4. Shell Programming and Scripting
Hi,
Pls assist.
dn: uid=test,ou=test,dc=com
description: password
sunIdentityServerDeviceStatus: Active
uid: test
objectClass: sunIdentityServerDevice
objectClass: iplanet-am-user-service
objectClass: top
objectClass: iPlanetPreferences
sunIdentityServerDeviceType: blabla
cn: default... (3 Replies)
Discussion started by: hudson03051nh
3 Replies
5. Shell Programming and Scripting
I am very new to scripting and need to write a script that will extract the account number from a line that begins with HDR. For example, the file is as follows
HDR2010072600300405505100726 00300405505
LBJ FREEWAY DALLAS
TELEGRAPH ... (9 Replies)
Discussion started by: bds052189
9 Replies
6. Shell Programming and Scripting
I am having a file from which i need to extract different length words into different file. For example 2 letter word into file2, 3 letter word into file3 and so on....
I did one using grep and shell script..
for (( i=1; i<7; i++))
do
egrep -o '\<\(?{$i}\)?\>' $1 | sort -u -f|tr >file$i... (4 Replies)
Discussion started by: akhay_ms
4 Replies
7. Shell Programming and Scripting
Hello.
From command line, the command zypper info nxclient
return a bloc of data :
linux local # zypper info nxclient
Loading repository data...
Reading installed packages...
Information for package nxclient:
Repository: zypper_local
Name: nxclient
Version: 3.5.0-7
Arch: x86_64... (7 Replies)
Discussion started by: jcdole
7 Replies
8. Shell Programming and Scripting
Hi there, Unix Gurus
Back in September last year you helped me find a way to extract the words in brackets in a textfile to a new one.
In that case my textfile was made up of sentences containing an only bracketed word per sentence/line:
1. If the boss's son had been , someone would... (9 Replies)
Discussion started by: eldeingles
9 Replies
9. Shell Programming and Scripting
Hello!
I'm trying to process a text file and am stuck at 2 extractions. Hoping someone can help me here:
1. Given a line in a text file and given a keyword, how can I extract the word preceeding the keyword using a shell command/script?
For example: Given a keyword "world" in the line: ... (2 Replies)
Discussion started by: seemad
2 Replies
10. Shell Programming and Scripting
Hello,
I want to grep a log ("server.log") for words in a separate file ("white-list.txt") and generate a separate log file containing each line that uses a word from the "white-list.txt" file.
Putting that in bullet points:
Search through "server.log" for lines that contain any word... (15 Replies)
Discussion started by: nbsparks
15 Replies
UNIQ(1) BSD General Commands Manual UNIQ(1)
NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-cdu] [-f fields] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the standard input comparing adjacent lines, and writes a copy of each unique input line to the standard output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Don't output lines that are not repeated in the input.
-f fields
Ignore the first fields in each input line when doing comparisons. A field is a string of non-blank characters separated from adja-
cent fields by blanks. Field numbers are one based, i.e. the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first fields fields will be ignored. Character numbers are one based, i.e. the first character is
character one.
-u Don't output lines that are repeated in the input.
If additional arguments are specified on the command line, the first such argument is used as the name of an input file, the second is used
as the name of an output file.
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO
sort(1)
STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible.
BSD
January 6, 2007 BSD