12-03-2011
Find redundant text in a file
I want to find which pattern or strings have occurred more than one time so that I can remove unnecessary redundancy.
For example:
If I have the sentence:
A quick brown brown fox jumps jumps jumps over the lazy dog
in a file, then I want to know that
1. the word "brown" has occurred 2 times.
2. the word "jump" has occurred 3 times.
in the above mentioned sentence.
Note that I have no idea which words have been repeated.
So I cannot make a pattern match search.
So I just need to know what are the texts/strings are redundant in a file. Is it possible?
Thanks.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hi
i have a very simple problem
iam moving files from download to archive folder
but before such a transfer want to make sure no two file of same
are present in my download directory
how to check for redundant file names
i thought of using WC but it counts inside the file (lines and... (5 Replies)
Discussion started by: maverick
5 Replies
2. Shell Programming and Scripting
Hi all
pls help me by providing soln for my problem
I'm having a text file which contains duplicate records .
Example:
abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452
abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452
tas 3420 3562 ... (1 Reply)
Discussion started by: G.Aavudai
1 Replies
3. UNIX for Dummies Questions & Answers
I have file named shortlist , and it contains this:
2233|charles harris |g.m. |sales |12/12/52| 90000
9876|bill johnson |director |production|03/12/50|130000
5678|robert dylan |d.g.m. |marketing |04/19/43| 85000
2365|john woodcock |director |personnel... (1 Reply)
Discussion started by: Cecko
1 Replies
4. Shell Programming and Scripting
Hi
Could you please help me out by solving teh below problem ?
I have a file with as below
source1|target1|yes
source2|target2|no
source1 is file in which i have to place some code under the <head> tag in it.
What code i have to place in source1 is something like this "abcd.....<target1>... (5 Replies)
Discussion started by: Tasha_T
5 Replies
5. Shell Programming and Scripting
Hi Everybody,
I have an unknown number of files that for some reason contain the ^Z character. I would need a command that helps me identifying these files.
Here is an example of a line:
JUAN HERN^ZNDEZ
I would greatly appreciate your help.
Thanks in advance,
Sebastian (3 Replies)
Discussion started by: hhoosscchhii
3 Replies
6. Shell Programming and Scripting
Hello!
Please, help me to write such script.
I have some text file with name filename.txt
I must check if this file contains string "test-string-first", I must cut from this file string which follows string "keyword-string:" and till first white-space and save it to some variable.
For... (3 Replies)
Discussion started by: optik77
3 Replies
7. Shell Programming and Scripting
Hi
I have a text file with rows like this:
7 Herman ASI-40 Jungle (L) Blueprint (L) Weapon Herman ASI-40 Jungle (L) 215.00 57 65.21 114.41
and
9 Herman CAP-505 (L) Blueprint (L) Weapon Herman CAP-505 (L) 220.00 46.84 49.1 104.82
and
2 ClericDagger 1C blueprint Melee - Shortblade... (2 Replies)
Discussion started by: pesa
2 Replies
8. UNIX for Dummies Questions & Answers
Hello i a script:
#!/bin/sh
count=0
for iname in `cat mysong`
do
for cname in `cat mysong`
do
if
then
count=`expr $count + 1`
fi
done
echo "word: $iname - found in the text: $count times"
count=0
donethe proplem: how i... (2 Replies)
Discussion started by: levitmic
2 Replies
9. Shell Programming and Scripting
Dear All,
I have to reduce the redundancy of a file that is like this:
a b 0
a c 0
a f 1
b a 1
b a 0
b c 1
d f 0
g h 1
f d 1
Basically, this file describe a network with relative nodes and edges.
The nodes are the different letters and the edges are represented by the numbers (in... (7 Replies)
Discussion started by: giuliangiuseppe
7 Replies
10. UNIX for Beginners Questions & Answers
Dear all,
I want to find all the "," in my text file and then replace the commas to a tab. I found a script online but I don't know how to modify the script for my case. Any one can help? Thank you.
@echo off &setlocal
set "search=%1"
set "replace=%2"
set "textfile=Input.txt"
set... (2 Replies)
Discussion started by: forevertl
2 Replies
LEARN ABOUT OPENSOLARIS
linsert
linsert(1T) Tcl Built-In Commands linsert(1T)
__________________________________________________________________________________________________________________________________________________
NAME
linsert - Insert elements into a list
SYNOPSIS
linsert list index element ?element element ...?
_________________________________________________________________
DESCRIPTION
This command produces a new list from list by inserting all of the element arguments just before the index'th element of list. Each ele-
ment argument will become a separate element of the new list. If index is less than or equal to zero, then the new elements are inserted
at the beginning of the list. If index has the value end, or if it is greater than or equal to the number of elements in the list, then
the new elements are appended to the list. end-integer refers to the last element in the list minus the specified integer offset.
EXAMPLE
Putting some values into a list, first indexing from the start and then indexing from the end, and then chaining them together:
set oldList {the fox jumps over the dog}
set midList [linsert $oldList 1 quick]
set newList [linsert $midList end-1 lazy]
# The old lists still exist though...
set newerList [linsert [linsert $oldList end-1 quick] 1 lazy]
SEE ALSO
list(1T), lappend(1T), lindex(1T), llength(1T), lsearch(1T), lset(1T), lsort(1T), lrange(1T), lreplace(1T) |
KEYWORDS
element, insert, list
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+--------------------+-----------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+--------------------+-----------------+
|Availability | SUNWTcl |
+--------------------+-----------------+
|Interface Stability | Uncommitted |
+--------------------+-----------------+
NOTES
Source for Tcl is available on http://opensolaris.org.
Tcl 8.2 linsert(1T)