Well good for you. We all learn better by trying, rather than being spoon-fed. With a nice pun like that, are you British?
You might need $1 in your awk rather than $0
It should still work though. This will give you the first half of each line, so you'd need to catch and compare that to the original, something like:-
Code:
while read line; do
((half=${#line}/2))
halfline=`echo $line | awk '{print substr($0,1,$half)}'`
if [ "${halfline}${halfline}" = "${line}" ]
then
echo "${line} is a duplicated entry"
else
echo "${line} is not repeated"
fi
done < $TEMP_1 > logfile
Personally, I'd replace the awk with a substitution, so you are not calling awk over and again, something like this:-
Code:
while read line; do
((half=${#line}/2))
h=1 # Set a counter
mask= # Null the variable
until [ $h -gt $half ] # Loop until counter is right
do
mask="${mask}?" # Add a ? (single character wildcard)
((h=$h+1))
done
halfline="${line#${mask}}" # Split the line
if [ "${halfline}${halfline}" = "${line}" ] # Match twice the split line with the original
then
echo "${line} is a duplicated entry"
else
echo "${line} is not repeated"
fi
done < $TEMP_1 > logfile
Hi,
I have 3 lines in a text file that is similar to this (as a result of a diff between 2 files):
35,36d34
< DATA.EVENT.EVENT_ID.s = "3661208"
< DATA.EVENT.EVENT_ID.s = "3661208"
I am trying to get it down to just this:
DATA.EVENT.EVENT_ID.s = "3661208"
How can I do this?... (11 Replies)
Hi,
I have two strings like this in an array:
For example:
@a=("Brain aging is associated with a progressive imbalance between intracellular concentration of Reactive Oxygen Species","Brain aging is associated with a progressive imbalance between intracellular concentration of Reactive... (9 Replies)
My input contains a single word lines.
From each line
data.txt
prjtestBlaBlatestBlaBla
prjthisBlaBlathisBlaBla
prjthatBlaBladpthatBlaBla
prjgoodBlaBladpgoodBlaBla
prjgood1BlaBla123dpgood1BlaBla123
Desired output -->
data_out.txt
prjtestBlaBla
prjthisBlaBla... (8 Replies)
Hi,
do you have awk or sed sommand taht will delete duplicate lines like.
sample:
server1-log1-14
server1-log2-14
superserver-time-2
superserver-log-2
output:
server-log1-14
superserver-time-2
thansk (2 Replies)
Hi Perl users,
I have another problem with text processing in Perl. I have a file below:
Linux Unix Linux Windows SUN
MACOS SUN SUN HP-AUX
I want the result below:
Unix Windows SUN
MACOS HP-AUX
so the duplicate string will be removed and also the keyword of the string on... (2 Replies)
I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code:
awk... (34 Replies)
Hi,
I have a file which is an extract of jil codes of all autosys jobs in our server.
Sample jil code:
**************************
permission:gx,wx
date_conditions:yes
days_of_week:all
start_times:"05:00"
condition: notrunning(appDev#box#ProductLoad)... (1 Reply)
Hi all
I have a grep written to pull out values; below (in the code snip-it) is an example of the output.
What I'm struggling to do, and looking for assistance on, is identifying the lines that have duplicate strings.
For example 74859915K74859915K in the below is 74859915K repeated twice but... (3 Replies)
Hi All,
I have a requirement where I have to get the duplicate string count and uniq error message. Below is my file:
Rejected - Error on table TableA, column ColA.
Error String 1.
Rejected - Error on table TableA, column ColB.
Error String 2.
Rejected - Error on table TableA, column... (6 Replies)
Discussion started by: Deekhari
6 Replies
LEARN ABOUT MINIX
grep
GREP(1) General Commands Manual GREP(1)NAME
grep - search a file for lines containing a given pattern
SYNOPSIS
grep [-elnsv] pattern [file] ...
OPTIONS -e-e pattern is the same as pattern
-c Print a count of lines matched
-i Ignore case
-l Print file names, no lines
-n Print line numbers
-s Status only, no printed output
-v Select lines that do not match
EXAMPLES
grep mouse file # Find lines in file containing mouse
grep [0-9] file # Print lines containing a digit
DESCRIPTION
Grep searches one or more files (by default, stdin) and selects out all the lines that match the pattern. All the regular expressions
accepted by ed and mined are allowed. In addition, + can be used instead of * to mean 1 or more occurrences, ? can be used to mean 0 or 1
occurrences, and | can be used between two regular expressions to mean either one of them. Parentheses can be used for grouping. If a
match is found, exit status 0 is returned. If no match is found, exit status 1 is returned. If an error is detected, exit status 2 is
returned.
SEE ALSO cgrep(1), fgrep(1), sed(1), awk(9).
GREP(1)
09-11-2014
?