Sponsored Content
Top Forums Shell Programming and Scripting Highlighting duplicate string on a line Post 302916702 by Scrutinizer on Thursday 11th of September 2014 11:40:15 AM
Old 09-11-2014
While you are one the right track, it is best not to call an external program inside a loop because that will make it very slow. You could do it all in shell inside the loop, or use a single utility instead of a shell loop..

--
Another option would be to use a back reference in a regex :
Code:
grep '^\(.*\)\1$' file

The anchors ^ and $ make sure the two identical patterns glued together form the whole line...
This User Gave Thanks to Scrutinizer For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

removing line and duplicate line

Hi, I have 3 lines in a text file that is similar to this (as a result of a diff between 2 files): 35,36d34 < DATA.EVENT.EVENT_ID.s = "3661208" < DATA.EVENT.EVENT_ID.s = "3661208" I am trying to get it down to just this: DATA.EVENT.EVENT_ID.s = "3661208" How can I do this?... (11 Replies)
Discussion started by: ocelot
11 Replies

2. Shell Programming and Scripting

How to remove duplicate sentence/string in perl?

Hi, I have two strings like this in an array: For example: @a=("Brain aging is associated with a progressive imbalance between intracellular concentration of Reactive Oxygen Species","Brain aging is associated with a progressive imbalance between intracellular concentration of Reactive... (9 Replies)
Discussion started by: vanitham
9 Replies

3. Shell Programming and Scripting

filtering out duplicate substrings, regex string from a string

My input contains a single word lines. From each line data.txt prjtestBlaBlatestBlaBla prjthisBlaBlathisBlaBla prjthatBlaBladpthatBlaBla prjgoodBlaBladpgoodBlaBla prjgood1BlaBla123dpgood1BlaBla123 Desired output --> data_out.txt prjtestBlaBla prjthisBlaBla... (8 Replies)
Discussion started by: kchinnam
8 Replies

4. Shell Programming and Scripting

Delete duplicate in certain number of string

Hi, do you have awk or sed sommand taht will delete duplicate lines like. sample: server1-log1-14 server1-log2-14 superserver-time-2 superserver-log-2 output: server-log1-14 superserver-time-2 thansk (2 Replies)
Discussion started by: kenshinhimura
2 Replies

5. Shell Programming and Scripting

find duplicate string in many different files

I have more than 100 files like this: SVEAVLTGPYGYT 2 SVEGNFEETQY 10 SVELGQGYEQY 28 SVERTGTGYT 6 SVGLADYNEQF 21 SVGQGYEQY 32 SVKTVLGYEQF 2 SVNNEQF 12 SVRDGLTNSPLH 3 SVRRDREGLEQF 11 SVRTSGSYEQY 17 SVSVSGSPLQETQY 78 SVVHSTSPEAF 59 SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies

6. Shell Programming and Scripting

Remove not only the duplicate string but also the keyword of the string in Perl

Hi Perl users, I have another problem with text processing in Perl. I have a file below: Linux Unix Linux Windows SUN MACOS SUN SUN HP-AUX I want the result below: Unix Windows SUN MACOS HP-AUX so the duplicate string will be removed and also the keyword of the string on... (2 Replies)
Discussion started by: askari
2 Replies

7. Shell Programming and Scripting

Honey, I broke awk! (duplicate line removal in 30M line 3.7GB csv file)

I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code: awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies

8. Red Hat

How to add a new string at the end of line by searching a string on the same line?

Hi, I have a file which is an extract of jil codes of all autosys jobs in our server. Sample jil code: ************************** permission:gx,wx date_conditions:yes days_of_week:all start_times:"05:00" condition: notrunning(appDev#box#ProductLoad)... (1 Reply)
Discussion started by: raghavendra
1 Replies

9. Shell Programming and Scripting

Highlighting duplicate string on a line

Hi all I have a grep written to pull out values; below (in the code snip-it) is an example of the output. What I'm struggling to do, and looking for assistance on, is identifying the lines that have duplicate strings. For example 74859915K74859915K in the below is 74859915K repeated twice but... (3 Replies)
Discussion started by: brighty
3 Replies

10. Shell Programming and Scripting

Shell script to get duplicate string

Hi All, I have a requirement where I have to get the duplicate string count and uniq error message. Below is my file: Rejected - Error on table TableA, column ColA. Error String 1. Rejected - Error on table TableA, column ColB. Error String 2. Rejected - Error on table TableA, column... (6 Replies)
Discussion started by: Deekhari
6 Replies
WMANAGER(1)						    BSD General Commands Manual 					       WMANAGER(1)

NAME
wmanager -- choose a window manager and launch it SYNOPSIS
wmanager [OPTIONS ...] DESCRIPTION
The wmanager program displays a choice of the window managers listed in the ~/.wmanagerrc file. When one is picked, the related command is written to standard output, intended to be used by shell scripts. If the program is exited without choosing a window manager, ``-1'' is written to standard output. -fg COLOR Set the foreground color. -bg COLOR Set the background color. -bg2 COLOR Set the widget background color -di[splay] host:n.n Set the X display. -dn[d], -nod[nd] Enable/disable drag & drop, probably does nothing. -g[eometry] WxH+X+Y Set the window size and location. -i[conic] Start as iconified. -k[bd], -nok[bd] Enable/disable keyboard support. -na[me] CLASSNAME Set the X window class. -s[cheme] SCHEME Unknown, probably does nothing useful. -ti[tle] WINDOWTITLE Set the window title. -to[oltips], -not[ooltips] Enable/disable tooltips, probably does nothing. EXAMPLE
To start using wmanager, create a ~/.wmanagerrc file - generally with wmanagerrc-update(1) - and add something like the following at the end of your ~/.xsession file: WM="$(wmanager -geometry +570+585)" ... exec $WM See also wmanager-loop(1) for a nicer way to start wmanager. SEE ALSO
wmanager(1), wmanager-loop(1), wmanagerrc-update(1), X(7x) HISTORY
The wmanager program was written by Meik Tessmer in 1999. This manual page was originally written in perldoc format by Tommi Virtanen in 2000, and converted to mdoc format by Peter Pentchev in 2008. AUTHORS
The wmanager program - Meik Tessmer <fuller@daemogorgon.net>. The manual page - Tommi Virtanen <tv@debian.org> and Peter Pentchev <roam@ringlet.net>. BSD
May 22, 2008 BSD
All times are GMT -4. The time now is 12:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy