Sponsored Content
Top Forums Shell Programming and Scripting Filter or remove duplicate block of text without distinguishing marks or fields Post 302563738 by samask on Wednesday 12th of October 2011 05:04:14 AM
Old 10-12-2011
@radoulov,

That is an *awesome* explanation. Smilie Smilie Smilie Smilie

I am so grateful for the code and the lesson.

Thank you!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate text

Hello, I have a log file which is generated by a script which looks like this: userid: 7 starttime: Sat May 24 23:24:13 CEST 2008 endtime: Sat May 24 23:26:57 CEST 2008 total time spent: 2.73072 minutes / 163.843 seconds date: Sat Jun 7 16:09:03 CEST 2008 userid: 8 starttime: Sun May... (7 Replies)
Discussion started by: dejavu88
7 Replies

2. Shell Programming and Scripting

Filter duplicate block of text using SED

Hi, I would like to print a block of text between 2 regular expression using Sed, This can be achieved by using the command as shown below, however my problem is the same block of text is repeated twice. I would like to eliminate the duplicate block of text. For Example If my file... (5 Replies)
Discussion started by: dkumar91
5 Replies

3. Shell Programming and Scripting

Remove duplicate files based on text string?

Hi I have been struggling with a script for removing duplicate messages from a shared mailbox. I would like to search for duplicate messages based on the “Message-ID” string within the messages files. I have managed to find the duplicate “Message-ID” strings and (if I would like) delete... (1 Reply)
Discussion started by: spangberg
1 Replies

4. Shell Programming and Scripting

Filter/remove duplicate .dat file with certain criteria

I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records. contents of file looks like 30002157,40342424,OTC,mart_rec,100, ,0 30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Discussion started by: mukeshguliao
6 Replies

5. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

6. Windows & DOS: Issues & Discussions

Remove duplicate lines from text files.

So, I have text files, one "fail.txt" And one "color.txt" I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file. Afterwards there shall be no duplicate lines. (1 Reply)
Discussion started by: pasc
1 Replies

7. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

8. Shell Programming and Scripting

How to remove duplicate text blocks from a file?

Hi All I have a list of files which will have duplicate list of blocks of text. Following is a sample of the file, I have removed the sensitive information from the file. All the code samples starts from <TR BGCOLOR="white"> and Ends with IP address and two html tags like this. 10.14.22.22... (3 Replies)
Discussion started by: mahasona
3 Replies

9. Shell Programming and Scripting

Remove duplicate occurrences of text pattern

Hi folks! I have a file which contains a 1000 lines. On each line i have multiple occurrences ( 26 to be exact ) of pattern folder#/folder#. # is depicting the line number in the file some text here folder1/folder1 some text here folder1/folder1 some text here folder1/folder1 some text... (7 Replies)
Discussion started by: martinsmith
7 Replies

10. Shell Programming and Scripting

Filter file to remove duplicate values in first column

Hello, I have a script that is generating a tab delimited output file. num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 -1.6120 3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
flaw(7) 						 Miscellaneous Information Manual						   flaw(7)

NAME
flaw - a Lua OO management framework for awesome WM widgets DESCRIPTION
flaw stands for Fully Loaded AWesome. It is a LUA object oriented library providing a thin abstraction layer above awesome widgets. It is aimed at being simple and resources efficient. To achieve these goals, flaw provides the following concepts. gadgets To add functionality to awesome widgets, flaw defines gadget objects, which are a wrapper around a widget. All gadgets have proper- ties, events, a refresh mechanism and a data provider (see below). Gadgets can wrap all awesome widget type, namely text boxes, image boxes, graphs or progress bars. flaw provides many gadgets for common system information (like CPU or memory activity). provider flaw tries to minimise system access and data refresh. Since all information do not have the same expiration rate, all gadgets refresh independently. And since some gadgets can share information, all data is provided by providers which can be shared among gadgets. Providers maintain status data from the system and refreshes only when necessary (ie. when the gadget with the shortest refresh rate demands it). events Events are a way for the user to modify the gadget behaviour or properties when some conditions are met. An event is composed of a trigger, which computes the condition, and an action. Event triggers are tested by the providers only when data changes. Both the condition and the action are provided by the user. USAGE
flaw depends on the lua-filesystem package which can be found http://luaforge.net/projects/luafilesystem. Once installed, the following statement must be inserted in awesome configuration before any flaw gadget or mechanism is used. require('flaw') Using flaw consists in creating gadgets, passing them parameters to customize their display and events to add some behaviour to them, and then adding them in a wibox like any other awesome widget. flaw and awful gadgets can live together. The complete flaw documentation is available in Luadoc format. Enter the following command to generate it. LUA_PATH="path_toflaw/doclet/html/?;;" luadoc -d html --nofiles *.lua DIAGNOSTICS
flaw tries to recover upon bad invocations. When something wrong occurs, it outputs warning or error messages to stderr (for example, if awesome was started with xsession, output will be found in ~/.xsession-errors ). BUGS
Certainly a lot. AUTHOR
David Soulayrol <david.soulayrol at gmail dot net> SEE ALSO
awesome(1), awesome-client(1), awesomerc(5) FEBRUARY 2009 flaw(7)
All times are GMT -4. The time now is 07:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy