Sponsored Content
Top Forums Shell Programming and Scripting Delete duplicate strings in a line Post 302879525 by Akshay Hegde on Thursday 12th of December 2013 09:55:57 PM
Old 12-12-2013
substr($1,2) ---> if your input is &aff2g0440 after using substr($1,2) you will get aff2g0440 second char onwards from column 1 and it searches for exact match in column2, if condition is true,

$2 = x ---> since x is not set, its NULL so field will be masked here(or empty field2)

$0 = $0 ---> recalculate field

$1=$1 ---> recalculate record, and remove space

finally

}1 --> 1 is true 0 is false since its one so prints all the line
This User Gave Thanks to Akshay Hegde For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Delete lines with duplicate strings based on date

Hey all, a relative bash/script newbie trying solve a problem. I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like 2007-11-03... (3 Replies)
Discussion started by: mattv
3 Replies

2. Shell Programming and Scripting

delete repeated strings (tags) in a line and concatenate corresponding words

Hello friends! Each line of my input file has this format: word<TAB>tag1<blankspace>lemma<TAB>tag2<blankspace>lemma ... <TAB>tag3<blankspace>lemma Of this file I need to eliminate all the repeated tags (of the same word) in a line, as in the example here below, but conserving both (all) the... (2 Replies)
Discussion started by: mjomba
2 Replies

3. UNIX for Dummies Questions & Answers

Delete strings in file1 based on the list of strings in file2

Hello guys, should be a very easy questn for you: I need to delete strings in file1 based on the list of strings in file2. like file2: word1_word2_ word3_word5_ word3_word4_ word6_word7_ file1: word1_word2_otherwords..,word3_word5_others... (7 Replies)
Discussion started by: roussine
7 Replies

4. Shell Programming and Scripting

How to delete a duplicate line and original with sed.

I am completely new to shell scripting but have been assigned the task of creating several batch files to manipulate data. My final task requires me to find lines that have duplicates present then delete not only the duplicate but the original as well. The script will be used in a windows... (9 Replies)
Discussion started by: chino_1
9 Replies

5. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ... (4 Replies)
Discussion started by: umapearl
4 Replies

6. Shell Programming and Scripting

Delete lines in file containing duplicate strings, keeping longer strings

The question is not as simple as the title... I have a file, it looks like this <string name="string1">RZ-LED</string> <string name="string2">2.0</string> <string name="string2">Version 2.0</string> <string name="string3">BP</string> I would like to check for duplicate entries of... (11 Replies)
Discussion started by: raidzero
11 Replies

7. Shell Programming and Scripting

Delete Duplicate line (not really) from the file

I need help in figuring out hoe to delete lines in a data file. The data file is huge. I am currently using "vi" to search and delete the lines - which is cumbersome since it takes lots of time to save that file (due to its huge size). Here is the issue. I have a data file with the following... (4 Replies)
Discussion started by: GosarJunk
4 Replies

8. Shell Programming and Scripting

Delete 2 strings from 1 line with sed?

Hi guys, I wonder if it's possible to search for a line containing 2 strings and delete that line and perhaps replace the source file with already deleted line(s). What I mean is something like this: sourcefile.txt line1: something 122344 somethin2 24334 45554676 line2: another something... (6 Replies)
Discussion started by: netrom
6 Replies

9. UNIX for Dummies Questions & Answers

Log file - Delete duplicate line & keep last date

Hello All ! I need your help on this case, I have a csv file with this: ITEM105;ARI FSR;2016-02-01 08:02;243 ITEM101;ARI FSR;2016-02-01 06:02;240 ITEM032;RNO TLE;2016-02-01 11:03;320 ITEM032;RNO TLE;2016-02-02 05:43;320 ITEM032;RNO TLE;2016-02-01 02:03;320 ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies

10. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies
AUSEARCH-EXPRESSION(5)						    Linux Audit 					    AUSEARCH-EXPRESSION(5)

NAME
ausearch-expression - audit search expression format OVERVIEW
This man page describes the format of "ausearch expressions". Parsing and evaluation of these expressions is provided by libauparse and is common to applications that use this library. LEXICAL STRUCTURE
White space (ASCII space, tab and new-line characters) between tokens is ignored. The following tokens are recognized: Punctuation ( ) Logical operators ! && || Comparison operators < <= == > >= !== i= i!= r= r!= Unquoted strings Any non-empty sequence of ASCII letters, digits, and the _ symbol. Quoted strings A sequence of characters surrounded by the " quotes. The character starts an escape sequence. The only defined escape sequences are \ and ". The semantics of other escape sequences is undefined. Regexps A sequence of characters surrounded by the / characters. The character starts an escape sequence. The only defined escape sequences are \ and /. The semantics of other escape sequences is undefined. Anywhere an unquoted string is valid, a quoted string is valid as well, and vice versa. In particular, field names may be specified using quoted strings, and field values may be specified using unquoted strings. EXPRESSION SYNTAX
The primary expression has one of the following forms: field comparison-operator value egexp string-or-regexp field is either a string, which specifies the first field with that name within the current audit record, or the escape character fol- lowed by a string, which specifies a virtual field with the specified name (virtual fields are defined in a later section). field is a string. operator specifies the comparison to perform r= r!= Get the "raw" string of field, and compare it to value. For fields in audit records, the "raw" string is the exact string stored in the audit record (with all escaping and unprintable character encoding left alone); applications can read the "raw" string using auparse_get_field_str(3). Each virtual field may define a "raw" string. If field is not present or does not define a "raw" string, the result of the comparison is false (regardless of the operator). i= i!= Get the "interpreted" string of field, and compare it to value. For fields in audit records, the "interpreted" string is an "user- readable" interpretation of the field value; applications can read the "interpreted" string using auparse_interpret_field(3). Each virtual field may define an "interpreted" string. If field is not present or does not define an "interpreted" string, the result of the comparison is false (regardless of the operator). < <= == > >= !== Evaluate the "value" of field, and compare it to value. A "value" may be defined for any field or virtual field, but no "value" is currently defined for any audit record field. The rules of parsing value for comparing it with the "value" of field are specific for each field. If field is not present, the result of the comparison is false (regardless of the operator). If field does not define a "value", an error is reported when parsing the expression. In the special case of egexp regexp-or-string, the current audit record is taken as a string (without interpreting field values), and matched against regexp-or-string. regexp-or-string is an extended regular expression, using a string or regexp token (in other words, delimited by " or /). If E1 and E2 are valid expressions, then ! E1, E1 && E2, and E1 || E2 are valid expressions as well, with the usual C semantics and evalu- ation priorities. Note that ! field op value is interpreted as !(field op value), not as (!field) op value. VIRTUAL FIELDS
The following virtual fields are defined: imestamp The value is the timestamp of the current event. value must have the ts:seconds.milli format, where seconds and milli are decimal numbers specifying the seconds and milliseconds part of the timestamp, respectively. ecord_type The value is the type of the current record. value is either the record type name, or a decimal number specifying the type. SEMANTICS
The expression as a whole applies to a single record. The expression is true for a specified event if it is true for any record associated with the event. EXAMPLES
As a demonstration of the semantics of handling missing fields, the following expression is true if field is present: (field r= "") || (field r!= "") and the same expression surrounded by !( and ) is true if field is not present. FUTURE DIRECTIONS
New escape sequences for quoted strings may be defined. For currently defined virtual fields that do not define a "raw" or "interpreted" string, the definition may be added. Therefore, don't rely on the fact that comparing the "raw" or "interpreted" string of the field with any value is false. New formats of value constants for the imestamp virtual field may be added. AUTHOR
Miloslav Trmac Red Hat Feb 2008 AUSEARCH-EXPRESSION(5)
All times are GMT -4. The time now is 09:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy