Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Use sed on column (csv) file if data in colmns is greater > than? Post 302717095 by bakunin on Wednesday 17th of October 2012 02:14:04 PM
Old 10-17-2012
Quote:
Originally Posted by Chris Eagleson
Code:
sed '/^[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,/d' infile
         1     2     3     4     5     6     7    8      9     10    11    12    13   14

Almost. ;-))

You can "multiply" regexp parts with the "\{...\}" operator. For instance: "x" means a single "x", but "x\{3\}" means "three x" and would be equivalent to "xxx". But it comes even better, the operator can be given two values, which will define a range: "x\{1,3\}" means "one, two or three x". Instead of "x" one could even put a complicated regexp in braces.

Now your problem: a single field in a comma-separated file would be matched by: "[^,]*,". You have already figured that out. Now we try to match something in the 11th field. This is: ten fields, followed by the field on question:

Code:
/^\([^,]*,\)\{10\}xxx/

This matches "xxx" in the eleventh field. You see you could shorten and simplify your regexp considerably using this device. Your search for "more than 999.9 in the 10th, 11th or 12th field" could look like this:

Code:
/^\([^,]*,\)\{9,11\}[0-9][0-9][0-9][0-9]/

9-11 fields, followed by a number of at least 4 digits.

I hope this helps.

bakunin
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK/SED print if 2nd character in a column is greater than 0

We have an access log where column 8 displays the time in seconds like below: Tj8nQAoNgwsAABov9cIAAAFL - 10.13.131.80 - - (0) - "GET /aaaaa/bbbb/bbbb where column 8 is printed (0). We are trying to find how many entries are there that has column 8 greater than 0. Remember $8 is (0) and not... (5 Replies)
Discussion started by: spacemtn5
5 Replies

2. Shell Programming and Scripting

compare files and remove a line from a file if first column is greater than 25

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (4 Replies)
Discussion started by: rajniman
4 Replies

3. UNIX for Dummies Questions & Answers

using sed delete a line from csv file based on specific data in two separate fields

Hello, :wall: I have a 12 column csv file. I wish to delete the entire line if column 7 = hello and column 12 = goodbye. I have tried everything that I can find in all of my ref books. I know this does not work /^*,*,*,*,*,*,"hello",*,*,*,*,"goodbye"/d Any ideas? Thanks Please... (2 Replies)
Discussion started by: Chris Eagleson
2 Replies

4. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

5. Shell Programming and Scripting

Generating CSV from Column data

Hi List, I have a chunk of data like so: User Account Control: User Account Control: User Account Control: User Account Control: Disabled User Account Control: User Account Control: User Account Control: Disabled User Account Control: User Account Control: ... (3 Replies)
Discussion started by: landossa
3 Replies

6. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

7. Shell Programming and Scripting

How to separate data coming in one column of CSV file?

I am running an ISQL command on Sybase DB and getting output of a query in an CSV file. The issue is that all the data comes in to the same column, i want them to be separated in different columns. SQL_COMMAND=command.sql file=file.txt formatFile=formatFile.txt report=report.csv echo... (1 Reply)
Discussion started by: Sharma331
1 Replies

8. UNIX for Beginners Questions & Answers

Compare first column from two csv files with greater than or equal, and less than

I have two csv files of different sizes. The output file needs to have file1 contents on top of file2 contents where file2 col1 is >= to file1 col1, and file2 col1(same value) is < file1 col1 (next value). So basically, some file2 rows will be matched to the same file1 row because it is the closet... (7 Replies)
Discussion started by: aachave1
7 Replies

9. Shell Programming and Scripting

Adding new column data in csv from UNIX

Adding new column data in csv from UNIX Hi I need to add new column data daily to existing csv file. Please assist 7/11 7/10 7/9 7/8 space 10 GB 20 GB I was able to generate current day's data in csv but unable to add the previous 30 days data to the same csv Please use code tags,... (2 Replies)
Discussion started by: archana25
2 Replies
AUSEARCH-EXPRESSION(5)						    Linux Audit 					    AUSEARCH-EXPRESSION(5)

NAME
ausearch-expression - audit search expression format OVERVIEW
This man page describes the format of "ausearch expressions". Parsing and evaluation of these expressions is provided by libauparse and is common to applications that use this library. LEXICAL STRUCTURE
White space (ASCII space, tab and new-line characters) between tokens is ignored. The following tokens are recognized: Punctuation ( ) Logical operators ! && || Comparison operators < <= == > >= !== i= i!= r= r!= Unquoted strings Any non-empty sequence of ASCII letters, digits, and the _ symbol. Quoted strings A sequence of characters surrounded by the " quotes. The character starts an escape sequence. The only defined escape sequences are \ and ". The semantics of other escape sequences is undefined. Regexps A sequence of characters surrounded by the / characters. The character starts an escape sequence. The only defined escape sequences are \ and /. The semantics of other escape sequences is undefined. Anywhere an unquoted string is valid, a quoted string is valid as well, and vice versa. In particular, field names may be specified using quoted strings, and field values may be specified using unquoted strings. EXPRESSION SYNTAX
The primary expression has one of the following forms: field comparison-operator value egexp string-or-regexp field is either a string, which specifies the first field with that name within the current audit record, or the escape character fol- lowed by a string, which specifies a virtual field with the specified name (virtual fields are defined in a later section). field is a string. operator specifies the comparison to perform r= r!= Get the "raw" string of field, and compare it to value. For fields in audit records, the "raw" string is the exact string stored in the audit record (with all escaping and unprintable character encoding left alone); applications can read the "raw" string using auparse_get_field_str(3). Each virtual field may define a "raw" string. If field is not present or does not define a "raw" string, the result of the comparison is false (regardless of the operator). i= i!= Get the "interpreted" string of field, and compare it to value. For fields in audit records, the "interpreted" string is an "user- readable" interpretation of the field value; applications can read the "interpreted" string using auparse_interpret_field(3). Each virtual field may define an "interpreted" string. If field is not present or does not define an "interpreted" string, the result of the comparison is false (regardless of the operator). < <= == > >= !== Evaluate the "value" of field, and compare it to value. A "value" may be defined for any field or virtual field, but no "value" is currently defined for any audit record field. The rules of parsing value for comparing it with the "value" of field are specific for each field. If field is not present, the result of the comparison is false (regardless of the operator). If field does not define a "value", an error is reported when parsing the expression. In the special case of egexp regexp-or-string, the current audit record is taken as a string (without interpreting field values), and matched against regexp-or-string. regexp-or-string is an extended regular expression, using a string or regexp token (in other words, delimited by " or /). If E1 and E2 are valid expressions, then ! E1, E1 && E2, and E1 || E2 are valid expressions as well, with the usual C semantics and evalu- ation priorities. Note that ! field op value is interpreted as !(field op value), not as (!field) op value. VIRTUAL FIELDS
The following virtual fields are defined: imestamp The value is the timestamp of the current event. value must have the ts:seconds.milli format, where seconds and milli are decimal numbers specifying the seconds and milliseconds part of the timestamp, respectively. ecord_type The value is the type of the current record. value is either the record type name, or a decimal number specifying the type. SEMANTICS
The expression as a whole applies to a single record. The expression is true for a specified event if it is true for any record associated with the event. EXAMPLES
As a demonstration of the semantics of handling missing fields, the following expression is true if field is present: (field r= "") || (field r!= "") and the same expression surrounded by !( and ) is true if field is not present. FUTURE DIRECTIONS
New escape sequences for quoted strings may be defined. For currently defined virtual fields that do not define a "raw" or "interpreted" string, the definition may be added. Therefore, don't rely on the fact that comparing the "raw" or "interpreted" string of the field with any value is false. New formats of value constants for the imestamp virtual field may be added. AUTHOR
Miloslav Trmac Red Hat Feb 2008 AUSEARCH-EXPRESSION(5)
All times are GMT -4. The time now is 07:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy