Sponsored Content
Full Discussion: Help required with awk/sed
Top Forums Shell Programming and Scripting Help required with awk/sed Post 302331414 by New to awk on Monday 6th of July 2009 06:28:15 AM
Old 07-06-2009
Help required with awk/sed

Hi

I have a file, with format like:

column1|coulumn2|column3|column4
A|X|K|18
L|O|R|31,42,25
G|H|I|55,66
L|E|Q|25,31,94

output required:

column1|coulumn2|column3|column4
A|X|K|18
L|O|R|31,25
L|E|Q|25,31

Input File Format: All columns are seperated using |, last column might have multiple values spearated with ,

Output file rules:

1. Only keep rows which has one of the 18,25 or 31 (in its comma separated values) in the last column's comma separated values.
2. if last column contains any of the 18,25,31 values and some any other number, then remove numbers other than 18,25,31 as in the example output file - 1st row remains as it is, in 2nd 42 removed as this does not belong to the set (18,25,31), 3rd line remove altoghther as this does not have any of the (18,25,31) in its comma seperaetd values, 4th row remains but 94 goes away.

Please help writing one liner using awk or sed.

Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed script required

Collegues I have a flat file with data in following structure. (S1 (S (NP (NP (DT The) (JJ beautiful) (NN view)) (PP (IN of) (NP (JJ outside) (NNP greenery)))) (VP (VBZ adds) (NP (DT a) (NN tranquil) (NN touch)) (PP (TO to) (NP (DT this) (NN place)))) (. .))) I have to extract the contents in... (2 Replies)
Discussion started by: jaganadh
2 Replies

2. Shell Programming and Scripting

Help in sed required.

Hi All, I am facing a small problem in sed. I want to insert a line in the existing file. Existing code: access to attr=userPassword by self write by * auth access to * by self write by users read by anonymous auth Desired code: access to attr=userPassword by self... (14 Replies)
Discussion started by: nua7
14 Replies

3. Shell Programming and Scripting

Help required in sed or awk.

Hi All, I need to pick up data on both sides of "=" sign. For eg, following is the context that I have. 125.156.125.147=machine1 147.125.185.156=machine2 147.125.185.159=machine3 Can I have the ip address in one variable and machine name in another variable using sed or awk. ... (1 Reply)
Discussion started by: nua7
1 Replies

4. UNIX for Advanced & Expert Users

Urgent Help required : awk/sed help to find pattern and delete till end of line

Hi, I need help with using an awk or sed filter on the below line ALTER TABLE "ACCOUNT" ADD CONSTRAINT "ACCOUNT_PK" PRIMARY KEY ("ACCT_ID") USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE(INITIAL 65536 FREELISTS 1 FREELIST GROUPS 1) TABLESPACE "WMC_DATA" LOGGING ENABLE Look for... (1 Reply)
Discussion started by: rajan_san
1 Replies

5. Shell Programming and Scripting

sed help required

Hi All, I have one file with below type of data in it, $ cat test.txt ###123 ###xyxytuerwb ###2 ###tyupe Here I would like to replace all the characters with "x" after the 3 "###" with the same number of characters. Can you please help me to achieve this. (7 Replies)
Discussion started by: gr8_usk
7 Replies

6. UNIX for Dummies Questions & Answers

Clarification required on sed

Hi Can some one tell what does this sed command do sed 's/*$//g I am more curious on the highlighted part , can some one explain what does that mean. Thanks Sri (1 Reply)
Discussion started by: Sri3001
1 Replies

7. Shell Programming and Scripting

Want to sort a file using awk & sed to get required output

Hi All, Need Suggestion, Want to sort a file using awk & sed to get required, output as below, such that each LUN shows correct WWPN and FA port Numbers correctly: Required output: 01FB 10000000c97843a2 8C 0 01FB 10000000c96fb279 9C 0 22AF 10000000c97843a2 8C 0 22AF 10000000c975adbd ... (10 Replies)
Discussion started by: aix_admin_007
10 Replies

8. Shell Programming and Scripting

Tricky sed required

Hi All I need to put some sed together for a task and its a bit advanced for me, so I thought I'd ask if anyone here could help. I have a csv file with content like this - "","abcde","","" "'","abcde","","" "","","","1234" "'e'","","","" I need to remove any single quotes that fall... (17 Replies)
Discussion started by: steadyonabix
17 Replies

9. Shell Programming and Scripting

Sed/awk : to grep only required pattern disk

Hi Experts, Need help with the following: Desired output: Only want to get the output marked in green. The file: --- Physical volumes --- PV Name /dev/disk/disk4704 PV Status available Total PE 6399 Free PE ... (3 Replies)
Discussion started by: rveri
3 Replies

10. Shell Programming and Scripting

REGEX help required and some sed/awk help as well

Hi guys, I am coding a bash script that makes use of php scripts to pull URL's from a website. These url links will have numbers in them like 0.2.3 I want to make a regex that will yield me such numbers if I use a command like preg_grep. Question1: I need a regex that will tell my preg_grep... (2 Replies)
Discussion started by: mojoman
2 Replies
funcone(1)							SAORD Documentation							funcone(1)

NAME
funcone - cone search of a binary table containing RA, Dec columns SYNOPSIS
funcone <switches> <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] OPTIONS
-d deccol:[hdr] # Dec column name, units (def: DEC:d) -j # join columns from list file -J # join columns from list file, output all rows -l listfile # read centers and radii from a list -L listfile # read centers and radii from a list, output list rows -n # don't use cone limits as a filter -r racol:[hdr] # RA column name, units (def: RA:h) -x # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols -X # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols, output all rows DESCRIPTION
Funcone performs a cone search on the RA and Dec columns of a FITS binary table. The distance from the center RA, Dec position to the RA, Dec in each row in the table is calculated. Rows whose distance is less than the specified radius are output. The first argument to the program specifies the FITS file, raw event file, or raw array file. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters. The second argument is the output FITS file. If "stdout" is specified, the FITS binary table is written to the standard output. The third and fourth required arguments are the RA and Dec center position. By default, RA is specified in hours while Dec is specified in degrees. You can change the units of either of these by appending the character "d" (degrees), "h" (hours) or "r" (radians). Sexagesimal notation is supported, with colons or spaces separating hms and dms. (When using spaces, please ensure that the entire string is quoted.) The fifth required argument is the radius of the cone search. By default, the radius value is given in degrees. The units can be changed by appending the character "d" (degrees), "r" (radians), "'" (arc minutes) or '"' (arc seconds). By default, all columns of the input file are copied to the output file. Selected columns can be output using an optional sixth argument in the form: "column1 column1 ... columnN" A seventh argument allows you to output selected columns from the list file when -j switch is used. Note that the RA and Dec columns used in the cone calculation must not be de-selected. Also by default, the RA and Dec column names are named "RA" and "Dec", and are given in units of hours and degrees respectively. You can change both the name and the units using the -r [RA] and/or -d [Dec] switches. Once again, one of "h", "d", or "r" is appended to the column name to specify units but in this case, there must be a colon ":" between the name and the unit specification. If the -l [listfile] switch is used, then one or more of the center RA, center Dec, and radius can be taken from a list file (which can be a FITS table or an ASCII column text file). In this case, the third (center RA), fourth (center Dec), and fifth (radius) command line argu- ments can either be a column name in the list file (if that parameter varies) or else a numeric value (if that parameter is static). When a column name is specified for the RA, Dec, or radius, you can append a colon followed by "h", "d", or "r" to specify units (also ' and " for radius). The cone search algorithm is run once for each row in the list, taking RA, Dec, and radius values from the specified columns or from static numeric values specified on the command line. When using a list, all valid rows from each iteration are written to a single output file. Use the -x switch to help delineate which line of the list file was used to produce the given output row(s). This switch causes the values for the center RA, Dec, radius, and row number to be appended to the output file, in columns called RA_CEN, DEC_CEN, RAD_CEN and CONE_KEY, respectively. Alternatively, the -j (join) switch will append all columns from the list row to the output row (essentially a join of the list row and input row), along with the CONE_KEY row number. These two switches are mutually exclusive. The -X and -J switches write out the same data as their lower case counterparts for each row satisfying a cone search. In addition, these switches also write out rows from the event file that do not satisfy any cone search. In such cases, that CONE_KEY column will be given a value of -1 and the center and list position information will be set to zero for the given row. Thus, all rows of the input event file are guaranteed to be output, with rows satisfying at least one cone search having additional search information. The -L switch acts similarly to the -l switch in that it takes centers from a list file. However, it also implicitly sets the -j switch, so that output rows are the join of the input event row and the center position row. In addition, this switch also writes out all center position rows for which no event satisfies the cone search criteria of that row. The CONE_KEY column will be given a value of -2 for cen- ter rows that were not close to any data row and the event columns will be zeroed out for such rows. In this way, all centers rows are guaranteed to be output at least once. If any of "all row" switches (-X, -J, or -L) are specified, then a new column named JSTAT is added to the output table. The positive values in this column indicate the center position row number (starting from 1) in the list file that this data row successful matched in a cone search. A value of -1 means that the data row did not match any center position. A value of -2 means that the center position was not matched by any data row. Given a center position and radius, the cone search algorithm calculates limit parameters for a box enclosing the specified cone, and only tests rows whose positions values lie within those limits. For small files, the overhead associated with this cone limit filtering can cause the program to run more slowly than if all events were tested. You can turn off cone limit filtering using the -n switch to see if this speeds up the processing (especially useful when processing a large list of positions). For example, the default cone search uses columns "RA" and "Dec" in hours and degrees (respectively) and RA position in hours, Dec and radius in degrees: funone in.fits out.fits 23.45 34.56 0.01 To specify the RA position in degrees: funcone in.fits out.fits 23.45d 34.56 0.01 To get RA and Dec from a list but use a static value for radius (and also write identifying info for each row in the list): funcone -x -l list.txt in.fits out.fits MYRA MYDec 0.01 User specified columns in degrees, RA position in hours (sexagesimal notation), Dec position in degrees (sexagesimal notation) and radius in arc minutes: funcone -r myRa:d -d myDec in.fits out.fits 12:30:15.5 30:12 15' SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funcone(1)
All times are GMT -4. The time now is 05:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy