Sponsored Content
Top Forums Shell Programming and Scripting Delete columns if a pattern met Post 302769272 by zajtat on Monday 11th of February 2013 06:40:56 PM
Old 02-11-2013
Delete columns if a pattern met

Hi,

I'd like to ask for some help with the following task, please:

there is a big file with a header (this is file.in):

HTML Code:
NAME A_1.X A_1.Y A_1.Z B_1.X B_1.Y B_1.Z
name1 AB 0.11 0.12 BB 0.45 0.67 
name2 BB 0.34 0.56 AA 0.89 0.68
what I need is to recognize a pattern in the header of this file (pattern is in another file) and delete the column with that header

for example, the file with the pattern looks like this (this is file.with.patterns)
HTML Code:
A_1
A_2
C_4
D_7
so, it would recognize A_1 and will delete all the columns containing A_1; thus, the output would look like this (this is file.out):

HTML Code:
NAME B_1.X B_1.Y B_1.Z
name1 BB 0.45 0.67 
name2 AA 0.89 0.68
I am not sure I've got the best approach. What I was thinking to do is to put all the columns whose header does not contain the specified pattern in one output file (so, those columns whose header does match the pattern will be let out, deleted):

HTML Code:
while read i
do
awk 'NR==1{for(a=1,a<=NF;a++) if ($a!~/$i/)f[n++]=a}
{for(a=0;a<=n;i++)printf"%s%s",a?":"",$f[a];print''} file.in >> file.out
done < file.with.patterns
one problem is that I would like to have all the columns whose header does not match the patterns in the file.with.patterns to be in the file.out and I am not sure if append sign (>>) would do that... it didn't really work well so far...


Another option I was thinking about is to establish the number of the columns whose header contains the pattern and then delete them with cut -f, but don't know how to do that.

Any ideas will be greatly appreciated!

Many thanks for your time!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

delete some columns

I've got a data file like the following format. 196004010000 196004020000 8192 24 ueaag 98.793 18.750 20 ---- - 36 23 9999 314.161773681641 196004020000 196004030000 8192 24 ueaag 98.793 18.750 20 ---- - 36 23 9999 314.71533203125 196004030000 196004040000 8192 24... (7 Replies)
Discussion started by: su_in99
7 Replies

2. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies

3. UNIX for Dummies Questions & Answers

How to delete last 3 columns in a file

Hii I have a file which contains huge amounts of data.I just want to delete last 3 columns in the without changing its format.The file contains data as shown below PDE 2001 10 29 202148.60 38.92 24.20 33 4.8 MLATH .F. ....... PDE 2001 10 29 203423.57 38.88 24.41 33 3.7 MLATH... (3 Replies)
Discussion started by: reva
3 Replies

4. UNIX for Dummies Questions & Answers

How to delete all columns that start with a specific value

I have this space delimited large text file with more than 1,000,000+ columns and about 100 rows. I want to delete all the columns that start with NA such that: File before modification aa bb cc NA100 dd aa b1 c2 NA101 de File after modification aa bb cc dd aa b1 c2 de How would I... (3 Replies)
Discussion started by: evelibertine
3 Replies

5. Shell Programming and Scripting

sed pattern to delete lines containing a pattern, except the first occurance

Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows. header for file 1111 2222 3333 header for file 1111 2222 3333 header for file... (8 Replies)
Discussion started by: gary_w
8 Replies

6. Shell Programming and Scripting

Replacing a pattern in different cases in different columns with a single pattern

Hi All I am having pipe seperated inputs like Adam|PeteR|Josh|PEter Nick|Rave|Simon|Paul Steve|smith|PETER|Josh Andrew|Daniel|StAlin|peter Rick|PETer|ADam|RAVE i want to repleace all the occurrence of peter (in any case pattern PeteR,PEter,PETER,peter,PETer) with Peter so that output... (5 Replies)
Discussion started by: sudeep.id
5 Replies

7. UNIX for Dummies Questions & Answers

Sed: delete columns 7,15,16

An extension from an earlier question. Now need a sed script to delete columns 7,15 and 16 from an example txt below.. Again, thanks in advance. 98M-01.WAV,98M,01,00:00:49,01:07:36:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", 98L-01.WAV,98L,01,00:00:51,01:01:45:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", (7 Replies)
Discussion started by: Vrc2250
7 Replies

8. Shell Programming and Scripting

Delete if condition met in a column

i have a table like this: id, senderNumber, blacklist ----------------------------- 1 0835636326 Y 2 0373562343 Y 3 0273646833 Y and I want to delete automatically if a new inserted row on another table consist anything on senderNumber column above using a BASH Script I... (9 Replies)
Discussion started by: jazzyzha
9 Replies

9. Shell Programming and Scripting

If first pattern is found, look for second pattern. If second pattern not found, delete line

I had a spot of trouble coming up with a title, hopefully you'll understand once you read my problem... :) I have the output of an ldapsearch that looks like this: dn: cn=sam,ou=company,o=com uidNumber: 7174 gidNumber: 49563 homeDirectory: /home/sam loginshell: /bin/bash uid: sam... (2 Replies)
Discussion started by: samgoober
2 Replies

10. Shell Programming and Scripting

How to delete all lines before a particular pattern when the pattern is defined in a variable?

I have a file Line 1 a Line 22 Line 33 Line 1 b Line 22 Line 1 c Line 4 Line 5 I want to delete all lines before last occurrence of a line which contains something which is defined in a variable. Say a variable var contains 'Line 1', then I need the following in the output. ... (21 Replies)
Discussion started by: Soham
21 Replies
MOKA(1)                                                      CAO-VLSI Reference Manual                                                     MOKA(1)

NAME
MOKA - Model checker ancestor ORIGIN
This software belongs to the ALLIANCE CAD SYSTEM developed by the ASIM team at LIP6 laboratory of Universite Pierre et Marie CURIE, in Paris, France. Web : http://asim.lip6.fr/recherche/alliance/ E-mail : alliance-users@asim.lip6.fr SYNOPSIS
moka [-VDB] fsm_filename ctl_filename DESCRIPTION
moka is a CTL model checker. Made to run on FSM or RTL descriptions, moka supports the same VHDL subset as syf or boom (for further informations about this subset see SYF(1), BOOM(1), FSM(5), VBE(5) ). Nevertheless moka imposes that each register of the behavioral description have the same clock condi- tion and that there are no tristate or multiplexed buses. In particular VHDL type MUX_BIT and WOR_BIT aren't not supported. First of all moka build the fonction transition of the FSM using a Reduced Ordered Binary Decision Diagrams representation. It then applies the initial conditions to find the first state (keyword INITIAL and/or RESET_COND in the CTL(5) file format). After it computes a symbolic simulation of the FSM in order to find all reachable states. This computation takes into account the assump- tions conditions (ASSUME keyword in the CTL(5) file format). moka finally verifies one by one each CTL formulae. (see CTL(5) for CTL file format details). CTL OPERATORS
For each CTL sub-expression moka will return the set of states that verifies the formula. For example EX(p) will return the set of reach- able states that verifies EX(p). CTL operators : EX(p) : returns all states which have almost one primary state successor that verifies p. EU(p,q) : returns all states that are the root of almost one path, such that p is true until q is always true. EG(p) : returns all states that are the root of almost one path, such that p is always true. AX(p) : returns all states which have all their primary state successor that verifies p. AU(p,q) : returns all states that are the root of only pathes from which p is true until q is always true. AG(p) : returns all states that are the root of only pathes, such that p is always true. ENVIRONMENT VARIABLES
MBK_WORK_LIB gives the path for the description and the CTL file. The default value is the current directory. MBK_CATA_LIB gives some auxiliary pathes for the descriptions and the CTL file. The default value is the current directory. OPTIONS
-V Sets verbose mode on. Each step of the model checking is displayed on the standard output. -D Sets debug mode on. Each step of the model checking is detailed on the standard output. In particular all states set are displayed for each CTL sub-expression. -B The input file is a VHDL description using the Alliance VHDL subset (see VBE(5) file format). FSM EXAMPLE
-- A multi fsm example ENTITY example is PORT ( ck : in BIT; data_in : in BIT; reset : in BIT; data_out : out BIT ); END example; ARCHITECTURE FSM OF example is TYPE A_ETAT_TYPE IS (A_E0, A_E1); SIGNAL A_NS, A_CS : A_ETAT_TYPE; TYPE B_ETAT_TYPE IS (B_E0, B_E1); SIGNAL B_NS, B_CS : B_ETAT_TYPE; --PRAGMA CURRENT_STATE A_CS FSM_A --PRAGMA NEXT_STATE A_NS FSM_A --PRAGMA CLOCK ck FSM_A --PRAGMA FIRST_STATE A_E0 FSM_A --PRAGMA CURRENT_STATE B_CS FSM_B --PRAGMA NEXT_STATE B_NS FSM_B --PRAGMA CLOCK ck FSM_B --PRAGMA FIRST_STATE B_E0 FSM_B SIGNAL ACK, REQ, DATA_INT : BIT; BEGIN A_1 : PROCESS ( A_CS, ACK ) BEGIN IF ( reset = '1' ) THEN A_NS <= A_E0; DATA_OUT <= '0'; REQ <= '0'; ELSE CASE A_CS is WHEN A_E0 => IF ( ACK ='1') THEN A_NS <= A_E1; ELSE A_NS <= A_E0; END IF; DATA_OUT <= '0'; REQ <= '1'; WHEN A_E1 => IF ( ACK ='1') THEN A_NS <= A_E1; ELSE A_NS <= A_E0; END IF; DATA_OUT <= DATA_INT; REQ <= '0'; END CASE; END IF; END PROCESS A_1; A_2 : PROCESS( ck ) BEGIN IF ( ck = '1' AND NOT ck'STABLE ) THEN A_CS <= A_NS; END IF; END PROCESS A_2; ------- B_1 : PROCESS ( B_CS, ACK ) BEGIN IF ( reset = '1' ) THEN B_NS <= B_E0; DATA_INT <= '0'; ACK <= '0'; ELSE CASE B_CS is WHEN B_E0 => IF ( REQ ='1') THEN B_NS <= B_E1; ELSE B_NS <= B_E0; END IF; DATA_INT <= '0'; ACK <= '0'; WHEN B_E1 => IF ( REQ ='1') THEN B_NS <= B_E1; ELSE B_NS <= B_E0; END IF; DATA_INT <= DATA_IN; ACK <= '1'; END CASE; END IF; END PROCESS B_1; B_2 : PROCESS( ck ) BEGIN IF ( ck = '1' AND NOT ck'STABLE ) THEN B_CS <= B_NS; END IF; END PROCESS B_2; END FSM; CTL EXAMPLE
-- A CTL file example TYPE A_ETAT_TYPE IS (A_E0, A_E1); TYPE B_ETAT_TYPE IS (B_E0, B_E1); VARIABLE A_NS, A_CS : A_ETAT_TYPE; VARIABLE B_NS, B_CS : B_ETAT_TYPE; VARIABLE ck : BIT; VARIABLE data_in : BIT; VARIABLE data_out : BIT; VARIABLE reset : BIT; VARIABLE ack : BIT; VARIABLE req : BIT; RESET_COND init1 := (reset='1'); ASSUME ass1 := (reset='0'); begin prop1 : EX( ack='1' ); prop2 : AG( req -> AF( ack ) ); prop4 : AU( req='1', ack='1'); end; MOKA EXAMPLE
moka -V example example SEE ALSO
syf (1), fsp (1), fsm (5), ctl (5), vbe(5). BUG REPORT
This tool is under development at the ASIM department of the LIP6 laboratory. We need your feedback to improve documentation and tools. ASIM
/LIP6 August 5, 2002 MOKA(1)
All times are GMT -4. The time now is 05:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy