02-11-2013
Delete columns if a pattern met
Hi,
I'd like to ask for some help with the following task, please:
there is a big file with a header (this is file.in):
HTML Code:
NAME A_1.X A_1.Y A_1.Z B_1.X B_1.Y B_1.Z
name1 AB 0.11 0.12 BB 0.45 0.67
name2 BB 0.34 0.56 AA 0.89 0.68
what I need is to recognize a pattern in the header of this file (pattern is in another file) and delete the column with that header
for example, the file with the pattern looks like this (this is file.with.patterns)
HTML Code:
A_1
A_2
C_4
D_7
so, it would recognize A_1 and will delete all the columns containing A_1; thus, the output would look like this (this is file.out):
HTML Code:
NAME B_1.X B_1.Y B_1.Z
name1 BB 0.45 0.67
name2 AA 0.89 0.68
I am not sure I've got the best approach. What I was thinking to do is to put all the columns whose header does not contain the specified pattern in one output file (so, those columns whose header does match the pattern will be let out, deleted):
HTML Code:
while read i
do
awk 'NR==1{for(a=1,a<=NF;a++) if ($a!~/$i/)f[n++]=a}
{for(a=0;a<=n;i++)printf"%s%s",a?":"",$f[a];print''} file.in >> file.out
done < file.with.patterns
one problem is that I would like to have all the columns whose header does not match the patterns in the file.with.patterns to be in the file.out and I am not sure if append sign (>>) would do that... it didn't really work well so far...
Another option I was thinking about is to establish the number of the columns whose header contains the pattern and then delete them with cut -f, but don't know how to do that.
Any ideas will be greatly appreciated!
Many thanks for your time!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I've got a data file like the following format.
196004010000 196004020000 8192 24 ueaag 98.793 18.750 20 ---- - 36 23 9999 314.161773681641
196004020000 196004030000 8192 24 ueaag 98.793 18.750 20 ---- - 36 23 9999 314.71533203125
196004030000 196004040000 8192 24... (7 Replies)
Discussion started by: su_in99
7 Replies
2. Shell Programming and Scripting
Hi,
I have file 1.txt with following entries as shown:
0152364|134444|10.20.30.40|015236433
0233654|122555|10.20.30.50|023365433
**
**
**
In file 2.txt I have the following entries as shown:
0152364|134444|10.20.30.40|015236433
0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies
3. UNIX for Dummies Questions & Answers
Hii I have a file which contains huge amounts of data.I just want to delete last 3 columns in the without changing its format.The file contains data as shown below
PDE 2001 10 29 202148.60 38.92 24.20 33 4.8 MLATH .F. .......
PDE 2001 10 29 203423.57 38.88 24.41 33 3.7 MLATH... (3 Replies)
Discussion started by: reva
3 Replies
4. UNIX for Dummies Questions & Answers
I have this space delimited large text file with more than 1,000,000+ columns and about 100 rows. I want to delete all the columns that start with NA such that:
File before modification
aa bb cc NA100 dd
aa b1 c2 NA101 de
File after modification
aa bb cc dd
aa b1 c2 de
How would I... (3 Replies)
Discussion started by: evelibertine
3 Replies
5. Shell Programming and Scripting
Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows.
header for file
1111
2222
3333
header for file
1111
2222
3333
header for file... (8 Replies)
Discussion started by: gary_w
8 Replies
6. Shell Programming and Scripting
Hi All
I am having pipe seperated inputs like
Adam|PeteR|Josh|PEter
Nick|Rave|Simon|Paul
Steve|smith|PETER|Josh
Andrew|Daniel|StAlin|peter
Rick|PETer|ADam|RAVE
i want to repleace all the occurrence of peter (in any case pattern PeteR,PEter,PETER,peter,PETer) with Peter so that output... (5 Replies)
Discussion started by: sudeep.id
5 Replies
7. UNIX for Dummies Questions & Answers
An extension from an earlier question. Now need a sed script to delete columns 7,15 and 16 from an example txt below..
Again, thanks in advance.
98M-01.WAV,98M,01,00:00:49,01:07:36:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"",
98L-01.WAV,98L,01,00:00:51,01:01:45:00,"MIX",,"BOOM-MKH50",,,,,,,,,,"", (7 Replies)
Discussion started by: Vrc2250
7 Replies
8. Shell Programming and Scripting
i have a table like this:
id, senderNumber, blacklist
-----------------------------
1 0835636326 Y
2 0373562343 Y
3 0273646833 Y
and I want to delete automatically if a new inserted row on another table consist anything on senderNumber column above using a BASH Script
I... (9 Replies)
Discussion started by: jazzyzha
9 Replies
9. Shell Programming and Scripting
I had a spot of trouble coming up with a title, hopefully you'll understand once you read my problem... :)
I have the output of an ldapsearch that looks like this:
dn: cn=sam,ou=company,o=com
uidNumber: 7174
gidNumber: 49563
homeDirectory: /home/sam
loginshell: /bin/bash
uid: sam... (2 Replies)
Discussion started by: samgoober
2 Replies
10. Shell Programming and Scripting
I have a file
Line 1 a
Line 22
Line 33
Line 1 b
Line 22
Line 1 c
Line 4
Line 5
I want to delete all lines before last occurrence of a line which contains something which is defined in a variable. Say a variable var contains 'Line 1', then I need the following in the output.
... (21 Replies)
Discussion started by: Soham
21 Replies
LEARN ABOUT FREEBSD
ascii
ASCII(7) BSD Miscellaneous Information Manual ASCII(7)
NAME
ascii -- octal, hexadecimal and decimal ASCII character sets
DESCRIPTION
The octal set:
000 NUL 001 SOH 002 STX 003 ETX 004 EOT 005 ENQ 006 ACK 007 BEL
010 BS 011 HT 012 NL 013 VT 014 NP 015 CR 016 SO 017 SI
020 DLE 021 DC1 022 DC2 023 DC3 024 DC4 025 NAK 026 SYN 027 ETB
030 CAN 031 EM 032 SUB 033 ESC 034 FS 035 GS 036 RS 037 US
040 SP 041 ! 042 " 043 # 044 $ 045 % 046 & 047 '
050 ( 051 ) 052 * 053 + 054 , 055 - 056 . 057 /
060 0 061 1 062 2 063 3 064 4 065 5 066 6 067 7
070 8 071 9 072 : 073 ; 074 < 075 = 076 > 077 ?
100 @ 101 A 102 B 103 C 104 D 105 E 106 F 107 G
110 H 111 I 112 J 113 K 114 L 115 M 116 N 117 O
120 P 121 Q 122 R 123 S 124 T 125 U 126 V 127 W
130 X 131 Y 132 Z 133 [ 134 135 ] 136 ^ 137 _
140 ` 141 a 142 b 143 c 144 d 145 e 146 f 147 g
150 h 151 i 152 j 153 k 154 l 155 m 156 n 157 o
160 p 161 q 162 r 163 s 164 t 165 u 166 v 167 w
170 x 171 y 172 z 173 { 174 | 175 } 176 ~ 177 DEL
The hexadecimal set:
00 NUL 01 SOH 02 STX 03 ETX 04 EOT 05 ENQ 06 ACK 07 BEL
08 BS 09 HT 0A NL 0B VT 0C NP 0D CR 0E SO 0F SI
10 DLE 11 DC1 12 DC2 13 DC3 14 DC4 15 NAK 16 SYN 17 ETB
18 CAN 19 EM 1A SUB 1B ESC 1C FS 1D GS 1E RS 1F US
20 SP 21 ! 22 " 23 # 24 $ 25 % 26 & 27 '
28 ( 29 ) 2a * 2b + 2c , 2d - 2e . 2f /
30 0 31 1 32 2 33 3 34 4 35 5 36 6 37 7
38 8 39 9 3a : 3b ; 3c < 3d = 3e > 3f ?
40 @ 41 A 42 B 43 C 44 D 45 E 46 F 47 G
48 H 49 I 4a J 4b K 4c L 4d M 4e N 4f O
50 P 51 Q 52 R 53 S 54 T 55 U 56 V 57 W
58 X 59 Y 5a Z 5b [ 5c 5d ] 5e ^ 5f _
60 ` 61 a 62 b 63 c 64 d 65 e 66 f 67 g
68 h 69 i 6a j 6b k 6c l 6d m 6e n 6f o
70 p 71 q 72 r 73 s 74 t 75 u 76 v 77 w
78 x 79 y 7a z 7b { 7c | 7d } 7e ~ 7f DEL
The decimal set:
0 NUL 1 SOH 2 STX 3 ETX 4 EOT 5 ENQ 6 ACK 7 BEL
8 BS 9 HT 10 NL 11 VT 12 NP 13 CR 14 SO 15 SI
16 DLE 17 DC1 18 DC2 19 DC3 20 DC4 21 NAK 22 SYN 23 ETB
24 CAN 25 EM 26 SUB 27 ESC 28 FS 29 GS 30 RS 31 US
32 SP 33 ! 34 " 35 # 36 $ 37 % 38 & 39 '
40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 /
48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7
56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ?
64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G
72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O
80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W
88 X 89 Y 90 Z 91 [ 92 93 ] 94 ^ 95 _
96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g
104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o
112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w
120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 DEL
FILES
/usr/share/misc/ascii
HISTORY
An ascii manual page appeared in Version 7 AT&T UNIX.
BSD
June 5, 1993 BSD