Sponsored Content
Top Forums Shell Programming and Scripting Delete repeated rows from a file Post 302137238 by tonet on Monday 24th of September 2007 02:16:01 AM
Old 09-24-2007
Error Delete repeated rows from a file

Hi everybody:
Could anybody tell me how I can delete repeated rows from a file?, this is, for exemple I have a file like this:

0.490 958.73 281.85 6.67985 0.002481
0.490 954.833 283.991 8.73019 0.002471
0.590 950.504 286.241 6.61451 0.002461
0.690 939.323 286.112 6.16451 0.00246
0.790 928.17 285.71 5.87057 0.002451
0.890 917.196 285.503 5.6777 0.002441
0.990 906.277 284.498 5.46275 0.00244
1.090 895.529 283.818 5.43785 0.002431
1.190 884.757 283.098 5.36579 0.002421
1.290 874.22 282.2 5.33933 0.00242
1.390 863.667 281.35 5.01376 0.002411
1.490 853.3 280.55 4.61738 0.00241
1.590 842.962 279.95 4.27487 0.002401
1.690 832.775 279.362 3.77744 0.002391
1.790 822.634 278.532 3.78002 0.00239
1.890 812.608 277.625 3.98339 0.002381
1.990 802.735 276.995 4.17061 0.00238
2.090 792.845 276.65 4.77151 0.002389
..
..

in this case I only would like this:

0.490 958.73 281.85 6.67985 0.002481
0.590 950.504 286.241 6.61451 0.002461
0.690 939.323 286.112 6.16451 0.00246
0.790 928.17 285.71 5.87057 0.002451
0.890 917.196 285.503 5.6777 0.002441
0.990 906.277 284.498 5.46275 0.00244
1.090 895.529 283.818 5.43785 0.002431
1.190 884.757 283.098 5.36579 0.002421
1.290 874.22 282.2 5.33933 0.00242
1.390 863.667 281.35 5.01376 0.002411
1.490 853.3 280.55 4.61738 0.00241
1.590 842.962 279.95 4.27487 0.002401
1.690 832.775 279.362 3.77744 0.002391
1.790 822.634 278.532 3.78002 0.00239
1.890 812.608 277.625 3.98339 0.002381
1.990 802.735 276.995 4.17061 0.00238
2.090 792.845 276.65 4.77151 0.002389
..
..

Note that the pattern that it repeat is $1 and I would like the first value that appear.

Thanks a lot and cheers . Smilie
tonet
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete particular rows from a file

Hi I have a file having 1000 rows. Now I would like to remove 10 rows from it. Plz give me the script. Eg: input file like 4 1 4500.0 1 5 1 1.0 30 6 1 1.0 4500 7 1 4.0 730 7 2 500000.0 730 8 1 785460.0 45 8 7 94255.0 30 9 1 31800.0 30 9 4 36000.0 30 10 1 15000.0 30... (5 Replies)
Discussion started by: suresh3566
5 Replies

2. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

3. UNIX for Dummies Questions & Answers

Delete repeated nos in a file

Hi, I need to delete repeated nos in a file and finally list the count. Can some one assist me? file: 12345 12345 56345 12345 23896 Output needed: 12345 56345 23896 Total count:3 Thanks (2 Replies)
Discussion started by: gini
2 Replies

4. Shell Programming and Scripting

Delete repeated word in text file

Hi expert, I am using C shell. And i trying to delete repeated word. Example file.txt: BLUE YELLOW RED VIOLET RED RED BLUE WHITE YELLOW BLACK and i wan store the output into a new file: BLUE (6 Replies)
Discussion started by: vincyoxy
6 Replies

5. Shell Programming and Scripting

[HELP] - Delete rows on a CSV file

Hello to all members, I am very new in unix stuff (shell scripting), but a want to learn a lot. I am a ex windows user but now i am absolutely Linux super user... :D So i am tryng to made a function to do this: I have two csv files only with numbers, the first one a have: 1 2 3 4 5... (6 Replies)
Discussion started by: Sadarrab
6 Replies

6. Shell Programming and Scripting

delete rows in a file based on the rows of another file

I need to delete rows based on the number of lines in a different file, I have a piece of code with me working but when I merge with my C application, it doesnt work. sed '1,'\"`wc -l < /tmp/fileyyyy`\"'d' /tmp/fileA > /tmp/filexxxx Can anyone give me an alternate solution for the above (2 Replies)
Discussion started by: Muthuraj K
2 Replies

7. UNIX for Advanced & Expert Users

Delete rows from a file...!!

Say i have a file with X rows and Y columns....i see that in some of the rows,some columns are blank (no value set)...i wish to delete such rows....how can it be done? e.g 181766 100 2009-06-04 184443 2009-06-04 10962 151 2009-06-04 161 2009-06-04... (7 Replies)
Discussion started by: ak835
7 Replies

8. Shell Programming and Scripting

Delete rows in text file

Hi I do have a text file with 1000's of lines with 1 row and column with a specific pattern. 1102 1 1 1 1 1234 1 1 1 1 1009 1 1 1 1 1056 1 (3 Replies)
Discussion started by: Lucky Ali
3 Replies

9. Shell Programming and Scripting

Converted repeated rows into splitted columns

Dear Friends, I have an input file contains lot of datas, which is like repaeated rows report. The output file need to have column wise report, rather than row-wise. Input File random line 1 random line 2 random line 3 ------------------------------------- Start line 1.1 (9.9) ... (1 Reply)
Discussion started by: vasanth.vadalur
1 Replies

10. Shell Programming and Scripting

Transposing Repeated Rows to Columns.

I have 1000s of these rows that I would like to transpose to columns. However I would like the transpose every 3 consecutive rows to columns like below, sorted by column 3 and provide a total for each occurrences. Finally I would like a grand total of column 3. 21|FE|41|0B 50\65\78 15... (2 Replies)
Discussion started by: ravzter
2 Replies
DRUC(1) 						      ALLIANCE USER COMMANDS							   DRUC(1)

NAME
druc - Design Rule Checker SYNOPSIS
druc input_name [ -v ] [ -h ] ORIGIN
This software belongs to the ALLIANCE CAD SYSTEM developed by the ASIM team at LIP6 laboratory of Universite Pierre et Marie CURIE, in Paris, France. Web : http://asim.lip6.fr/recherche/alliance/ E-mail : alliance-users@asim.lip6.fr DESCRIPTION
DRuC is a general parametrized VLSI design rule checker. This tool replace the VERSATIL tool that is not anymore supported. This manual presents the layout rules for tle ALLIANCE symbolic layout approach. The rules are described in a technology file defined by the environment variable RDS_TECHNO_NAME (see bellow). The root cell and all the instanciated cells (except the intanciated libraries cells) must be in the current directory. The default mode of DRuC is (currently) full flat: it first flatten all the hierarchy in order to obtain a flat, rectangle level descrip- tion. OPTIONS
-v Verbose mode on. Each step of the DRC is output on the standard output -h Hierarchical design rule checking. Generates lots of files locally, to be used by future invocation of druc. Don't ask me if and how this works, but since you also fetched the sources, ... DESIGN RULES DESCRIPTION
0: LAYER NAME. This section explicits the layer name used in the following rules. - NWELL : N well - NTIE : N well polarisation - PTIE : P substrat polarisation - NDIF : N diffusion - PDIF : P diffusion - GATE : transistor gate - POLY : polysilicon wire - ALU1 : first level of metal - ALU2 : second level of metal - CONT : contact between ALU1 and POLY or DIFF - VIA : contact between ALU1 and ALU2 I: LAYER WIDTH. This class of rules deals with the width limits of a layer. and the conditions for equipotentiality between two overlapping or abutting segments. rule 1 : the minimum width for a segment of NWELL is 4 corresponding error codes are : 100 101 rule 2 : the minimum width for a segment of NTIE is 2 corresponding error code is : 119 rule 3 : the minimum width for a segment of PTIE is 2 corresponding error codes are : 122 123 rule 4 : the minimum width for a segment of NDIF is 2 corresponding error codes are : 140 141 rule 5 : the minimum width for a segment of PDIF is 2 corresponding error codes are : 165 166 rule 6 : the minimum width for a segment of GATE is 1 corresponding error codes are : 234 235 rule 7 : the minimum width for a segment of POLY is 1 corresponding error codes are : 234 235 rule 8 : the minimum width for a segment of ALU1 is 1 corresponding error codes are : 238 239 rule 9 : the minimum width for a segment of ALU2 is 2 corresponding error codes are : 242 243 rule 10 : the width of a CONT must be equal to 1 corresponding error codes are : 246 247 rule 11 : the width of a VIA must be equal to 1 corresponding error codes are : 261 262 II: FORBIDDEN OVERLAP This class of rules specifies the forbidden overlaps between two layers. (The distance between them must be strictly positive) rule 12 : contact between PTIE and NWELL is forbidden. corresponding error codes are : 126 127 128 129 130 131 rule 13 : contact between PTIE and NTIE is forbidden. corresponding error codes are : 133 134 135 136 137 138 rule 14 : contact between NDIF and NWELL is forbidden. corresponding error codes are : 144 145 146 147 148 149 rule 15 : contact between NDIF and NTIE is forbidden. corresponding error codes are : 151 152 153 154 155 156 rule 16 : contact between NDIF and PTIE is forbidden. corresponding error codes are : 158 159 160 161 162 163 rule 17 : contact between PDIF and NTIE is forbidden. corresponding error codes are : 169 170 171 172 173 174 rule 18 : contact between PDIF and PTIE is forbidden. corresponding error codes are : 176 177 178 179 180 181 rule 19 : contact between PDIF and NDIF is forbidden. corresponding error codes are : 183 184 185 186 187 188 rule 20 : contact between GATE and NTIE is forbidden. corresponding error codes are : 191 192 193 194 195 196 rule 21 : contact between GATE and PTIE is forbidden. corresponding error codes are : 198 199 200 201 202 203 rule 22 : contact between POLY and NTIE is forbidden. corresponding error codes are : 207 208 209 210 211 212 rule 23 : contact between POLY and PTIE is forbidden. corresponding error codes are : 214 215 216 217 218 219 rule 24 : contact between POLY and NDIF is forbidden. corresponding error codes are : 221 222 223 224 225 226 rule 25 : contact between POLY and PDIF is forbidden. corresponding error codes are : 228 229 230 231 232 233 rule 26 : contact between CONT and GATE or POLY is forbidden. corresponding error codes are : 249 250 251 252 253 rule 27 : contact between VIA and GATE is forbidden. corresponding error codes are : 264 265 266 267 268 269 rule 28 : contact between VIA and POLY is forbidden. corresponding error codes are : 271 272 273 274 275 276 rule 29 : contact between VIA and CONT is forbidden. corresponding error codes are : 278 279 280 281 282 283 rule 30 : contact between NTIE and NWELL is forbidden. corresponding error code is : 109 rule 31 : contact between PDIF and NWELL is forbidden. corresponding error code is : 117 III: LAYER NOTCH. This class of rules deals with the notch limits of a layer. rule 32 : the minimum notch for a segment of NWELL is 4 corresponding error code is : 102 rule 33 : the minimum notch for a segment of NTIE is 2 corresponding error code is : 120 rule 34 : the minimum notch for a segment of PTIE is 2 corresponding error code is : 124 rule 35 : the minimum notch for a segment of NDIF is 2 corresponding error code is : 142 rule 36 : the minimum notch for a segment of PDIF is 2 corresponding error code is : 167 rule 37 : the minimum notch for a segment of POLY is 1 corresponding error code is : 236 rule 38 : the minimum notch for a segment of ALU1 is 2.5 corresponding error code is : 240 rule 39 : the minimum notch for a segment of ALU2 is 2 corresponding error code is : 244 IV: MINIMUM SPACING This class of rules specifies the minimum edge-to-edge distance allowed between two layers. rule 40 : the minimum distance between NWELL and NWELL is 12 corresponding error code is : 118 rule 42 : the minimum distance between NTIE and NTIE is 3 corresponding error code is : 121 rule 43 : the minimum distance between PTIE and NWELL is 7.5 corresponding error code is : 125 rule 44 : the minimum distance between PTIE and NTIE is 8 corresponding error code is : 132 rule 45 : the minimum distance between PTIE and PTIE is 3 corresponding error code is : 139 rule 46 : the minimum distance between NDIF and NWELL is 7.5 corresponding error code is : 143 rule 47 : the minimum distance between NDIF and NTIE is 8 corresponding error code is : 150 rule 48 : the minimum distance between NDIF and PTIE is 3 corresponding error code is : 157 rule 49 : the minimum distance between NDIF and NDIF is 3 corresponding error code is : 164 rule 51 : the minimum distance between PDIF and NTIE is 3 corresponding error code is : 168 rule 52 : the minimum distance between PDIF and PTIE is 8 corresponding error code is : 175 rule 53 : the minimum distance between PDIF and NDIF is 8 corresponding error code is : 182 rule 54 : the minimum distance between PDIF and PDIF is 3 corresponding error code is : 189 rule 55 : the minimum distance between GATE and NTIE is 1 corresponding error code is : 190 rule 56 : the minimum distance between GATE and PTIE is 1 corresponding error code is : 197 rule 57 : the minimum distance between GATE and NDIF is 1 corresponding error code is : 204 rule 58 : the minimum distance between GATE and PDIF is 1 corresponding error code is : 205 rule 59 : the minimum distance between GATE and GATE is 2 corresponding error code is : 237 rule 60 : the minimum distance between POLY and NTIE is 1 corresponding error code is : 206 rule 61 : the minimum distance between POLY and PTIE is 1 corresponding error code is : 213 rule 62 : the minimum distance between POLY and NDIF is 1 corresponding error code is : 220 rule 63 : the minimum distance between POLY and PDIF is 1 corresponding error code is : 227 rule 64 : the minimum distance between POLY and GATE is 2 corresponding error code is : 237 rule 65 : the minimum distance between POLY and POLY is 2 corresponding error code is : 237 rule 66 : the minimum distance between ALU1 and ALU1 is 2.5 corresponding error code is : 241 rule 67 : the minimum distance between ALU2 and ALU2 is 2 corresponding error code is : 245 rule 68 : the minimum distance between CONT and CONT is 3 corresponding error code is : 254 rule 69 : the minimum distance between VIA and GATE is 2 corresponding error code is : 263 rule 70 : the minimum distance between VIA and POLY is 2 corresponding error code is : 270 rule 71 : the minimum distance between VIA and CONT is 2 corresponding error code is : 277 rule 72 : the minimum distance between VIA and VIA is 3 corresponding error code is : 284 rule 73 : the minimum distance between CONT and GATE or POLY is 1.5 corresponding error code is : 248 V: TOTAL INCLUSION. The last class of rules deals with the inclusion of a layer in another one. rule 74 : NTIE must be included in NWELL with a minimun margin of 0.5 corresponding error code is : 103 rule 75 : PDIF must be included in NWELL with a minimun margin of 0.5 corresponding error code is : 110 FILES
If design errors are found, DRuC produces the list of them in two files : - <root_name.drc>: This ascii file contains the list of DRC violations. - <root_name.iii>: This gds ro cif file contains only rectangles detected in violation. ( suffix iii is defined with the environment ) RDS_OUT_PH is default setted to gds. ENVIRONMENT VARIABLES
DRuC uses several environment variables: - MBK_IN_PH - defines the layout input format. - RDS_OUT_PH - defines the layout output format. - RDS_TECHNO_NAME - defines the technology file. - MBK_CATA_LIB - defines the catalog directory. See the corresponding manual pages for further informations. EXAMPLE
druc register BUG REPORT
This tool is under development at the ASIM department of the LIP6 laboratory. We need your feedback to improve documentation and tools. ASIM
/LIP6 October 1, 1997 DRUC(1)
All times are GMT -4. The time now is 06:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy