Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Delete repeated rows from a file
Top Forums Shell Programming and Scripting Delete repeated rows from a file Post 302137244 by jaduks on Monday 24th of September 2007 02:35:38 AM
Old 09-24-2007
[jsaikia] ~/prac/ $ cat file | awk '{print $1}' | sort | uniq | while read firstf; do awk '$1=="'"$firstf"'"' file | sed '1!d' ; done

0.490 958.73 281.85 6.67985 0.002481
0.590 950.504 286.241 6.61451 0.002461
0.690 939.323 286.112 6.16451 0.00246
0.790 928.17 285.71 5.87057 0.002451
0.890 917.196 285.503 5.6777 0.002441
0.990 906.277 284.498 5.46275 0.00244
1.090 895.529 283.818 5.43785 0.002431
1.190 884.757 283.098 5.36579 0.002421
1.290 874.22 282.2 5.33933 0.00242
1.390 863.667 281.35 5.01376 0.002411
1.490 853.3 280.55 4.61738 0.00241
1.590 842.962 279.95 4.27487 0.002401
1.690 832.775 279.362 3.77744 0.002391
1.790 822.634 278.532 3.78002 0.00239
1.890 812.608 277.625 3.98339 0.002381
1.990 802.735 276.995 4.17061 0.00238
2.090 792.845 276.65 4.77151 0.002389
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete particular rows from a file

Hi I have a file having 1000 rows. Now I would like to remove 10 rows from it. Plz give me the script. Eg: input file like 4 1 4500.0 1 5 1 1.0 30 6 1 1.0 4500 7 1 4.0 730 7 2 500000.0 730 8 1 785460.0 45 8 7 94255.0 30 9 1 31800.0 30 9 4 36000.0 30 10 1 15000.0 30... (5 Replies)
Discussion started by: suresh3566
5 Replies

2. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

3. UNIX for Dummies Questions & Answers

Delete repeated nos in a file

Hi, I need to delete repeated nos in a file and finally list the count. Can some one assist me? file: 12345 12345 56345 12345 23896 Output needed: 12345 56345 23896 Total count:3 Thanks (2 Replies)
Discussion started by: gini
2 Replies

4. Shell Programming and Scripting

Delete repeated word in text file

Hi expert, I am using C shell. And i trying to delete repeated word. Example file.txt: BLUE YELLOW RED VIOLET RED RED BLUE WHITE YELLOW BLACK and i wan store the output into a new file: BLUE (6 Replies)
Discussion started by: vincyoxy
6 Replies

5. Shell Programming and Scripting

[HELP] - Delete rows on a CSV file

Hello to all members, I am very new in unix stuff (shell scripting), but a want to learn a lot. I am a ex windows user but now i am absolutely Linux super user... :D So i am tryng to made a function to do this: I have two csv files only with numbers, the first one a have: 1 2 3 4 5... (6 Replies)
Discussion started by: Sadarrab
6 Replies

6. Shell Programming and Scripting

delete rows in a file based on the rows of another file

I need to delete rows based on the number of lines in a different file, I have a piece of code with me working but when I merge with my C application, it doesnt work. sed '1,'\"`wc -l < /tmp/fileyyyy`\"'d' /tmp/fileA > /tmp/filexxxx Can anyone give me an alternate solution for the above (2 Replies)
Discussion started by: Muthuraj K
2 Replies

7. UNIX for Advanced & Expert Users

Delete rows from a file...!!

Say i have a file with X rows and Y columns....i see that in some of the rows,some columns are blank (no value set)...i wish to delete such rows....how can it be done? e.g 181766 100 2009-06-04 184443 2009-06-04 10962 151 2009-06-04 161 2009-06-04... (7 Replies)
Discussion started by: ak835
7 Replies

8. Shell Programming and Scripting

Delete rows in text file

Hi I do have a text file with 1000's of lines with 1 row and column with a specific pattern. 1102 1 1 1 1 1234 1 1 1 1 1009 1 1 1 1 1056 1 (3 Replies)
Discussion started by: Lucky Ali
3 Replies

9. Shell Programming and Scripting

Converted repeated rows into splitted columns

Dear Friends, I have an input file contains lot of datas, which is like repaeated rows report. The output file need to have column wise report, rather than row-wise. Input File random line 1 random line 2 random line 3 ------------------------------------- Start line 1.1 (9.9) ... (1 Reply)
Discussion started by: vasanth.vadalur
1 Replies

10. Shell Programming and Scripting

Transposing Repeated Rows to Columns.

I have 1000s of these rows that I would like to transpose to columns. However I would like the transpose every 3 consecutive rows to columns like below, sorted by column 3 and provide a total for each occurrences. Finally I would like a grand total of column 3. 21|FE|41|0B 50\65\78 15... (2 Replies)
Discussion started by: ravzter
2 Replies

LEARN ABOUT OPENSOLARIS

iconv_8859-5

iconv_8859-5(5)                                         Standards, Environments, and Macros                                        iconv_8859-5(5)

NAME

       iconv_8859-5 - code set conversion tables for ISO 8859-5 (Cyrillic)

DESCRIPTION

       The following code set conversions are supported:

       +-------------------------------------------------------------------------+
       |                     Code Set Conversions Supported                      |
       +--------------+--------+--------------+--------+-------------------------+
       |    Code      |Symbol  |Target Code   |Symbol  |      Target Output      |
       +--------------+--------+--------------+--------+-------------------------+
       |ISO 8859-5    |iso5    |KOI8-R        |koi8    | KOI8-R                  |
       +--------------+--------+--------------+--------+-------------------------+
       |ISO 8859-5    |iso5    |PC Cyrillic   |alt     | Alternative PC Cyrillic |
       +--------------+--------+--------------+--------+-------------------------+
       |ISO 8859-5    |iso5    |MS 1251       |win5    | Windows Cyrillic        |
       +--------------+--------+--------------+--------+-------------------------+
       |ISO 8859-5    |iso5    |Mac Cyrillic  |mac     | Macintosh Cyrillic      |
       +--------------+--------+--------------+--------+-------------------------+

CONVERSIONS

       The conversions are performed according to the following tables. All values in the tables are given in octal.

   ISO 8859-5 to KOI8-R
       For the conversion of ISO 8859-5 to KOI8-R, all characters not in the following table are mapped unchanged.

       +-----------------------------------------------------------------+
       |               |     Conversions|Performed       |               |
       |  ISO 8859-5   |    KOI8-R      |  ISO 8859-5    |    KOI8-R     |
       |24             | 4              |320             |301            |
       |241            | 263            |321             |302            |
       |242            | 261            |322             |327            |
       |243            | 262            |323             |307            |
       |244            | 264            |324             |304            |
       |245            | 265            |325             |305            |
       |246            | 266            |327             |332            |
       |247            | 267            |330             |311            |
       |250            | 270            |331             |312            |
       |251            | 271            |332             |313            |
       |252            | 272            |333             |314            |
       |253            | 273            |334             |315            |
       |254            | 274            |335             |316            |
       |256            | 276            |336             |317            |
       |257            | 277            |337             |320            |
       |260            | 341            |340             |322            |
       |261            | 342            |341             |323            |
       |262            | 367            |342             |324            |
       |263            | 347            |343             |325            |
       |264            | 344            |344             |306            |
       |265            | 345            |345             |310            |
       |266            | 366            |346             |303            |
       |267            | 372            |347             |336            |
       |270            | 351            |350             |333            |
       |271            | 352            |351             |335            |
       |272            | 353            |352             |337            |
       |273            | 354            |353             |331            |
       |274            | 355            |354             |330            |
       |275            | 356            |355             |334            |
       |276            | 357            |356             |300            |
       |277            | 360            |357             |321            |
       |300            | 362            |360             |260            |
       |301            | 363            |361             |243            |
       |302            | 364            |362             |241            |
       |303            | 365            |363             |242            |
       |304            | 346            |364             |244            |
       |305            | 350            |365             |245            |
       |306            | 343            |366             |246            |
       |307            | 376            |367             |247            |
       |310            | 373            |370             |250            |
       |311            | 375            |371             |251            |
       |312            | 377            |372             |252            |
       |313            | 371            |373             |253            |
       |314            | 370            |374             |254            |
       |315            | 374            |375             |255            |
       |316            | 340            |376             |256            |
       |317            | 361            |                |               |
       +---------------+----------------+----------------+---------------+

   ISO 8859-5 to PC Cyrillic
       For the conversion of ISO 8859-5 to PC Cyrillic, all characters not in the following table are mapped unchanged.

       +-----------------------------------------------------------------+
       |               |     Conversions|Performed       |               |
       |  ISO 8859-5   |  PC Cyrillic   |  ISO 8859-5    | PC Cyrillic   |
       |24             | 4              |307             |227            |
       |200-240        | 40             |310             |230            |
       |241            | 360            |311             |231            |
       |242-254        | 40             |312             |232            |
       |255            | 260            |313             |233            |
       |256-257        | 40             |314             |234            |
       |260            | 200            |315             |235            |
       |261            | 201            |316             |236            |
       |262            | 202            |317             |237            |
       |263            | 203            |320             |240            |
       |264            | 204            |321             |241            |
       |265            | 205            |322             |242            |
       |266            | 206            |323             |243            |
       |267            | 207            |324             |244            |
       |270            | 210            |325             |245            |
       |271            | 211            |326             |246            |
       |272            | 212            |327             |247            |
       |273            | 213            |330             |250            |
       |274            | 214            |331             |251            |
       |275            | 215            |332             |252            |
       |276            | 216            |333             |253            |
       |277            | 217            |334             |254            |
       |300            | 220            |335             |255            |
       |301            | 221            |336             |256            |
       |302            | 222            |337             |257            |
       |303            | 223            |360-374         |40             |
       |304            | 224            |375             |260            |
       |305            | 225            |376             |40             |
       |306            | 226            |365             |40             |
       +---------------+----------------+----------------+---------------+

   ISO 8859-5 to MS 1251
       For the conversion of ISO 8859-5 to MS 1251, all characters not in the following table are mapped unchanged.

       +-----------------------------------------------------------------+
       |               |     Conversions|Performed       |               |
       |  ISO 8859-5   |    MS 1251     |  ISO 8859-5    |   MS 1251     |
       |24             | 4              |317             |337            |
       |200-237        | 40             |320             |340            |
       |241            | 250            |321             |341            |
       |242            | 200            |322             |342            |
       |243            | 201            |323             |343            |
       |244            | 252            |324             |344            |
       |245            | 275            |325             |345            |
       |246            | 262            |326             |346            |
       |247            | 257            |327             |347            |
       |250            | 243            |330             |350            |
       |251            | 212            |331             |351            |
       |252            | 214            |332             |352            |
       |253            | 216            |333             |353            |
       |254            | 215            |334             |354            |
       |255            | 210            |335             |355            |
       |256            | 241            |336             |356            |
       |257            | 217            |337             |357            |
       |260            | 300            |340             |360            |
       |261            | 301            |341             |361            |
       |262            | 302            |342             |362            |
       |263            | 303            |343             |363            |
       |264            | 304            |344             |364            |
       |265            | 305            |345             |365            |
       |266            | 306            |346             |366            |
       |267            | 307            |347             |367            |
       |270            | 310            |350             |370            |
       |271            | 311            |351             |371            |
       |272            | 312            |352             |372            |
       |273            | 313            |353             |373            |
       |274            | 314            |354             |374            |
       |275            | 315            |355             |375            |
       |276            | 316            |356             |376            |
       |277            | 317            |357             |377            |
       |300            | 320            |360             |271            |
       |301            | 321            |361             |270            |
       |302            | 322            |362             |220            |
       |303            | 323            |363             |203            |
       |304            | 324            |364             |272            |
       |305            | 325            |365             |276            |
       |306            | 326            |366             |263            |
       |307            | 327            |367             |277            |
       |310            | 330            |370             |274            |
       |311            | 331            |371             |232            |
       |312            | 332            |372             |234            |
       |313            | 333            |373             |236            |
       |314            | 334            |374             |235            |
       |315            | 335            |375             |210            |
       |316            | 336            |376             |242            |
       |376            | 331            |                |               |
       +---------------+----------------+----------------+---------------+

   ISO 8859-5 to Mac Cyrillic
       For the conversion of ISO 8859-5 to Mac Cyrillic, all characters not in the following table are mapped unchanged.

       +-----------------------------------------------------------------+
       |               |     Conversions|Performed       |               |
       |  ISO 8859-5   | Mac Cyrillic   |  ISO 8859-5    | Mac Cyrillic  |
       |24             | 4              |317             |237            |
       |200-237        | 40             |320             |340            |
       |240            | 312            |321             |341            |
       |241            | 335            |322             |342            |
       |242            | 253            |323             |343            |
       |243            | 256            |324             |344            |
       |244            | 270            |325             |345            |
       |245            | 301            |326             |346            |
       |246            | 247            |327             |347            |
       |247            | 272            |330             |350            |
       |250            | 267            |331             |351            |
       |251            | 274            |332             |352            |
       |252            | 276            |333             |353            |
       |253            | 40             |334             |354            |
       |254            | 315            |335             |355            |
       |255            | 40             |336             |356            |
       |256            | 330            |337             |357            |
       |257            | 332            |340             |360            |
       |260            | 200            |341             |361            |
       |261            | 201            |342             |362            |
       |262            | 202            |343             |363            |
       |263            | 203            |344             |364            |
       |264            | 204            |345             |365            |
       |265            | 205            |346             |366            |
       |266            | 206            |347             |367            |
       |267            | 207            |350             |370            |
       |270            | 210            |351             |371            |
       |271            | 211            |352             |372            |
       |272            | 212            |353             |373            |
       |273            | 213            |354             |374            |
       |274            | 214            |355             |375            |
       |275            | 215            |356             |376            |
       |276            | 216            |357             |337            |
       |277            | 217            |360             |334            |
       |300            | 220            |361             |336            |
       |301            | 221            |362             |254            |
       |302            | 222            |363             |257            |
       |303            | 223            |364             |271            |
       |304            | 224            |365             |317            |
       |305            | 225            |366             |264            |
       |306            | 226            |367             |273            |
       |307            | 227            |370             |300            |
       |310            | 230            |371             |275            |
       |311            | 231            |372             |277            |
       |312            | 232            |373             |40             |
       |313            | 233            |374             |316            |
       |314            | 234            |375             |40             |
       |315            | 235            |376             |331            |
       |316            | 236            |                |               |
       +---------------+----------------+----------------+---------------+

FILES

       /usr/lib/iconv/*.so             conversion modules

       /usr/lib/iconv/*.t              conversion tables

       /usr/lib/iconv/iconv_data       list of conversions supported by conversion tables

SEE ALSO

       iconv(1), iconv(3C), iconv(5)

SunOS 5.10                                                          18 Apr 1997                                                    iconv_8859-5(5)
All times are GMT -4. The time now is 09:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy