Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Removing repeating lines from a data frame (AWK)
Top Forums Shell Programming and Scripting Removing repeating lines from a data frame (AWK) Post 302539932 by bartus11 on Tuesday 19th of July 2011 06:33:58 AM
Old 07-19-2011
As you can see below my code is working as expected...
Code:
solaris% cat file
//DATE TIME                  FRAC_DAYS_SINCE_JAN1       EPOCH_TIME
25/06/2011 07:03           175.2938079                  1308985385
25/06/2011 07:03           175.2938657                  1308985390
25/06/2011 07:03           175.2939236                  1308985395
//DATE TIME                  FRAC_DAYS_SINCE_JAN1       EPOCH_TIME
25/06/2011 07:03           175.2939815                  1308985400
25/06/2011 07:03           175.2940394                  1308985405
25/06/2011 07:03           175.2940972                  1308985410
25/06/2011 07:03           175.2941551                  1308985415
25/06/2011 07:03           175.294213                   1308985420
solaris% awk '!/^\/\//' file                 
25/06/2011 07:03           175.2938079                  1308985385
25/06/2011 07:03           175.2938657                  1308985390
25/06/2011 07:03           175.2939236                  1308985395
25/06/2011 07:03           175.2939815                  1308985400
25/06/2011 07:03           175.2940394                  1308985405
25/06/2011 07:03           175.2940972                  1308985410
25/06/2011 07:03           175.2941551                  1308985415
25/06/2011 07:03           175.294213                   1308985420

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Omit repeating lines

Can someone help me with the following 2 objectives? 1) The following command is just an example. It gets a list of all print jobs. From there I am trying to extract the printer name. It works with the following command: lpstat -W "completed" -o | awk -F- '{ print $1}' Problem is, I want... (6 Replies)
Discussion started by: TheCrunge
6 Replies

2. Shell Programming and Scripting

frame multiple lines into one

Hi, i have a file with contents like below ( any number of entries can be there) 111 222 333 444 555 i need to make another file with single line like below: 111,222,333,444,555 (without ending , ) TIA Prvn (8 Replies)
Discussion started by: prvnrk
8 Replies

3. UNIX for Advanced & Expert Users

removing frame charecters

Hi I have a requirement as follows. My Input file is as follows. COL1,COL2,COL3,COL4,COL5 987,2,3~7~5,400~468~598,0005~4687~5980 1111,2,2~7,400~468,0005~897 Expected OUTPUT ============ COL1,COL2,COL3,COL4,COL5 987,2,3,400,0005 987,2,7,468,4687 987,2,5,598,5980 1111,2,2,400,0005... (6 Replies)
Discussion started by: tkbharani
6 Replies

4. Shell Programming and Scripting

Merging non-repeating columns of lines

Hello, I have file to work with. It has 5 columns. The first three, altogether, constitutes the position. The 4th column contains some values for downstream analysis and the fifth column contains some values that I want to add to 4th column (only if they happen to be in the same position). My... (5 Replies)
Discussion started by: menenuh
5 Replies

5. UNIX for Dummies Questions & Answers

Extract repeating data from file

I want to extract the last rows of a data file, similar to that one below: C1 xxx C2 rrr C3 ttt .... Cn-1 hhh Cn bbb C1 yyy C2 sss C3 uuu ... Cn-1 iii Cn ccc ... I just want to extract the final rows between C1 and Cn at each data file. n is not a constant,... (2 Replies)
Discussion started by: natasha
2 Replies

6. Shell Programming and Scripting

awk removing data before or after a pattern

I have the following data: 01:00:00 29 10 20 41 01:20:00 18 6 34 42 01:40:00 28 5 24 43 02:00:01 11 7 8 74 02:20:01 19 15 12 54 02:40:01 1 4 0 95 03:00:01 1... (6 Replies)
Discussion started by: BeefStu
6 Replies

7. UNIX for Dummies Questions & Answers

Remove groups of repeating lines

I know uniq exists, but am not sure how to remove repeating lines when they are groups of two different lines repeating themselves, without using sort. I need them to be sorted in the original order, just to remove repeats. cd /media/AUDIO/WAVE/9780743518673/mp3 ~/Desktop/mp3-to-m4b... (1 Reply)
Discussion started by: glev2005
1 Replies

8. Shell Programming and Scripting

perform actions at specific locations in data frame

Hi everyone, I got a data frame like the one below and and would like to do the following: Ignore the first 3 rows and check in all following rows the second position. If the value is >500, subtract 100. Example DF: ABC 22 DE 12 BCD 223 GH 12 EFG 2104 DH ... (4 Replies)
Discussion started by: TuAd
4 Replies

9. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

10. Shell Programming and Scripting

Removing certain lines from results - awk

im using the code below to monitor a file: gawk '{ a += gsub("(^| )accepted( |$)", "&") a += gsub("(^| )open database( |$)", "&") } END { for (i in a) printf("%s=%s\n", i, a) }' /var/log/syslog the code is searching the syslog file for the string "accepted" and "open... (2 Replies)
Discussion started by: SkySmart
2 Replies

LEARN ABOUT OPENSOLARIS

iconv_8859-1

iconv_8859-1(5) 					Standards, Environments, and Macros					   iconv_8859-1(5)

NAME

       iconv_8859-1 - code set conversion tables for ISO 8859-1 (Latin 1)

DESCRIPTION

       The following code set conversions are supported:

       +--------------------------------------------------------------------+
       |		  Code Set Conversions Supported		    |
       +--------------+--------+--------------+--------+--------------------+
       |    Code      |Symbol  |Target Code   |Symbol  |   Target Output    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646       |646     | 7 bit ASCII	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646de     |646de   | German 	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646da     |646da   | Danish 	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646en     |646en   | English ASCII	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646es     |646es   | Spanish	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646fr     |646fr   | French 	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646it     |646it   | Italian	    |
       +--------------+--------+--------------+--------+--------------------+
       |ISO 8859-1    |8859    |ISO 646sv     |646sv   | Swedish	    |
       +--------------+--------+--------------+--------+--------------------+

CONVERSIONS

       The conversions are performed according to the following tables. All values in the tables are given in octal.

   ISO 8859-1 to ISO 646 (7-bit ASCII)
       For the conversion of ISO 8859-1 to ISO 646, all characters not in the following table are mapped unchanged.

       Converted to Underscore '_' (137)
       ---------------------------------
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242 243 244 245 246 247
       250 251 252 253 254 255 256 257
       260 261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303 304 305 306 307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325 326 327
       330 331 332 333 334 335 336 337
       340 341 342 343 344 345 346 347
       350 351 352 353 354 355 356 357
       360 361 362 363 364 365 366 367
       370 371 372 373 374 375 376 377

   ISO 8859-1 to ISO 646de (GERMAN)
       For the conversion of ISO 8859-1 to ISO 646de, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |	       |     Conversions|Performed	 |		 |
       |  ISO 8859-1   |   ISO 646de	|  ISO 8859-1	 |  ISO 646de	 |
       |247	       | 100		|337		 |176		 |
       |304	       | 133		|344		 |173		 |
       |326	       | 134		|366		 |174		 |
       |334	       | 135		|374		 |175		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       100 133 134 135 173 174 175 176
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242 243 244 245 246
       250 251 252 253 254 255 256 257
       260 261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303	   305 306 307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325	   327
       330 331 332 333	   335 336 337
       340 341 342 343	   345 346 347
       350 351 352 353 354 355 356 357
       360 361 362 363 364 365	   367
       370 371 372 373	   375 376 377

   ISO 8859-1 to ISO 646da (DANISH)
       For the conversion of ISO 8859-1 to ISO 646da, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |		     Conversions Performed			 |
       +---------------+----------------+----------------+---------------+
       |  ISO 8859-1   |   ISO 646da	|  ISO 8859-1	 |     ISO	 |
       +---------------+----------------+----------------+---------------+
       |646da	       |		|		 |		 |
       +---------------+----------------+----------------+---------------+
       |305	       | 135		|345		 |175		 |
       +---------------+----------------+----------------+---------------+
       |306	       | 133		|346		 |173		 |
       +---------------+----------------+----------------+---------------+
       |330	       | 134		|370		 |174		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       133 134 135 173 174 175
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242 243 244 245 246 247
       250 251 252 253 254 255 256 257
       260 261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303 304	   307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325 326 327
	   331 332 333 334 335 336 337
       340 341 342 343 344	   347
       350 351 352 353 354 355 356 357
       360 361 362 363 364 365 366 367
       371 372 373 374	       376 377

   ISO 8859-1 to ISO 646en (ENGLISH ASCII)
       For the conversion of ISO 8859-1 to ISO 646en, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------+
       |		  Conversions Performed 		   |
       +-----------------------------+-----------------------------+
       |	ISO 8859-1	     |	       ISO 646en	   |
       +-----------------------------+-----------------------------+
       |243			     |043			   |
       +-----------------------------+-----------------------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       043
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242     244 245 246 247
       250 251 252 253 254 255 256 257
       260 261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303 304 305 306 307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325 326 327
       330 331 332 333 334 335 336 337
       340 341 342 343 344 345 346 347
       350 351 352 353 354 355 356 357
       360 361 362 363 364 365 366 367
       370 371 372 373 374 375 376 377

   ISO 8859-1 to ISO 646fr (FRENCH)
       For the conversion of ISO 8859-1 to ISO 646fr, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |	       |     Conversions|Performed	 |		 |
       |  ISO 8859-1   |   ISO 646fr	|  ISO 8859-1	 |  ISO 646fr	 |
       |243	       | 043		|347		 |134		 |
       |247	       | 135		|350		 |175		 |
       |250	       | 176		|351		 |173		 |
       |260	       | 133		|371		 |174		 |
       |340	       | 100		|		 |		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       043
       100 133 134 135 173 174 175 176
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242     244 245 246
	   251 252 253 254 255 256 257
	   261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303 304 305 306 307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325 326 327
       330 331 332 333 334 335 336 337
	   341 342 343 344 345 346
	       352 353 354 355 356 357
       360 361 362 363 364 365 366 367
       370     372 373 374 375 376 377

   ISO 8859-1 to ISO 646it (ITALIAN)
       For the conversion of ISO 8859-1 to ISO 646it, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |	       |     Conversions|Performed	 |		 |
       |  ISO 8859-1   |   ISO 646it	|  ISO 8859-1	 |  ISO 646it	 |
       |243	       | 043		|350		 |175		 |
       |247	       | 100		|351		 |135		 |
       |260	       | 133		|354		 |176		 |
       |340	       | 173		|362		 |174		 |
       |347	       | 134		|371		 |140		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       043
       100 133 134 135 173 174 175 176
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242     244 245 246
       250 251 252 253 254 255 256 257
	   261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303 304 305 306 307
       310 311 312 313 314 315 316 317
       320 321 322 323 324 325 326 327
       330 331 332 333 334 335 336 337
	   341 342 343 344 345 346
	       352 353 354 355 356 357
       360 361 362 363 364 365 366 367
       370     372 373 374 375 376 377

   ISO 8859-1 to ISO 646es (SPANISH)
       For the conversion of ISO 8859-1 to ISO 646es, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |	       |     Conversions|Performed	 |		 |
       |  ISO 8859-1   |   ISO 646es	|  ISO 8859-1	 |  ISO 646es	 |
       |241	       | 133		|321		 |134		 |
       |247	       | 100		|347		 |175		 |
       |260	       | 173		|361		 |174		 |
       |277	       | 135		|		 |		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       100 133 134 135 173 174 175
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240     242 243 244 245 246
       250 251 252 253 254 255 256 257
	   261 262 263 264 265 266 267
       270 271 272 273 274 275 276
       300 301 302 303 304 305 306 307
       310 311 312 313 314 315 316 317
       320     322 323 324 325 326 327
       330 331 332 333 334 335 336 337
       340 341 342 343 344 345 346
       350 351 352 353 354 355 356 357
       360     362 363 364 365 366 367
       370 371 372 373 374 375 376 377

   ISO 8859-1 to ISO 646sv (SWEDISH)
       For the conversion of ISO 8859-1 to ISO 646sv, all characters not in the following tables are mapped unchanged.

       +-----------------------------------------------------------------+
       |	       |     Conversions|Performed	 |		 |
       |  ISO 8859-1   |   ISO 646sv	|  ISO 8859-1	 |  ISO 646sv	 |
       |304	       | 133		|344		 |173		 |
       |305	       | 135		|345		 |175		 |
       |311	       | 100		|351		 |140		 |
       |326	       | 134		|366		 |174		 |
       |334	       | 136		|374		 |176		 |
       +---------------+----------------+----------------+---------------+

       Converted to Underscore '_' (137)
       ---------------------------------
       100 133 134 135 136 140
       173 174 175 176
       200 201 202 203 204 205 206 207
       210 211 212 213 214 215 216 217
       220 221 222 223 224 225 226 227
       230 231 232 233 234 235 236 237
       240 241 242 243 244 245 246 247
       250 251 252 253 254 255 256 257
       260 261 262 263 264 265 266 267
       270 271 272 273 274 275 276 277
       300 301 302 303	       306 307
       310     312 313 314 315 316 317
       320 321 322 323 324 325	   327
       330 331 332 333	   335 336 337
       340 341 342 343	       346 347
       350     352 353 354 355 356 357
       360 361 362 363 364 365	   367
       370 371 372 373	   375 376 377

FILES

       /usr/lib/iconv/*.so	       conversion modules

       /usr/lib/iconv/*.t	       conversion tables

       /usr/lib/iconv/iconv_data       list of conversions supported by conversion tables

SEE ALSO

       iconv(1), iconv(3C), iconv(5)

SunOS 5.10							    28 Apr 1997 						   iconv_8859-1(5)
All times are GMT -4. The time now is 06:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy