Sponsored Content
Top Forums Shell Programming and Scripting Remove duplicate line detail based on column one data Post 302384953 by rdcwayx on Wednesday 6th of January 2010 06:36:33 PM
Old 01-06-2010
Quote:
Originally Posted by gaurav1086
try this,
Code:
awk 'n!=$1{print;n=$1}' infile

regards
this is better, which can remove duplicated empty line.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

2. Shell Programming and Scripting

remove new line characters from a partcular column data

Dear friends, I have a pipe delimited file having 5 columns. However the column no-3 is having extra new line characters as the data owing to owing , I am having issues. Ideally my file should have only newline termination at the end of each record and not within column data of any of... (1 Reply)
Discussion started by: sureshg_sampat
1 Replies

3. Shell Programming and Scripting

Help with remove duplicate content and only keep the first content detail

Input data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_21 SSA data_19 TYUEC data_14 TYUE data_15 SSA data_32 PEOCV . . Desired Output data_10 SSA data_2 TYUE data_3 PEOCV data_6 SSAT data_19 TYUEC (9 Replies)
Discussion started by: patrick87
9 Replies

4. Shell Programming and Scripting

How to remove a subset of data from a large dataset based on values on one line

Hello. I was wondering if anyone could help. I have a file containing a large table in the format: marker1 marker2 marker3 marker4 position1 position2 position3 position4 genotype1 genotype2 genotype3 genotype4 with marker being a name, position a numeric... (2 Replies)
Discussion started by: davegen
2 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicate rows when >10 based on single column value

Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column. e.g. a 1 a 2 a 3 b 1 c 1 gives b 1 c 1 but requires 11 duplicates before it deletes. Thanks for the help Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies

6. Shell Programming and Scripting

Help with figuring division and addition based on column data and line numbers

I have a data file in the format of 1234 xxx 1234 xxx 1234 xxx 1234 xxxI want to be able to calculate the following - COLUMN1+((LINENUMBER-1)/365) The output needs to preserve the 2nd column - 1234 xxx 1234.00274 xxx 1234.00548 xxx What is the best way to do this? I am somewhat... (9 Replies)
Discussion started by: ncwxpanther
9 Replies

7. Shell Programming and Scripting

How to remove a line based on contents of the first column?

Good day all. Using basic UNIX/Linux tools, how would you delete a line based on a character found in column 1? For example, if the CITY name contains an 'a' or 'A', delete the line: New York City; New York Los Angeles; California Chicago; Illinois Houston; Texas Philadelphia;... (3 Replies)
Discussion started by: BRH
3 Replies

8. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

9. Shell Programming and Scripting

Remove sections based on duplicate first line

Hi, I have a file with many sections in it. Each section is separated by a blank line. The first line of each section would determine if the section is duplicate or not. if the section is duplicate then remove the entire section from the file. below is the example of input and output.... (5 Replies)
Discussion started by: ahmedwaseem2000
5 Replies

10. Shell Programming and Scripting

To append new data at the end of each line based on substring of last column

Hi guys, I need to append new data at the end of each line of the files. This new data is based on substring (3rd fields) of last column. Input file xxx.csv: U1234|1-5X|orange|1-5X|Act|1-5X|0.1 /sac/orange 12345 0 U5678|1-7X|grape|1-7X|Act|1-7X|0.1 /sac/grape 5678 0... (5 Replies)
Discussion started by: null7
5 Replies
yuv2lav(1)							MJPEG tools manual							yuv2lav(1)

NAME
yuv2lav - encodes lavtool's raw YUV frame streams into MJPEG files SYNOPSIS
yuv2lav [-f a|A|q|m] [-q num] [-b num] [-I num] [-m num] [-v num] [-w file] -o output-file DESCRIPTION
yuv2lav is a simple MJPEG encoder for raw YUV frame streams as they are emitted by lav2yuv(1) or lavpipe(1), for example. It reads its input from stdin and is capable of writing AVI and Quicktime. OPTIONS
The command line options allow to specify the output file name, JPEG encoding quality and the amount of memory that will be allocated for JPEG compressed data. -o output-file This (compulsorary) option sets the name of the file that yuv2lav will write its output to. If the output file contains a % (e.g. file%02d.avi), the output will be written to multiple files in case this is necessary (file00.avi, file01.avi and so on). -f a|A|q This sets the output file format, which has to be one of a - for AVI file output, A - for AVI with reversed fields, q - for Quicktime output or AVI with reversed field of course only makes sense if yuv2lav's feed is an interlaced video source and should only be needed if you experience frame reversal problems in your output. If you don't use that option yuv2lav will set the output format to AVI if the last 3 char are avi, or to Quicktime if you use mov. If you use the -f option it overides the setting in the filename -q num This option specifies the JPEG encoding quality as passed through to libjpeg. The default value is 80, num must lie within 24...100 inclusive (think of it as percentage values, although with a lossy compression algorithm like JPEG, keeping 100% of the original quality is of course not really possible). -b num This option specifies the size of the buffer (in kBytes) that yuv2lav allocates in order to store the compressed JPEG data in it. The default value (256k) is perfectly enough for 100% quality full sized PAL/NTSC, but if you want to compress Super-Mega-HDTV or if you are low on memory, you can use this option to suit your needs. -m num This is the maximum size (in MB) per file. Normally this depends on the output file type. -I num Force a specific interlacing type. 0 means no interlacing, 1 means top-field-first, 2 means bottom-field-first. -w file This is the WAV file containing audio data to be combined with the video stream into the output file. -v num Verbosity level. 0 means only print error messages, 2 prints full debug output. -? Display a synopsis of the command syntax. EXAMPLES
lav2yuv movie.avi | yuv2lav -fq -q 30 lowbitrate.qt would recompress movie.avi as a low bit rate Quicktime file. lavpipe input.pli | yuv2lav -q80 output.avi would save the movie assembled by lavpipe as a single AVI file. BUGS
If you experience any problems with this tool, please feel free to contact the developers (see below). AUTHOR
This man page was written by pHilipp Zabel. If you have questions, remarks, problems or you just want to contact the developers, the main mailing list for the MJPEG-tools is: mjpeg-users@lists.sourceforge.net For more info, see our website at http://mjpeg.sourceforge.net/ SEE ALSO
lav2yuv(1), lavpipe(1), lavplay(1), lavrec(1), mpeg2enc(1), yuvscaler(1) MJPEG Linux Square 2 June 2001 yuv2lav(1)
All times are GMT -4. The time now is 03:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy