Sponsored Content
Top Forums Shell Programming and Scripting How do a distinct from a file using sort uniq in bash? Post 302806455 by punticci on Monday 13th of May 2013 05:48:38 AM
Old 05-13-2013
Quote:
Originally Posted by Skrynesaver
could you provide some sample data, identifying the ID?
This is an example of data:

Code:
ID                          COD1        DATA                    COD2                   COD3                    DATA2

;0@13013008021958146502;2@*;3@MGF;4@30-JAN-13 07.57.35.000000 AM;5@0757350024630007;6@DSHDQR;7@30-JAN-13;8@30-JAN-13;9@S;10@
;0@13013008021958146502;2@*;3@MGF;4@30-JAN-13 07.57.35.000000 AM;5@0757350086490008;6@SGCFXT;7@30-JAN-13;8@30-JAN-13;9@S;10@
;0@13013008021958146502;2@*;3@ARC;4@30-JAN-13 07.57.35.000000 AM;5@27864130075735394277000;6@FSC4;7@30-JAN-13;8@30-JAN-13;9@S;10@
;0@13013008021958146502;2@*;3@TER;4@30-JAN-13 07.57.35.000000 AM;5@20130130075735448370000;6@HBG4;7@30-JAN-13;8@30-JAN-13;9@S;10@
;0@13013008021958146502;2@*;3@REB;4@30-JAN-13 07.57.39.000000 AM;5@20130130075741502619000;6@HBG4 ;7@30-JAN-13;8@30-JAN-13;9@S;10@
;0@13013008021958146502;2@*;3@IGI;4@30-JAN-13 08.02.38.000000 AM;5@20154210080238521224000;6@FSC4;7@30-JAN-13;8@30-JAN-13;9@S;10@

Every Col is separated with this ;0@ for the 1st filed ;1@ for the second and so on
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sort/uniq

I have a file: Fred Fred Fred Jim Fred Jim Jim If sort is executed on the listed file, shouldn't the output be?: Fred Fred Fred Fred Jim Jim Jim (3 Replies)
Discussion started by: jimmyflip
3 Replies

2. Shell Programming and Scripting

sort and uniq in perl

Does anyone have a quick and dirty way of performing a sort and uniq in perl? How an array with data like: this is bkupArr BOLADVICE_VN this is bkupArr MLT6800PROD2A this is bkupArr MLT6800PROD2A this is bkupArr BOLADVICE_VN_7YR this is bkupArr MLT6800PROD2A I want to sort it... (4 Replies)
Discussion started by: reggiej
4 Replies

3. Shell Programming and Scripting

Help with Uniq and sort

The key is first field i want only uniq record for the first field in file. I want the output as or output as Appreciate help on this (4 Replies)
Discussion started by: pinnacle
4 Replies

4. UNIX for Dummies Questions & Answers

Sort and uniq lines of a file while keeping a header line

So, I have a file that has some duplicate lines. The file has a header line that I would like to keep at the top. I could do this by extracting the header from the file, 'sort -u' the remaining lines, and recombine them. But they are quite big, so if there is a way to do it with a single... (1 Reply)
Discussion started by: Digby
1 Replies

5. UNIX for Advanced & Expert Users

$cat file|sort|uniq

I have a file with list of redundant server names and want only unique ones of those. I have used following command but still redudant ones are listing $cat file|sort|uniq where could be the problem. Thanks, Srinivas (3 Replies)
Discussion started by: srinivas Juluri
3 Replies

6. Shell Programming and Scripting

sort | uniq question

Hello, I have a large data file: 1234 8888 bbb 2745 8888 bbb 9489 8888 bbb 1234 8888 aaa 4838 8888 aaa 3977 8888 aaa I need to remove duplicate lines (where the first column is the duplicate). I have been using: sort file.txt | uniq -w4 > newfile.txt However, it seems to keep the... (11 Replies)
Discussion started by: palex
11 Replies

7. Shell Programming and Scripting

Sort uniq or awk

Hi again, I have files with the following contents datetime,ip1,port1,ip2,port2,number How would I find out how many times ip1 field shows up a particular file? Then how would I find out how many time ip1 and port 2 shows up? Please mind the file may contain 100k lines. (8 Replies)
Discussion started by: LDHB2012
8 Replies

8. Shell Programming and Scripting

Trying to find the distinct lines using uniq command

Platform :Oracle Linux 6.4 Shell : bash The below file has 7 lines , some of them are duplicates. There are only 3 distinct lines. But why is the uniq command still showing 7 ? I just want the distinct lines to be returned. $ cat test.txt SELECT FC.COORD_SET_ID FROM OM_ORDER_FLOW F, -... (2 Replies)
Discussion started by: kraljic
2 Replies

9. UNIX for Dummies Questions & Answers

Uniq and sort -u

Hello all, Need to pick your brains, I have a 10Gb file where each row is a name, I am expecting about 50 names in total. So there are a lot of repetitions in clusters. So I want to do a sort -u file Will it be considerably faster or slower to use a uniq before piping it to sort... (3 Replies)
Discussion started by: senhia83
3 Replies

10. Shell Programming and Scripting

Sort & Uniq -u

Hi All, Below the actual file which i like to sort and Uniq -u /opt/oracle/work/Antony/Shell_Script> cat emp.1st 2233|a.k. shukula |g.m. |sales |12/12/52 |6000 1006|chanchal singhvi |director |sales |03/09/38 |6700... (8 Replies)
Discussion started by: Antony Ankrose
8 Replies
OGGCAT(1)							   User Manuals 							 OGGCAT(1)

NAME
oggCat - concatenates two ogg video files (.ogv, .ogg or oga) SYNOPSIS
oggCat [options] outfile.ogv file1.ogv file2.ogv [ file3.ogv [...] ] DESCRIPTION
oggCat concatenates two or more ogg files. The parameters of the resulting ogg file is defined by the first file in the concatenation list. The parameters could be changed by some options, explained below. All subsequent files that does not match these video file parameters are automatically transcoded. A concatenation will fail in case that there are not enought streams available. This could happen e.g. if the first file consists of one audio and one video stream and the second file only carries a video stream. If there is more than one video or audio stream within the file, the first stream is used. OPTIONS
-s Sets the size of the video frame. The size is given as <width>x<height>. At default, the video frame size is the size of the first video stream. Example: -s 320x240 -d Sets the datarate in byte per seconds for the video encoder (theora). This meant to be a upper threshold. So the file may be smaller than assumed. If not set, the datarate of the first file is used. Example: -d 1024000 -D Sets the datarate in byte per seconds for the audio encoder (vorbis). If not set, the datarate of the first file is used. Example: -D 64000 -f Sets the frame rate of the video with numinator and demoninator and is the pictures per second. If only one number is given, the denominator is set to 1. If not set, the framerate of the first file is used. Example: -f 25:2 -F Sets the sample frequency (sample rate) of the audio data in Hertz. If the sample frequency does not match the one with the original file, resamling is invoked. Example: -F 32000 -c Adds comments to the video (theora) stream. Comments are given by a pair of type and value in the form 'type=value'. More than one comment can be concatenated with a semicolon. It is recommended to use apostrophes as the command line may use the semicolon as a seperator. Example: -c 'AUTHOR=yorn;DATE=03.07.09' -C Adds comments to the audio (vorbis) stream. Comments are given by a pair of type and value in the form 'type=value'. More than one comment can be concatenated with a semicolon. It is recommended to use apostrophes as the command line may use the semicolon as a seperator. Example: -C 'AUTHOR=yorn;DATE=03.07.09' -q Specifies the video quality for the newly created stream. Valid values can be chosen between 0 and 63 (best). Example: -q 63 -rv Force to reencode the video stream. This is sometimes neccessary, if the video stream match in video parameters, but not in the the- ora version. In that case, reencoding the video stream helps creating good results. -x Force to overwrite the output file, even if it exists. This is mainly helpful if the oggCat is server controlled. EXAMPLES
oggCat concatFile.ogv myfile1.ogv myfile2.ogv myfile3.ogv or oggCat -o concatFile.ogv myfile1.ogv myfile2.ogv myfile3.ogv This command creates a cancatenated file concatFile.ogv that consists of the three files myfile1.ogv myfile2.ogv myfile3.ogv oggCat -s320x240 -q63 concatFile.ogv myfile1.ogv myfile2.ogv myfile3.ogv This command connects the three files myfile[1-3].ogv to file concatFile.ogv with the size of 320x240 and best quality. AUTHOR
Joern Seger <yorn at gmx dot net> SEE ALSO
oggCut(1), oggJoin(1), oggSplit(1), oggTranscode(1), oggSlideshow(1), oggThumb(1), oggSilence(1) Linux JAN 2010 OGGCAT(1)
All times are GMT -4. The time now is 09:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy