Sponsored Content
Top Forums Shell Programming and Scripting squeeze duplicates from a table Post 302423933 by Scott on Sunday 23rd of May 2010 01:49:28 PM
Old 05-23-2010
Code:
sort -t\| -nrsk2 file1 | awk -F\| '!A[$1]++'

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

TR squeeze oddity

I discovered that where 'tr -s' works as expected on grepped input, it appears to completely fail on dig results. I am not sure if this is because of some sort of non posix compliancy, or what. Here is what I did: The command below works as expected, squeezing all repeated spaces to a single... (3 Replies)
Discussion started by: bdmeyersc
3 Replies

2. Shell Programming and Scripting

Merge Two Tables with duplicates in first table

Hi.. File 1: 1 aa rep 1 dd rep 1 kk rep 2 bb sad 2 ss sad 3 ee dam File 2 1 apple fruit 2 mango tree 3 lilly flower output: 1 aaple fruit aa,dd,kk rep (7 Replies)
Discussion started by: empyrean
7 Replies

3. UNIX for Advanced & Expert Users

Load average in squeeze too low

sorry, not yelling, its just copied from a bug report and this prob is driving me crazy... ran stress --cpu 4 on a xeon and no problem the load went up to 4, but running mysql server for example the load is almost 0.00, where the same mysql server with same throughput gets 0.8 on another lenny... (15 Replies)
Discussion started by: suffeks
15 Replies

4. UNIX Desktop Questions & Answers

How to squeeze multiple pipe character '|' into single '|' using sed?

Hi, I am trying to convert multiple Unix pipe symbol or bar into single |. I have tried with the following sed statements, but, no success :(. I need it using sed only echo "sed 's/\|\+/\|/g' sed 's/*/\|/' sed 's/\|*/|/' sed -r 's/\|+/\|/' However, the below awk code is working fine.... (4 Replies)
Discussion started by: royalibrahim
4 Replies

5. UNIX for Dummies Questions & Answers

Creating a condensed table from a pre-existing table in putty

Hello, I'm working with putty on Windows 7 professional and I'd like to know if there's a way to gather specific lines from a pre-existing table and make a new table with that information. More specifically, I'd like the program to look at a specific column, say column N, and see if any of the... (5 Replies)
Discussion started by: Deedee393
5 Replies

6. Shell Programming and Scripting

Help me please: UNIX command to extract substring not squeeze spaces

Hi experts, Please help me!... I have a string " test1 test2 test3 ". There are two spaces before "test1"; There are four spaces between "test1" and "test2"; there are two spaces between "test2 and "test3". I want to extract a substring "2 test3" using positions. Below is my test... (5 Replies)
Discussion started by: sophiez16
5 Replies

7. Debian

VPN service fails after update applied in Debian Squeeze

Hello everybody, I used to log in to my office via PPTP VPN, but on last October 5th I updated my installed Debian Squeeze and it caused my VPN service (client-side) to fail. After this upgrade I'm unable to log in to the VPN server. Here follows the log: #tail -f /var/log/messages Plugin... (0 Replies)
Discussion started by: r4ym4r
0 Replies

8. Shell Programming and Scripting

Filtering duplicates based on lookup table and rules

please help solving the following. I have access to redhat linux cluster having 32gigs of ram. I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates. Lookup... (4 Replies)
Discussion started by: ritakadm
4 Replies

9. Web Development

Getting Rid of Annoying Bootstrap Table Borders and Wayward Table Lines

Bootstrap is great; but we have had some issues with Bootstrapped <tables> (and legacy <fieldset> elements) showing annoying, wayward lines. I solved that problem today with this simple jQuery in the footer: <script> $(function(){ $('tr, td, fieldset,... (0 Replies)
Discussion started by: Neo
0 Replies

10. UNIX for Beginners Questions & Answers

Remove duplicates in a dataframe (table) keeping all the different cells of just one of the columns

Hello all, I need to filter a dataframe composed of several columns of data to remove the duplicates according to one of the columns. I did it with pandas. In the main time, I need that the last column that contains all different data ( not redundant) is conserved in the output like this: A ... (5 Replies)
Discussion started by: pedro88
5 Replies
COMBINE(1)																COMBINE(1)

NAME
combine - combine sets of lines from two files using boolean operations SYNOPSIS
combine file1 and file2 combine file1 not file2 combine file1 or file2 combine file1 xor file2 _ file1 and file2 _ _ file1 not file2 _ _ file1 or file2 _ _ file1 xor file2 _ DESCRIPTION
combine combines the lines in two files. Depending on the boolean operation specified, the contents will be combined in different ways: and Outputs lines that are in file1 if they are also present in file2. not Outputs lines that are in file1 but not in file2. or Outputs lines that are in file1 or file2. xor Outputs lines that are in either file1 or file2, but not in both files. "-" can be specified for either file to read stdin for that file. The input files need not be sorted, and the lines are output in the order they occur in file1 (followed by the order they occur in file2 for the two "or" operations). Bear in mind that this means that the operations are not commutative; "a and b" will not necessarily be the same as "b and a". To obtain commutative behavior sort and uniq the result. Note that this program can be installed as "_" to allow for the syntactic sugar shown in the latter half of the synopsis (similar to the test/[ command). It is not currently installed as "_" by default, but you can alias it to that if you like. SEE ALSO
join(1) AUTHOR
Copyright 2006 by Joey Hess <joey@kitenet.net> Licensed under the GNU GPL. moreutils 2012-04-09 COMBINE(1)
All times are GMT -4. The time now is 02:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy