Sponsored Content
Top Forums Shell Programming and Scripting Removing non-unique prefixed lines from a list Post 302886876 by chrissycc on Tuesday 4th of February 2014 06:03:46 AM
Old 02-04-2014
Removing non-unique prefixed lines from a list

Not quite sure how to explain what I need to do (or how to title the post!) so will try and show it!

Basically I have a list of 'modules' which takes the form seen below, where there can be a module name, module type and module version (a module may not have each of those and could in theory have more (subversion etc). Also version can be name or number). The list contains an entry for each name; name and type; and name, type version, so for example:

Code:
apache-ant
apache-ant/1.8.4
atlas
atlas/gcc
atlas/gcc/3.8.4
atlas/gcc/3.8.6
binutils
binutils/2.22
binutils/2.23
codeutils/
codeutils/testing
codeutils/working
codeutils/live/1.2

I'm only interested in the final unique entry for each module, so what I want to do is remove the non-unique branches of list, i.e. going from the above list to the list below:

Code:
apache-ant/1.8.4
atlas/gcc/3.8.4
atlas/gcc/3.8.6
binutils/2.22
binutils/2.23
codeutils/testing
codeutils/working
codeutils/live/1.2

I'm not even sure where to start with this, after disregarding rev, uniq etc, though no doubt I'm missing some simple solution!

Any help as always appreciated
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove All Lines Between Two Unique Lines

Hi all! Im wondering if its possible to remove all lines between two lines. Im working with a document like this: data1 data2 <Remove> data3 data4 </Remove> data5 data6 I need it to end up like this if that possible: data1 data2 data5 data6 There are multiple instances of... (2 Replies)
Discussion started by: Grizzly
2 Replies

2. Shell Programming and Scripting

Removing lines from one list from another list.

Hello, I was wondering if there was an easy way to take lines from a single-column list, and remove them from a second single-column list. For example, I want to remove the contents of list 1 from list 2. How would I do this? Contents of list 1: server1a server2b server3c server4a... (2 Replies)
Discussion started by: LinuxRacr
2 Replies

3. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

4. UNIX for Dummies Questions & Answers

Help removing multiple lines from one file with a list from a second file!

I am trying to remove the lines listed in example File A from File B to achieve File C. Both files are much larger than the examples below. (File B has up to 6,000 lines). I have searched the forums and I have not been able to find an answer to this particular question. I tried grep -v -f... (2 Replies)
Discussion started by: pezziza
2 Replies

5. Shell Programming and Scripting

Transpose lines from individual blocks to unique lines

Hello to all, happy new year 2013! May somebody could help me, is about a very similar problem to the problem I've posted here where the member rdrtx1 and bipinajith helped me a lot. https://www.unix.com/shell-programming-scripting/211147-map-values-blocks-single-line-2.html It is very... (3 Replies)
Discussion started by: Ophiuchus
3 Replies

6. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

7. Emergency UNIX and Linux Support

Removing ONLY UNIQUE blank spaces with sed?

Hi, I have data that looks similar to this: In which the sentences are written horizontally and the beginning of a sentence is indicated by a 1 in the first column and the number increments until the last item of the sentence. The end of the sentence and the beginning of the next is then indicate... (1 Reply)
Discussion started by: owwow14
1 Replies

8. UNIX for Dummies Questions & Answers

Removing PATTERN from txt without removing lines and general text formatting

Hi Everybody! First post! Totally noobie. I'm using the terminal to read a poorly formatted book. The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this. I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies

9. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

10. Shell Programming and Scripting

Joining broken lines and removing empty lines

Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char. thanks in advance Source:- 2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver;... (7 Replies)
Discussion started by: Jackceasar123
7 Replies
cxref-cpp(1)						      General Commands Manual						      cxref-cpp(1)

NAME
cxref-cpp - A modified C preprocessor to use with cxref. SYNOPSIS
cxref-cpp ... DESCRIPTION
To improve the output that is available for the source code for cross-referencing a modified version of the GNU CPP v2.7.2 is supplied (named cxref-cpp). This modified C preprocessor allows for a finer control over some features of the preprocessing that are not important for a compiler. In a standard preprocessor, the preprocessor directives are intended for use only by the preprocessor, so passing the information through is not important. With cxref-cpp, there are some features that are different to the standard GNU CPP: Compared to gcc versions earlier than version 2.8.0 there is an extra option that will output the #include lines from the source file. In version 2.8.0 and later this option is present. Comments trailing a #include or a #define are not preserved by all versions of gcc even if the -C option is used. This is not important while compiling, but is useful for documenting. The cxref-cpp program will take on the personality of the installed version of gcc so that the gcc header files can be parsed. This means that it includes the same default include directory paths and macro definitions. The file that contains these definitions is called cxref-cpp.defines and is installed by the cxref-cpp-configure program or specified by the -cxref-cpp-defines command line option. OPTIONS
The same as for gcc, apart from '-cxref-cpp-defines' described above. SEE ALSO
cxref(1), cxref-cpp-configure(1), gcc(1) May 9, 2004 cxref-cpp(1)
All times are GMT -4. The time now is 01:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy