Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Reducing text file using similar lines
Top Forums Shell Programming and Scripting Reducing text file using similar lines Post 302781841 by Yoda on Sunday 17th of March 2013 10:46:26 PM
Old 03-17-2013
Please do not bump up questions if they are not answered promptly.

Here is an awk program based on some assumptions:
Code:
awk -F';' ' {
                if ( F1[i] == $1 && i > 1 )
                {
                        F2[i] += $2
                        F3[i] += $3
                        gsub (/\[|\]/, X, $4)
                        gsub (/\[|\]/, X, $5)
                        if ( F4[i] == "" )
                                F4[i] = $4
                        else if ( $4 != "" )
                        {
                                F4[i] = F4[i] "," $4
                                n = split (F4[i], V, ",")
                                F4[i] = ""
                                for ( k = 1; k <= n; k++ )
                                {
                                        if ( k > 1 && V[k] !~ V[k-1] )
                                                F4[i] = F4[i] "," V[k]
                                        else if ( k == 1 )
                                                F4[i] = V[k]
                                }
                        }
                        if ( F5[i] == "" )
                                F5[i] = $5
                        else if ( $5 != "" )
                        {
                                F5[i] = F5[i] "," $5
                                n = split (F5[i], V, ",")
                                F5[i] = ""
                                for ( k = 1; k <= n; k++ )
                                {
                                        if ( k > 1 && V[k] !~ V[k-1] )
                                                F5[i] = F5[i] "," V[k]
                                        else if ( k == 1 )
                                                F5[i] = V[k]
                                }
                        }
                }
                else
                {
                        ++i
                        F1[i] = $1
                        F2[i] = $2
                        F3[i] = $3
                        gsub (/\[|\]/, X, $4)
                        gsub (/\[|\]/, X, $5)
                        F4[i] = $4
                        F5[i] = $5
                }
} END {
                for ( j = 1; j <= i; j++ )
                        print F1[j], F2[j], F3[j], "[" F4[j] "]", "[" F5[j] "]"
} ' OFS=';' file

This User Gave Thanks to Yoda For This Post:
Don Cragun
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to sort a file and then print similar lines once

Hi! I have a trouble with the sort and the uniq. I know I have to use them, I just have trouble with putting them in the right order. I have a text file with unsorted lines (each line has a few words, the first word in the line is a number.). I need to sort this file in order to be... (6 Replies)
Discussion started by: shira
6 Replies

2. Infrastructure Monitoring

Remove Similar Lines from a File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead... (4 Replies)
Discussion started by: Nysif Steve
4 Replies

3. Shell Programming and Scripting

Counting similar lines from file UNIX

I have a file which contains data as below: nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1wqe... (6 Replies)
Discussion started by: mohsin.quazi
6 Replies

4. Shell Programming and Scripting

remove one of each similar lines in a file

Hello folks I have a question for you gurus of sed or grep (maybe awk, but I would prefer the first two) I have a file (f1) that says: (actually, these are not numbers but md5sum, but for simplicity, let's assume these numbers.) 1 2 3 4 5And I have a file (f2) that says 1|a 1|b 1|c 2|d... (3 Replies)
Discussion started by: tukuyomi
3 Replies

5. UNIX for Dummies Questions & Answers

merge lines within a file that start with a similar pattern

Hello! i have a text file.. which contains the data as follows i want to merge the declarations lines pertaining to one datatype in to a single line as follows i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies

6. Shell Programming and Scripting

Reducing file lines in awk

Hi, Here i have to check first record $3 $4 with second record $1 $2 respectively. If match found, then check first record $2 == second record $4 , if it equals , then reduce two records to single record like as desired output. Input_file 1 1 2 1 2 1 3 1 3 1 4 1 3 1 3 2 desired... (3 Replies)
Discussion started by: vasanth.vadalur
3 Replies

7. UNIX for Dummies Questions & Answers

Matching and reporting near-similar lines in a file

Hi, I have a file with the lines as below: C_10_A05_T7 C_10_A06_SP6 C_10_B05_SP6 C_10_B05_T7 C_10_B01_SP6 C_10_B01_T7 C_12_G07_SP6 C_12_G11_SP6 C_12_G11_T7 C_2_H18_T7 C_2_I02_SP6 C_2_I02_T7 C_2_I13_SP6 C_2_I17_SP6 The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies

8. Shell Programming and Scripting

extracting lines from a file with similar first name

consider i have two files cat onlyviews1.sql CREATE VIEW V11 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V22 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V33 AS (10 Replies)
Discussion started by: vivek d r
10 Replies

9. Shell Programming and Scripting

removing lines with similar values from file

Hello, got a file with this structure: 33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26 33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27 33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26 33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies

10. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

LEARN ABOUT DEBIAN

freemind

FREEMIND(1)						      General Commands Manual						       FREEMIND(1)

NAME

       freemind -- A Java program for creating and viewing mindmaps.

SYNOPSIS

       freemind [no_options]

DESCRIPTION

       This manual page documents briefly the freemind command.

       This  manual  page was written for the Debian distribution because the original program does not have a manual page.  Instead, it has docu-
       mentation as a mindmap (see Help menu in FreeMind).

       freemind is a program that allows to create and view so-called mindmaps, as well as export them to HTML or  different  image  formats.	It
       also has a plugin concept to extend its functionality.

OPTIONS

       This program does not have any option. Just start it!

ENVIRONMENT VARIABLES

       DEBUG	 If   this  environment  variable  is set, freemind gives more information on how it finds its different elements, as well as some
		 other information.

       JAVACMD	 Default value for the java virtual machine (the java command).

       JAVA_BINDIR
		 If $JAVACMD is not set, the java virtual machine is searched under $JAVA_BINDIR/java.

       JAVA_HOME If $JAVACMD and $JAVA_BINDIR are not set, the java virtual machine is searched under $JAVA_HOME/bin/java.

       PATH	 If all $JAVA... environment variables fail, the java virtual machine command (java) is searched in the $PATH (with which).

       CLASSPATH	ADD_JARS
		 The $CLASSPATH variable also lets one specify additional jars, which is good, if you want to add a new Look&Feel jar  (the  motif
		 one is so ugly...). Alternatively, the content of $ADD_JARS will be prepended to $CLASSPATH.

       FREEMIND_BASE_DIR
		 Directory where FreeMind searches for its plugins and libraries

FILES

       /etc/freemind/freemindrc        ~/.freemind/freemindrc
		 freemind  tries to read its environment variables from /etc/freemind/freemindrc and ~/.freemind/freemindrc, so that specific set-
		 tings are possible at machine and user level.

       /usr/bin/java
		 If all searches through environment variables fail, the java virtual machine is expected to be /usr/bin/java;	if  not,  freemind
		 exits with an error.

SEE ALSO

       The  online  documentation under http://freemind.sf.net/, especially the Linux related one under the Wiki page link http://freemind.source-
       forge.net/wiki/index.php/FreeMind_on_Linux.

AUTHOR

       This manual page was written by Eric Lavarde deb@zorglub.s.bawue.de for the Debian system (but may  be  used  by  others).   Permission	is
       granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version pub-
       lished by the Free Software Foundation.

       On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL.

																       FREEMIND(1)
All times are GMT -4. The time now is 11:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy