Sponsored Content
Top Forums Shell Programming and Scripting Reducing text file using similar lines Post 302781841 by Yoda on Sunday 17th of March 2013 10:46:26 PM
Old 03-17-2013
Please do not bump up questions if they are not answered promptly.

Here is an awk program based on some assumptions:
Code:
awk -F';' ' {
                if ( F1[i] == $1 && i > 1 )
                {
                        F2[i] += $2
                        F3[i] += $3
                        gsub (/\[|\]/, X, $4)
                        gsub (/\[|\]/, X, $5)
                        if ( F4[i] == "" )
                                F4[i] = $4
                        else if ( $4 != "" )
                        {
                                F4[i] = F4[i] "," $4
                                n = split (F4[i], V, ",")
                                F4[i] = ""
                                for ( k = 1; k <= n; k++ )
                                {
                                        if ( k > 1 && V[k] !~ V[k-1] )
                                                F4[i] = F4[i] "," V[k]
                                        else if ( k == 1 )
                                                F4[i] = V[k]
                                }
                        }
                        if ( F5[i] == "" )
                                F5[i] = $5
                        else if ( $5 != "" )
                        {
                                F5[i] = F5[i] "," $5
                                n = split (F5[i], V, ",")
                                F5[i] = ""
                                for ( k = 1; k <= n; k++ )
                                {
                                        if ( k > 1 && V[k] !~ V[k-1] )
                                                F5[i] = F5[i] "," V[k]
                                        else if ( k == 1 )
                                                F5[i] = V[k]
                                }
                        }
                }
                else
                {
                        ++i
                        F1[i] = $1
                        F2[i] = $2
                        F3[i] = $3
                        gsub (/\[|\]/, X, $4)
                        gsub (/\[|\]/, X, $5)
                        F4[i] = $4
                        F5[i] = $5
                }
} END {
                for ( j = 1; j <= i; j++ )
                        print F1[j], F2[j], F3[j], "[" F4[j] "]", "[" F5[j] "]"
} ' OFS=';' file

This User Gave Thanks to Yoda For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to sort a file and then print similar lines once

Hi! I have a trouble with the sort and the uniq. I know I have to use them, I just have trouble with putting them in the right order. I have a text file with unsorted lines (each line has a few words, the first word in the line is a number.). I need to sort this file in order to be... (6 Replies)
Discussion started by: shira
6 Replies

2. Infrastructure Monitoring

Remove Similar Lines from a File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead... (4 Replies)
Discussion started by: Nysif Steve
4 Replies

3. Shell Programming and Scripting

Counting similar lines from file UNIX

I have a file which contains data as below: nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1wqe... (6 Replies)
Discussion started by: mohsin.quazi
6 Replies

4. Shell Programming and Scripting

remove one of each similar lines in a file

Hello folks I have a question for you gurus of sed or grep (maybe awk, but I would prefer the first two) I have a file (f1) that says: (actually, these are not numbers but md5sum, but for simplicity, let's assume these numbers.) 1 2 3 4 5And I have a file (f2) that says 1|a 1|b 1|c 2|d... (3 Replies)
Discussion started by: tukuyomi
3 Replies

5. UNIX for Dummies Questions & Answers

merge lines within a file that start with a similar pattern

Hello! i have a text file.. which contains the data as follows i want to merge the declarations lines pertaining to one datatype in to a single line as follows i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies

6. Shell Programming and Scripting

Reducing file lines in awk

Hi, Here i have to check first record $3 $4 with second record $1 $2 respectively. If match found, then check first record $2 == second record $4 , if it equals , then reduce two records to single record like as desired output. Input_file 1 1 2 1 2 1 3 1 3 1 4 1 3 1 3 2 desired... (3 Replies)
Discussion started by: vasanth.vadalur
3 Replies

7. UNIX for Dummies Questions & Answers

Matching and reporting near-similar lines in a file

Hi, I have a file with the lines as below: C_10_A05_T7 C_10_A06_SP6 C_10_B05_SP6 C_10_B05_T7 C_10_B01_SP6 C_10_B01_T7 C_12_G07_SP6 C_12_G11_SP6 C_12_G11_T7 C_2_H18_T7 C_2_I02_SP6 C_2_I02_T7 C_2_I13_SP6 C_2_I17_SP6 The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies

8. Shell Programming and Scripting

extracting lines from a file with similar first name

consider i have two files cat onlyviews1.sql CREATE VIEW V11 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V22 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V33 AS (10 Replies)
Discussion started by: vivek d r
10 Replies

9. Shell Programming and Scripting

removing lines with similar values from file

Hello, got a file with this structure: 33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26 33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27 33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26 33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies

10. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
java-functions(7)						RPM Java packaging						 java-functions(7)

NAME
java-functions - Functions library for Java applications. Written for the JPackage Project <http://www.jpackage.org/>: SYNOPSIS
set_jvm() Set the java virtual machine. Use a JAVA_HOME if defined, or try to find it from java command set_classpath() Set the classpath - this functions requires a valid JAVA_HOME, JAVACMD, and JAVA_LIBDIR. JARs could be specified as an argument, or via ADDITIONAL_JARS variable. set_javacmd() set the JAVACMD variable. Options should be passed via JAVACMD_OPTS variable. set_flags() set FLAGS variable. They could be specified as an argument, or via ADDITIONAL_FLAGS variable. set_options() set OPTIONS variable. They could be specified as an argument, or via ADDITIONAL_OPTIONS variable. run() run the application. It executed a following command. If VERBOSE is defined, then it prints the command to stdout. exec $JAVACMD $FLAGS -classpath $CLASSPATH $OPTIONS $MAIN_CLASS "$@" set_jvm_dirs() set JVM-related directories (JVM_LIBDIR, JAVA_VERSION, JAVAVER_LIBDIR and JAVAVER_JNIDIR variables). Requires a correct JAVA_LIBDIR, JAVA_HOME and JAVA_CMD. link_jar_repository() links a jar repository. Options could be found in build-jar-repository(1) find_jar() finds a specific extention (jar or directory). Requires a correct JAVA_LIBDIR, JAVAVER_LIBDIR and JVM_LIBDIR. Used by find-jar(1) com- mand. do_find_jar() core routine used by find_jar() check_java_env() checks java environment - the JAVA_HOME, JAVACMD, JAVA_LIBDIR, JNI_LIBDIR variables. DESCRIPTION
This is a library of generic shell functions which should be used on jpackage.org compatible distributions. FILES
/use/share/java-utils/java-functions shell script functions library for Java applications /etc/java/java.conf system-wide Java configuration file ~/.java/java.conf user's Java configuration SEE ALSO
Regular Manual Pages build-jar-repository(1) find-jar(1) java.conf(5) jpackage-utils(7) AUTHORS
Guillaume Rousse <guillomovitch@sourceforge.net> Ville Skytta <scop at jpackage.org> David Walluck <david@jpackage.org> Nicolas Mailhot <Nicolas.Mailhot at laPoste.net> REPORTING BUGS
Report bugs using JPackage Bugzilla (http://www.jpackage.org/bugzilla/) jpackage-utils 1.7.5 February 2009 java-functions(7)
All times are GMT -4. The time now is 08:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy