Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Shell script to apply functions to multiple columns dynamically
Top Forums Shell Programming and Scripting Shell script to apply functions to multiple columns dynamically Post 303025752 by mkathi on Sunday 11th of November 2018 12:47:25 PM
Old 11-11-2018
Quote:
Originally Posted by RudiC
Please be aware that the md5sum of '10,abc' will NEVER be 73aca49763216fb96bbc2acef7b60afb as it is case sensitive.
Looks like you want a comma included. Try
Code:
awk -F\| '
NR == 1         {for (i=1; i<=NF; i++) if ("," MCOL "," ~ "," $i ",") COL[++CNT] = i
                 print $0, "HASHED COLUMNS", "HASHVALUE"
                 next
                }
                {TMP = ""
                 for (i=1; i<=CNT; i++) TMP = TMP "," $(COL[i])
                 ("echo -n " substr (TMP, 2) " | md5sum") | getline MD5
                 sub (/ *-/, "", MD5)
                 print $0, MCOL, MD5
                }
' OFS="|" MCOL="ID,NAME" file
ID|NAME|AGE|GENDER|HASHED COLUMNS|HASHVALUE
10|ABC|30|M|ID,NAME|73aca49763216fb96bbc2acef7b60afb
20|DEF|20|F|ID,NAME|9d6555fe65eb60b2f7d9174b56f667f5

Yes thank-you, you are right I should be more careful with the cases but the output now is as expected
Also can the last line MCOL="ID,NAME" be parameterized example

Code:
mcols=$1
filename=$2
awk -F\| '
NR == 1         {for (i=1; i<=NF; i++) if ("," MCOL "," ~ "," $i ",") COL[++CNT] = i
                 print $0, "HASHED COLUMNS", "HASHVALUE"
                 next
                }
                {TMP = ""
                 for (i=1; i<=CNT; i++) TMP = TMP "," $(COL[i])
                 ("echo -n " substr (TMP, 2) " | md5sum") | getline MD5
                 sub (/ *-/, "", MD5)
                 print $0, MCOL, MD5
                }
' OFS="|" MCOL="$(mcols)" file

and call the script like sh myscript.sh "ID,NAME"(I think it is a silly question but never the less I am asking)

Also I am trying to understand your code line by line could you point me how debug this code I am not asking you to explain line by line but can you pls point me towards the direction for me to better understand the code.

Thanks.
 

10 More Discussions You Might Find Interesting

1. Programming

reallocating structures dynamically in functions

I've recently started using structures, but I am having problems in allocating the structure dynamically. In the code below if i allocate the structure in the main program it works fine, and i get the expected output. However if i use the function rper below to increase the size of the structure i... (0 Replies)
Discussion started by: cezaryn
0 Replies

2. Shell Programming and Scripting

Managing dynamically multiple shell

I want to launch some shell scripts. I would have the possibility to change the number of shell scripts launched dynamically by modifying a variable, or a configuration file. For example, I start to launch 4 scripts at the same time, and after that, by modifying a variable, 6 scripts are... (0 Replies)
Discussion started by: gonzo38
0 Replies

3. Shell Programming and Scripting

Shell script dynamically case in VAR

Hallo, I am working on a kdialog. This shall be able to load the required commands from a .conf file. First step runs good by loading the entries (selectabel entries) in a variable: MIRRORSELECT=$(kdialog --radiolist "Select your nearest mirror" $VAR1) The kdialog is accordingly correct... (2 Replies)
Discussion started by: ACTGADE
2 Replies

4. Shell Programming and Scripting

how to create the files dynamically in c shell script

how can i CREATE a txt file dynamically in c shell: for instance: #! /bin/csh for each i (*) cat>file$i.txt for each j do .... (1 Reply)
Discussion started by: jdsignature88
1 Replies

5. Shell Programming and Scripting

How to run multiple functions in Background in UNIX Shell Scripting?

Hi, I am using ksh , i have requirement to run 4 functions in background , 4 functions call are available in a case that case is also in function, i need to execute 1st function it should run in background and return to case and next i will call 2nd function it should run in background and... (8 Replies)
Discussion started by: karthikram
8 Replies

6. Shell Programming and Scripting

Creating IN list in PLSQL script dynamically by using shell script

Hi all, I have a PLSQL script which has a IN list where it takes some ids as input. For example SELECT * FROM EMPLOYEE WHERE EMPLOYEE_ID IN (comma separated list ) I want to run this quest inside a shell script but I would like to prepare the IN list dynamically where the employee ids... (1 Reply)
Discussion started by: LoneRanger
1 Replies

7. Shell Programming and Scripting

Read Two Columns - Apply Condition on Six other columns

Hello All, Here is my input univ1 chr1 100 200 - GeneA 500 1 0 0.1 0.2 0.3 0.4 0.5 univ1 chr1 100 200 - GeneA 600 1 0 0.0 0.0 0.0 0.0 0.1 univ1 chr1 100 200 - GeneA 700 1 0 0.4 0.4 ... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

8. Shell Programming and Scripting

How can I apply 'date' command to specific columns, in a BASH script?

Hi everyone, I have a situation in which I have multiple (3 at last count) date columns in a CSV file (, delim), which need to be changed from: January 1 2017 (note, no comma after day) to: YYYY-MM-DD So far, I am able to convert a date using: date --date="January 12, 1990" +%Y-%m-%d ... (7 Replies)
Discussion started by: richardsantink
7 Replies

9. UNIX for Beginners Questions & Answers

How to apply the update statement in multiple servers on multiple dbs at a time .?

Hi , Can any please help the below requirement on all multiple servers and multiple dbs. update configuration set value='yes' ;1) the above statement apply on 31 Databases at a time on different Ip address eg : 10.104.1.12 (unix ip address ) the above ip box contains 4 db's eg : db... (2 Replies)
Discussion started by: venkat918
2 Replies

10. UNIX for Beginners Questions & Answers

How to print multiple required columns dynamically in a file using the header name?

Hi All, i am trying to print required multiple columns dynamically from a fie. But i am able to print only one column at a time. i am new to shell script, please help me on this issue. i am using below script awk -v COLT=$1 ' NR==1 { for (i=1; i<=NF; i++) { ... (2 Replies)
Discussion started by: balu1234
2 Replies

LEARN ABOUT CENTOS

gst-md5sum

gst-md5sum-0.8(1)						   User Commands						 gst-md5sum-0.8(1)

NAME

       gst-md5sum-0.8, gst-md5sum - get an md5sum of a GStreamer pipeline through md5sink

SYNOPSIS

       gst-md5sum-0.8 [--verbose] [gst-std-options] [partial-pipeline-description]

       gst-md5sum [--verbose] [gst-std-options] [partial-pipeline-description] [--gst-list-mm] [--gst-mm=string] [--print]

DESCRIPTION

       gst-md5sum-0.8 generates MD5 checksums of the data generated by a GStreamer pipeline.

       In theory, both of the following commands should print out the same checksum:

       gst-md5sum-0.8 filesrc location=music.mp3

       md5sum music.mp3

       If  the	pipeline  contains  an md5sink element, gst-md5sum-0.8 queries it for the md5sum at the end of pipeline iteration. If the pipeline
       does not contain an md5sink element, gst-md5sum-0.8 automatically connects an md5sink to the right-hand side of the given pipeline.

       See gst-launch(1) or the GStreamer documentation for more information on how to create a partial-pipeline-description.

       gst-md5sum is a wrapper script that runs the latest installed version of gst-md5sum-X.X. For  example,  if  both  gst-md5sum-0.7  and  gst-
       md5sum-0.8 are installed on your system, gst-md5sum runs gst-md5sum-0.8.

OPTIONS

       The following options are supported by gst-md5sum-0.8 and gst-md5sum:

       --verbose       Output verbose information. You can also use -v to specify this option.

       gst-std-options Standard options available for use with most GStreamer applications.  See gst-std-options(5) for more information.

       The following options are supported by gst-md5sum only:

       --gst-list-mm   List found major/minor versions. This option displays the versions that are available.

       --gst-mm=string Force  major/minor  version.  This  option  enables you to specify a specific version to run, if you do not want to run the
		       default version.

       --print	       Print wrapped command line. This option displays the command that will be run, and then runs the command.

OPERANDS

       The following operands are supported:

       partial-pipelinePartialppipeline description.

EXAMPLES

       Example 1: Running the Wrapper Script and Displaying the Command Name

       example% gst-md5sum --print
       /usr/bin/gst-md5sum-0.8

FILES

       The following files are used by this application:

       /usr/bin/gst-md5sum-0.8 Executable for GNOME help browser

       /usr/bin/gst-md5sum     Wrapper script that runs the latest installed version of gst-md5sum-X.X

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWgnome-media		   |
       +-----------------------------+-----------------------------+
       |Interface stability	     |External			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       gst-complete(1), gst-compprep(1), gst-feedback(1), gst-inspect(1),  gst-launch(1),  gst-launch-ext(1),  gst-register(1),  gst-thumbnail(1),
       gst-typefind(1), gst-xmlinspect(1), gst-xmllaunch(1), gstreamer-properties(1), libgstreamer-0.8(3), libgstgetbits(3), gst-std-options(5)

NOTES

       Original man page written by the GStreamer team at http://gstreamer.net/.

       Updated by Brian Cameron, Sun Microsystems Inc., 2004.

SunOS 5.10							    14 Oct 2004 						 gst-md5sum-0.8(1)
All times are GMT -4. The time now is 10:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy