Sponsored Content
Top Forums Shell Programming and Scripting awk script to append suffix to column when column has duplicated values Post 303017089 by as7951 on Wednesday 9th of May 2018 03:39:12 AM
Old 05-09-2018
awk script to append suffix to column when column has duplicated values

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2

Scenario 1
i need to do below changes only when column1 is CR and column3 has duplicates rows/values. This inputfile can contain 100 of this duplicated rows of column3 Value in column 3 to be suffixe with sequence starting from "a" followed by CR.
if we have appended suffix till a to z with CR like(aCR, bCR, cCR till zCR), then next suffix will be aaCR, abCR, acCR ans so on for column3

Input File

Code:
a||c
CR||2157237496
CR||2157237496
CR||2157237496
INV||2157237496
RNV||3457634589

Output File

Code:
a||c
CR||2157237496aCR
CR||2157237496bCR
CR||2157237496cCR
INV||2157237496
RNV||3457634589

Scenario 2 Required separate code for this scenario Below is for different file i need to do below changes only when column1 is DR and column3 has duplicates rows/values.
Value in column 3 to be suffixe with sequence starting from "a" except for first document followed by DR.
if we have appended suffix till a to z with DR like(aDR, bDR, cDR till zDR), then next suffix will be aaDR, abDR, acDR ans so on for column3

Input File

Code:
a||c
DR||3770022521
DR||3770022521
DR||3770022521
INV||9876543738

output file

a||c
Code:
DR||3770022521
DR||3770022521aDR
DR||3770022521bDR
INV||9876543738

I tried below code, it is giving me output for scenario 2 but not able to append sequence number to column3. I am able to suffix only "a" which is static. Here need to consider column1 equal to CR for scenario 1 and DR for scenario2 as well(that im unable to do)

Code:
awk -F"|" -v OFS="|" '{if(++a[$3]>1)$3=$3"a"}1' d1.txt

Code output :

Code:
a||c
CR||2157237496
CR||2157237496a
CR||2157237496a
INV||2157237496a
RNV||3457634589

Please help me to get required output for both scenario 1 and scenario 2 and need separate code for both scenario 1 and scenario 2
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

for each different entry in column 1 extract maximum values from column 2 in unix/awk

Hello, I have 2 columns (1st column has multiple entries but the corresponding values in the column 2 may be the same or different.) however I want to extract unique values for each entry in column 1 by assigning the max value from column 2 SDF4 -0.211654 SDF4 0.978068 ... (1 Reply)
Discussion started by: Diya123
1 Replies

2. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

3. UNIX for Dummies Questions & Answers

append column and row header to a file in awk script.

Hi! Is there a way to append column and row header to a file in awk script. For example if I have Jane F 39 manager Carlos M 40 system administrator Sam F 20 programmer and I want it to be # name gend age occup 1 Jane F 39 manager 2 Carlos M ... (4 Replies)
Discussion started by: FUTURE_EINSTEIN
4 Replies

4. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

5. Shell Programming and Scripting

Match value in column and append file with new values

Hi, I need help to match two files based on two columns. file_1 ID AA An Ca Ele Pro Su Ot Tra g13950 No No Yes No Yes Yes Yes Yes g05760 Yes No No No No Yes Yes Yes g12640 No No No No No No No No k17720 No Yes No No No No No Yes g05640 Yes Yes Yes No No Yes Yes Yes file_2 ... (8 Replies)
Discussion started by: redse171
8 Replies

6. Shell Programming and Scripting

How to delete 'duplicated' column values and make a delimited file too?

Hi, I have the following output from an Oracle SQL statement and I want to remove duplicated column values. I know it is possible using Oracle analytical/statistical functions but unfortunately I don't know how to use any of those. So now, I've gone to PLAN B using awk/sed maybe or any... (5 Replies)
Discussion started by: newbie_01
5 Replies

7. Shell Programming and Scripting

Trying to get an awk script to replace values in column

I'm trying to make an awk script to compare values I've set as var1, var2, and var3 earlier in the script to the values in the userinputted column of four text files called Node1.txt, Node2.txt, Node3.txt, and Node4.txt and then replace the values in that userinputted column with either ttt or gcc,... (8 Replies)
Discussion started by: Eric1
8 Replies

8. Shell Programming and Scripting

Matching column value from 2 different file using awk and append value from different column

Hi, I have 2 csv files. a.csv HUAWEI,20LMG011_DEKET_1296_RTN-980_IDU-1-11-ISV3-1(to LAMONGAN_M),East_Java,20LMG011_DEKET_1296_RTN-980_IDU-1,20LMG011,20LMG 027_1287_LAMONGAN_RTN980_IDU1,20LMG027,1+1(HSB),195.675,20LMG011-20LMG027,99.9995,202.6952012... (7 Replies)
Discussion started by: tententen
7 Replies

9. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

10. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies
Test::BDD::Cucumber::Manual::Tutorial(3pm)		User Contributed Perl Documentation		Test::BDD::Cucumber::Manual::Tutorial(3pm)

NAME
Test::BDD::Cucumber::Manual::Tutorial - Quick Start Guide VERSION
version 0.11 Introduction In this article we're going to jump straight in to using Test::BDD::Cucumber to build some simple tests for Digest, a core Perl module which provides message digests. We'll create a "features/" directory, and put our first test case in it, "features/basic.feature" in it. The contents of it are, in their entirity: # Somehow I don't see this replacing the other tests this module has... Feature: Simple tests of Digest.pm As a developer planning to use Digest.pm I want to test the basic functionality of Digest.pm In order to have confidence in it Background: Given a usable Digest class Scenario: Check MD5 Given a Digest MD5 object When I've added "foo bar baz" to the object And I've added "bat ban shan" to the object Then the hex output is "bcb56b3dd4674d5d7459c95e4c8a41d5" Then the base64 output is "1B2M2Y8AsgTpgAmY7PhCfg" Scenario: Check SHA-1 Given a Digest SHA-1 object When I've added "<data>" to the object Then the hex output is "<output>" Examples: | data | output | | foo | 0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33 | | bar | 62cdb7020ff920e5aa642c3d4066950dd1f01f4d | | baz | bbe960a25ea311d21d40669e93df2003ba9b90a2 | Scenario: MD5 longer data Given a Digest MD5 object When I've added the following to the object """ Here is a chunk of text that works a bit like a HereDoc. We'll split off indenting space from the lines in it up to the indentation of the first """ """ Then the hex output is "75ad9f578e43b863590fae52d5d19ce6" This is a complete test, and if you run pherkin against it, you will get sane output! It just doesn't do anything ... yet. In the "features/" we'll add a "step_definitions/" directory, and add our first (and again, only) step definitions "features/step_definitions/basic_steps.pl" file in it: #!perl use strict; use warnings; use Test::More; use Test::BDD::Cucumber::StepFile; use Method::Signatures; Given qr/a usable (S+) class/, func ($c) { use_ok( $1 ); }; Given qr/a Digest (S+) object/, func ($c) { my $object = Digest->new($1); ok( $object, "Object created" ); $c->stash->{'scenario'}->{'object'} = $object; }; When qr/I've added "(.+)" to the object/, func ($c) { $c->stash->{'scenario'}->{'object'}->add( $1 ); }; When "I've added the following to the object", func ($c) { $c->stash->{'scenario'}->{'object'}->add( $c->data ); }; Then qr/the (.+) output is "(.+)"/, func ($c) { my $method = {base64 => 'b64digest', 'hex' => 'hexdigest' }->{ $1 } || do { fail("Unknown output type $1"); return }; is( $c->stash->{'scenario'}->{'object'}->$method, $2 ); }; When you run pherkin or the Test::Builder-based test which does the same thing ("900_run_features.thttps://github.com/sheriff/test-bdd-cucumber-perl/blob/master/t/900_run_features.t" in t), we look for a "features/" directory, and search for step definitions files (matched by "*_steps.pl") and feature files (matched by "*.feature"). The step matchers (the code that starts with "Given", "When" and "Then") are all loaded, and then we execture the feature files one by one. Let's step through the feature file, and look at how it matches up to the step definitions file. Name and conditions of satisfaction Feature: Simple tests of Digest.pm As a developer planning to use Digest.pm I want to test the basic functionality of Digest.pm In order to have confidence in it The first non-comment line of your feature file is a description of what you intend to do. You need to start the name itself with the string "Feature:", and that should be the first line of your file, save comments (denoted by #). Anything after that before the next new-line are your conditions of satisfaction. These aren't parsed, they're treated as human-readable text, and by convention, they're a user story <http://en.wikipedia.org/wiki/User_story>. Background Background: Given a usable Digest class Next up, we have the Background section. The Background is a special kind of Scenario that doesn't have an explicit name, and should occur only once in your feature file. Its steps are run before the steps of every other scenario - the harnesses distributed with this distro won't display the Background section separately, they'll just subsume the steps in to the other scenarios. This is matched by: Given qr/a usable (S+) class/, func ($c) { use_ok( $1 ); }; "Given()" is a function exported by Test::BDD::Cucumber::StepFile that accepts two arguments: a regular expression (or a string when you don't need to do any smart matching) and a coderef. We use Method::Signatures to save some typing here, so note that written longer hand, the above might be written: Given( qr/a usable (S+) class/, sub { my $c = shift; use_ok( $1 ); }; ); If you're paying attention, you might notice that "use_ok" comes from Test::More. Each step is run, from a Test::Builder perspective, as its own distinct test file. This happens seamlessly, so you can use any Test::Builder-based testing tools in your step definitions without really worrying about it. There's some more detail in Test::BDD::Cucumber::Manual::Steps. The First Scenario... Scenario: Check MD5 Given a Digest MD5 object When I've added "foo bar baz" to the object And I've added "bat ban shan" to the object Then the hex output is "bcb56b3dd4674d5d7459c95e4c8a41d5" Then the base64 output is "1B2M2Y8AsgTpgAmY7PhCfg" The first scenario is delimited from the previous steps by a blank line, and it's called Check MD5. Scenarios are marked out using the "Scenario:" keyword, and just like the Background section before, it's a series of steps. These steps rely on the step before, which means we can examine the Test::Builder::StepContext object $c a little more closely. Given qr/a Digest (S+) object/, func ($c) { my $object = Digest->new($1); ok( $object, "Object created" ); $c->stash->{'scenario'}->{'object'} = $object; }; Creates a step definition. We create a new Digest object, and then use Test::More's "ok()" function to check that worked. We then put it in the stash for other steps to use. There are three stashes documented in Test::Builder::StepContext, "feature", "scenario" and "step". As you might expect, "feature" is available to all step definitions that are being executed as part of a feature, and "scenario" is available to all steps that are being executed as part of a feature. The context is the single argument that gets passed to each step, and it contains evertything that step should need to execute. We'll be looking at some of the methods you can call against it as we look at other steps, and you can find complete documentation for it here: Test::Builder::StepContext. This scenario also introduce several ways of starting a step, Given, When, and Then, as well as And. These are used to organize steps by type, with Given tending to describe setup steps, When describing the key actions that you're testing, and Then describing the outputs you're looking for. You can find more on this here: https://github.com/cucumber/cucumber/wiki/Given-When-Then <https://github.com/cucumber/cucumber/wiki/Given-When-Then>. A step definition you've declared with Given will not match a step starting with Then. You can use the keyword Step to declare general matching steps in your step definition files, although it's considered bad practice. Finally, the keywords And and But are simply replaced with the verb on the line before them. Scenario Outlines Scenario: Check SHA-1 Given a Digest SHA-1 object When I've added "<data>" to the object Then the hex output is "<output>" Examples: | data | output | | foo | 0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33 | | bar | 62cdb7020ff920e5aa642c3d4066950dd1f01f4d | | baz | bbe960a25ea311d21d40669e93df2003ba9b90a2 | The next scenario adds one of the three ways you can provide structured data to your steps, using placeholders and a table. This scenario is run three times, one for each table row, and with the " <placeholders" > being replaced by the appropriate row's column. These are called Scenario Outlines <https://github.com/cucumber/cucumber/wiki/Scenario-outlines>. Multiline Step Arguments Scenario: MD5 longer data Given a Digest MD5 object When I've added the following to the object """ Here is a chunk of text that works a bit like a HereDoc. We'll split off indenting space from the lines in it up to the indentation of the first """ """ Then the hex output is "75ad9f578e43b863590fae52d5d19ce6" While before we were looking at structured data on a Scenario level, we can also provide it on a Step level, in two ways. Firstly, we can provide multi-line strings, as above, using a feature that is syntactically similar to "pystring"s, and conceptually similar to HEREDOCs. The contents of the string will be available to the step definition via the "data()" method of the context: When "I've added the following to the object", func ($c) { $c->stash->{'scenario'}->{'object'}->add( $c->data ); }; While we don't have an example of it here, you can also provide tables to your steps, which will also be available via "data()": Scenario: Sort Employees Given a set of employees | name | wage | hair color | | Peter | 10,000 | brown | | John | 20,000 | blond | | Joan | 30,000 | green | You can find out more about these features in the Cucumber documentation here: https://github.com/cucumber/cucumber/wiki/Multiline-Step-Arguments <https://github.com/cucumber/cucumber/wiki/Multiline-Step-Arguments>. Next Steps... That's the tutorial done! You can find out more about writing steps in Test::BDD::Cucumber::Manual::Steps, the documentation for our simple command-line tool in App::pherkin, and how to integrate with Test::Builder in Test::BDD::Cucumber::Manual::Integration. AUTHOR
Peter Sergeant "pete@clueball.com" LICENSE
Copyright 2011, Peter Sergeant; Licensed under the same terms as Perl perl v5.14.2 2012-05-20 Test::BDD::Cucumber::Manual::Tutorial(3pm)
All times are GMT -4. The time now is 11:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy