Print all with along with stripped one column string


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print all with along with stripped one column string
# 1  
Old 09-23-2018
Print all with along with stripped one column string

Hi,

My output data from software tool is like

Code:
1234 84 test1 file:about user_detials(bankUser) :jpeg 10 40 xys
1356  2 branches dir:list of files(dirlisting:1) :directory 20 80 abc

and want this to be printed as below. where first and last 3 columns along with the string which is in the '()' brackets only needed.

Code:
1234 84 test1 bankUser 10 40 xys
1356  2 branches dirlisting:1  20 80 abc

------ Post updated at 11:45 AM ------

Also the column for string between brackets() are dynamic
Moderator's Comments:
Mod Comment As you have been told many times before... Please use CODE tags when displaying sample input, sample output, and code segments (as required by forum rules).

Last edited by Don Cragun; 09-23-2018 at 03:43 AM.. Reason: Add CODE tags again.
# 2  
Old 09-23-2018
Hello,

My initial thought was that this would be straightforward, and something like this one line of awk would work:

Code:
$ cat test.txt
1234 84 test1 file:about user_detials(bankUser) :jpeg 10 40 xys
1356 2 branches dir:list of files(dirlisting:1) :directory 20 80 abc
$ awk '{ gsub("\\(", " ") ; gsub("\\)", " ") ; print $1,$2,$3,$6,$8,$9,$10}' test.txt
1234 84 test1 bankUser 10 40 xys
1356 2 branches files :directory 20 80
$

So the idea is that we use awk, first replacing the brackets with spaces, and then just do a straightforward print of the fields we need.

But as you'll have seen, the output we get doesn't quite match what you want. It turns out that the reason for this is that the number of fields is inconsistent, and changes from one line of your test data to the other.

To be specific, the first line has nine fields, whereas the second line has ten:

Code:
$ head -1 test.txt | awk '{print NF}'
9
$ tail -1 test.txt | awk '{print NF}'
10

Now others with greater familiarity with awk than myself may well be able to still come up with an elegant one-line way to deal with this, but personally I would think that you have to make sure that the only spaces in your data are used strictly as field separators, or alternatively that any fields using spaces are at least quoted.

The following script would seem to do what you need:

Code:
$ cat script.sh 
#!/bin/bash
file=test.txt

while read -r line
do
        type1=`echo "$line" | awk -F\( '{print $2}' | awk -F\) '{print $1}'`
        num1=`echo "$line" | awk -F\) '{print $2}' | awk '{print $2}'`
        num2=`echo "$line" | awk -F\) '{print $2}' | awk '{print $3}'`
        text1=`echo "$line" | awk -F\) '{print $2}' | awk '{print $4}'`
        echo $line | awk -v type1=$type1 -v num1=$num1 -v num2=$num2 -v text1=$text1 '{print $1,$2,$3,type1,num1,num2,text1}'
done < "$file"

$ ./script.sh 
1234 84 test1 bankUser 10 40 xys
1356 2 branches dirlisting:1 20 80 abc
$

So we use what field separators we can always count on (the brackets, or so I'm hoping !) to strip out the fields we need one at a time, then we build up our awk line at the end.

Hope this helps. If not, then if you can let me know in what respects it fails we can take things from there. The main thing in general with this type of problem is to make your input as consistently- and reliably-formatted as you possibly can, always.
# 3  
Old 09-23-2018
Quote:
Originally Posted by ratheeshp
Hi,

My output data from software tool is like

Code:
1234 84 test1 file:about user_detials(bankUser) :jpeg 10 40 xys
1356  2 branches dir:list of files(dirlisting:1) :directory 20 80 abc

and want this to be printed as below. where first and last 3 columns along with the string which is in the '()' brackets only needed.

Code:
1234 84 test1 bankUser 10 40 xys
1356  2 branches dirlisting:1  20 80 abc

------ Post updated at 11:45 AM ------

Also the column for string between brackets() are dynamic
Moderator's Comments:
Mod Comment As you have been told many times before... Please use CODE tags when displaying sample input, sample output, and code segments (as required by forum rules).
What operating system are you using?

What shell are you using?

What have you tried to solve this problem on your own?

What is a "stripped one column string"?

Is it crucial that the varying numbers of spaces between the various columns in the input be duplicated in the output?

What, exactly, does "the column for string between brackets() are dynamic" mean? Will there ever be more than one pair of parentheses on an input line?
# 4  
Old 09-23-2018
What operating system are you using?
linux(centos6)
What shell are you using?
bash/sh
What have you tried to solve this problem on your own?
still in early stages of awk so nothing I got on this
What is a "stripped one column string"?
Sorry if that was confusing but what I was trying to say it to print the line with few columns which are consistent(first and last 3 columns) and along with that in between data to be cut/stripped and print data that is lying between () brackets.

Is it crucial that the varying numbers of spaces between the various columns in the input be duplicated in the output?
Yes, as I said above, I need first and last 3 columns and the string mentioned inside () only as rest all doesn't make any sense to me and it sometimes goes too lengthy..

What, exactly, does "the column for string between brackets() are dynamic" mean? Will there ever be more than one pair of parentheses on an input line?
using the dynamic means the number of columns between first and last 3 columns are unpredictable as it's kind of description for developers but meaningless long data to me.

It'll be helpful If I can have single line command so that I can pipe to the output and get the desired results.

Again sorry for the confusion if any.
# 5  
Old 09-23-2018
Moderator's Comments:
Mod Comment Thread title changed from "Print all with along with stipped one column string" to "Print all with along with stripped one column string" hoping that searching for related threads will have a better chance of working.
# 6  
Old 09-23-2018
Quote:
Originally Posted by ratheeshp
What operating system are you using?
linux(centos6)
What shell are you using?
bash/sh
What have you tried to solve this problem on your own?
still in early stages of awk so nothing I got on this
What is a "stripped one column string"?
Sorry if that was confusing but what I was trying to say it to print the line with few columns which are consistent(first and last 3 columns) and along with that in between data to be cut/stripped and print data that is lying between () brackets.

Is it crucial that the varying numbers of spaces between the various columns in the input be duplicated in the output?
Yes, as I said above, I need first and last 3 columns and the string mentioned inside () only as rest all doesn't make any sense to me and it sometimes goes too lengthy..

What, exactly, does "the column for string between brackets() are dynamic" mean? Will there ever be more than one pair of parentheses on an input line?
using the dynamic means the number of columns between first and last 3 columns are unpredictable as it's kind of description for developers but meaningless long data to me.

It'll be helpful If I can have single line command so that I can pipe to the output and get the desired results.

Again sorry for the confusion if any.
I fail to understand how changing varying numbers of <space>s between fields in your input lines (such as 1 <space> between most fields, but 2 <spaces> between the 1st and 2nd fields in the 2nd input line in your file) into single <space>s between fields would make it hard for you to identify the first three fields or the last three fields in the output you're trying to produce. This is especially confusing since the output that you say is needed for the 2nd line from your input file must create output that has 2 <space>s before the last three fields in the output when there was only 1 <space> in that position in your sample input file??? Insisting on this requirement does invalidate everything that drysdalk suggested. But, I don't understand how requiring additional <space>s in the output helps keep the output from being "too lengthy".

Please clearly explain why changing 2 <space>s between the 1st two fields in your sample input to 1 <space> between those fields in the output would make it hard for you to determine the data in those fields. And, please clearly explain why additional <space>s need to be added between some output fields when they were not present in your input file? (I.e., describe the logic that determines when additional <space>s need to be added!)

The number of lines in an awk script has absolutely no effect on whether or not you can pipe data into that script or pipe data out of that script. The only reason to insist on a one line awk script is to make it harder to read (especially for someone who is learning how to use awk or who is using a screen that doesn't wrap long lines of code). Our purpose on this site is to help people learn how to use the tools that are available to them on the systems they're using. Writing obfuscated or otherwise unreadable code does not help us achieve that goal.

Please either remove this requirement or make a MUCH more convincing argument as to why you need a one line solution!

P.S. And please answer the question I asked in post #3:
Quote:
Will there ever be more than one pair of parentheses on an input line?
If the answer to this question is "Yes" there might not be any way to achieve your goal.
# 7  
Old 09-23-2018
Notwithstanding the missing answer to Don Cragun's open questions, and for exactly the sample given in post#1, this might come close to a solution:

Code:
awk '{LEAD = $1 FS  $2 FS $3; TRAIL = $(NF-2) FS $(NF-1) FS $(NF); gsub (/^[^(]*\(|\)[^)]*$/, ""); print LEAD, $0, TRAIL}' file
1234 84 test1 bankUser 10 40 xys
1356 2 branches dirlisting:1 20 80 abc

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

2. Shell Programming and Scripting

Help with compare two column and print out column with smallest number

Input file : 5 20 500 2 20 41 41 0 23 1 Desired output : 5 2 20 0 1 By comparing column 1 and 2 in each line, I hope can print out the column with smallest number. I did try the following code, but it don't look good :( (2 Replies)
Discussion started by: perl_beginner
2 Replies

3. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

4. Shell Programming and Scripting

Print every 5 4th column values as separate row with different first column

Hi, I have the following file, chr1 100 200 20 chr1 201 300 22 chr1 220 345 23 chr1 230 456 33.5 chr1 243 567 90 chr1 345 600 20 chr1 430 619 21.78 chr1 870 910 112.3 chr1 914 920 12 chr1 930 999 13 My output would be peak1 20 22 23 33.5 90 peak2 20 21.78 112.3 12 13 Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

5. Shell Programming and Scripting

awk command to print only selected rows in a particular column specified by column name

Dear All, I have a data file input.csv like below. (Only five column shown here for example.) Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 3,2,4,5,6 5,3,5,5,6 From this I want the below output Data1,StepNo,Data2,Data3,Data4 2,1,3,4,5 3,1,5,6,7 where the second column... (4 Replies)
Discussion started by: ks_reddy
4 Replies

6. UNIX for Dummies Questions & Answers

how to print the column that contains a string in delimited file in unix

Hi Team, I have a requirement to print the column or find the column number of a particular string in delimited (;) file. Ex: aaa;bbb;ccc;rrr;mmm; gggg;rrr;mmmm;ssss;eee;aaa; ccc;gggg;tttt;bbbb;dddd;ggggg;rrr; my file contains 1000+ rows like above example with semicolon... (3 Replies)
Discussion started by: baskivs
3 Replies

7. Shell Programming and Scripting

comparing column of two different files and print the column from in order of 2nd file

Hi friends, My file is like: Second file is : I need to print the rows present in file one, but in order present in second file....I used while read gh;do awk ' $1=="' $gh'" {print >> FILENAME"output"} ' cat listoffirstfile done < secondfile but the output I am... (14 Replies)
Discussion started by: CAch
14 Replies

8. Shell Programming and Scripting

print first few lines, then apply regex on a specific column to print results.

abc.dat tty cpu tin tout us sy wt id 0 0 7 3 19 71 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 133.2 0.0 682.9 0.0 1.0 0.0 7.2 0 79 c1t0d0 0.2 180.4 0.1 5471.2 3.0 2.8 16.4 15.6 15 52 aaaaaa1-xx I want to skip first 5 line... (4 Replies)
Discussion started by: kchinnam
4 Replies

9. Shell Programming and Scripting

find expression with awk in only one column, and if it fits, print whole column

Hi. How do I find an expression with awk in only one column, and if it fits, then print that whole column. 1 apple oranges 2 bannanas pears 3 cats dogs 4 hesaid shesaid echo "which number:" read NUMBER (user inputs number 2 for this example) awk " /$NUMBER/ {field to search is field... (2 Replies)
Discussion started by: glev2005
2 Replies

10. Shell Programming and Scripting

Question about sort specific column and print other column at the same time !

Hi, This is my input file: ali 5 usa abc abu 4 uk bca alan 6 brazil bac pinky 10 utah sdc My desired output: pinky 10 utah sdc alan 6 brazil bac ali 5 usa abc abu 4 uk bca Based on the column two, I want to do the descending order and print out other related column at the... (3 Replies)
Discussion started by: patrick87
3 Replies
Login or Register to Ask a Question