Keep only columns in first two rows based on partial header pattern.

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Keep only columns in first two rows based on partial header pattern.
# 1  
Old 08-30-2016
Keep only columns in first two rows based on partial header pattern.

I have this code below that only prints out certain columns from the first two rows (doesn't affect rows 3 and beyond). How can I do the same on a partial header pattern “G_TP” instead of having to know specific column numbers (e.g. 374-479)? I've tried many other commands within this pipe with no luck.

I have 53 columns out of 600 or so that the header pattern begins with “G_TP”, but the columns may change positions and if that happens, my method below won't work any longer because of the fixed column number “374-479”.

Code:
(head -n 2 file.csv | cut -d ',' -f 374-479; tail -n +3 file.csv) > file.txt

Thank you!
# 2  
Old 08-30-2016
Please post a reasonably reduced but representative sample of your input and the related output.
# 3  
Old 08-30-2016
Here are the full input (..fsw1952) and output (..fsw1953) files attached. As you can see, after I run the code from my initial post, the output file (..fsw1953) only contains the header pattern "G_TP" columns in the first two rows only (header and data).

This is the correct output, but I just want my code to perform this based on partial pattern instead of fixed column numbers (as mentioned before).

Thanks!
# 4  
Old 08-30-2016
Try:
Code:
#!/bin/ksh
ERE=${1:-"^G_TP"}
file=${2:-"f16_may_25_16_23_04_fsw1952.txt"}

awk -F, -v ERE="$ERE" '
NR == 1 {
	for(i = 1; i <= NF; i++)
		if($i ~ ERE)
			of[++nof] = i
}
NR < 3 {for(i = 1; i <= nof; i++)
		printf("%s%s", $of[i], (i == nof) ? "\n" : FS)
	next
}
1' "$file"

Although written and tested using a Korn shell, this should work with any shell supporting POSIX shell syntax. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.

When this script is run with no operands, it produces output identical to the file you attached named f16_may_25_16_23_04_fsw1953.txt.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 08-30-2016
Okay, thank you! Let me try this and get back with the results.

---------- Post updated at 05:19 PM ---------- Previous update was at 04:59 PM ----------

Don, that script seemed to work very well!

Thank you for your time!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Conversion of rows to columns using awk based om column value

HI, My Input file data is dn:adcfgeneral id:13343 Name:xxxxxx Password:iutyerwuitywue wpuwt tuiytruityrutyrwtyrwp dn:cdferwjyyyy id:3875 Name:yyyy Password :hgfdsjkfhdsfkdlshf dshfkldshfdklsfh interset:uiuiufj My output should be ... (6 Replies)
Discussion started by: dineshaila
6 Replies

2. Emergency UNIX and Linux Support

Average columns based on header name

Hi Friends, I have files with columns like this. This sample input below is partial. Please check below for main file link. Each file will have only two rows. ... (8 Replies)
Discussion started by: jacobs.smith
8 Replies

3. Shell Programming and Scripting

Convert rows to columns based on key and count

Team, I am having requirement to convert rows to columns Input is: key ,count, id1, pulse1, id2, pulse2 ,id3, pulse3 12, 2 , 14 , 56 , 15, 65 13, 3, 12, 32, 14, 23, 18, 54 22, 1 , 32, 42 Expected Out put: key, id,pulse 12, 14, 56 12, 15, 65 13 ,12, 32 13, 14 ,23 13, 18 ,54 22 ,32,... (3 Replies)
Discussion started by: syam1406
3 Replies

4. Shell Programming and Scripting

Make copy of text file with columns removed (based on header)

Hello, I have some tab delimited text files with a three header rows. The headers look like, (sorry the tabs look so messy). index group Name input input input input input input input input input input input... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

5. Shell Programming and Scripting

Convert rows to columns based on condition

I have a file some thing like this: GN Name=YWHAB; RC TISSUE=Keratinocyte; RC TISSUE=Thymus; CC -!- FUNCTION: Adapter protein implicated in the regulation of a large CC spectrum of both general and specialized signaling pathways GN Name=YWHAE; RC TISSUE=Liver; RC ... (13 Replies)
Discussion started by: raj_k
13 Replies

6. Shell Programming and Scripting

Extracting rows and columns in a matrix based on condition

Hi I have a matrix with n rows and m columns like below example. i want to extract all the pairs with values <200. Input A B C D A 100 206 51 300 B 206 100 72 48 C 351 22 100 198 D 13 989 150 100 Output format A,A:200 A,C:51 B,B:100... (2 Replies)
Discussion started by: anurupa777
2 Replies

7. Shell Programming and Scripting

Extract columns based on header

Hi to all, I have two files. File1 has no header, two columns: sample1 A sample2 B sample3 B sample4 C sample5 A sample6 D sample7 D File2 has a header, except for the first 3 columns (chr,start,end). "sample1" is the header for the 4th ,5th ,6th columns, "sample2" is the header... (4 Replies)
Discussion started by: aec
4 Replies

8. Shell Programming and Scripting

Skipping rows based on columns

Hi, suppose I have the following file and certain rows have missing columns, how do i skip these rows and create an output file which has all the columns in it E/N Ko_exp %err Ko_calc %err diff diff- diff+ 0.95 ======== ======= ==== ======= ==== ===== ===== =====... (12 Replies)
Discussion started by: ramky79
12 Replies

9. Shell Programming and Scripting

Selecting rows based on values in columns

Hi My pipe delimited .txt file contains rows with 10 columns. Can anyone advise how I output to file only those rows with the letters ‘ci' as the first 2 characters in the 3rd column ? Many thanks (4 Replies)
Discussion started by: malts18
4 Replies

10. Shell Programming and Scripting

Arrange output based on rows into columns

Hi All, I would like to ask help on how can i achieve below output. Inputfile: Oct11,apa1-daily,01:25:01 Oct11,apa2-daily,01:45:23 Oct12,apa1-daily,02:30:11 Oct12,apa2-daily,01:55:01 Oct13,apa1-off,01:43:34 Oct13,apa2-off,01:22:04 Desired output: Clients ... (3 Replies)
Discussion started by: mars101
3 Replies
Login or Register to Ask a Question