Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Fetching 1st Column and Last n Columns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Fetching 1st Column and Last n Columns
# 8  
Old 10-06-2016
@Don.. Thank you for fixing the code and telling me the issue.

Actually I need to extract last 100 columns and columns numbers changes everyday that's why can't use
Code:
awk '{print $1,$(NF-9),$(NF-8),$(NF-7),$(NF-6),$(NF-5),$(NF-4),$(NF-3),$(NF-2),$(NF-1),$NF}'

.

Thank you for help and fixing the issue in my code. It is working fine now Smilie
# 9  
Old 10-06-2016
Is it always a fixed number of columns in each file? If so, you could use cut:-
Code:
tab=$(printf "\t")                                             # Just defining the tab character for clarity

columns=$(set - $(head -1 input_file | tr " " "_") ; echo $#)  # Count number of columns today.  The tr eliminates spaces, just for counting columns

((start_col=$columns-99))                                      # Count back 99 columns (else you will get an extra one)

cut -d "$tab" -f1,${start_col}- input_file                     # Note the trailing hyphen on the field/column definition




Robin
Last edited by rbatte1; 10-06-2016 at 01:23 PM.. Reason: Defined tab character for clarity
This User Gave Thanks to rbatte1 For This Post:
RavinderSingh13
# 10  
Old 10-06-2016
Quote:
Originally Posted by rbatte1
Is it always a fixed number of columns in each file? If so, you could use cut:-
Code:
tab=$(printf "\t")                                             # Just defining the tab character for clarity
columns=$(set - $(head -1 input_file | tr " " "_") ; echo $#)  # Count number of columns today.  The tr eliminates spaces, just for counting columns
((start_col=$columns-99))                                      # Count back 99 columns (else you will get an extra one)
cut -d "$tab" -f1,${start_col}- input_file                     # Note the trailing hyphen on the field/column definition

Robin
Hello rbatte1,

Thank you for your nice code sir, may I add here my humble view here(hope I am correct here, kindly correct me if I am wrong here).
I think if we need to print last 100 columns then final line in above code could be following.
Code:
cut -d "$tab" -f${start_col},${columns}- Input_file

I haven't tested it though.

Thanks,
R. Singh
# 11  
Old 10-06-2016
Quote:
Originally Posted by RavinderSingh13
Hello rbatte1,

Thank you for your nice code sir, may I add here my humble view here(hope I am correct here, kindly correct me if I am wrong here).
I think if we need to print last 100 columns then final line in above code could be following.
Code:
cut -d "$tab" -f${start_col},${columns}- Input_file

I haven't tested it though.

Thanks,
R. Singh
Hi Ravinder,
No. The list that is the option-argument to the cut -b list, -c list, and -f list options is a comma separated set of specifiers of one of four forms:
  • number: specifying that that byte, character, or field number number, respectively, (with the 1st item on a line being number 1) be output,
  • number-: specifying that byte, character, or field number number and every byte, character, or field following it on that line be output,
  • num1-num2: specifying that byte, character, or field numbers num1 through num2, inclusive, be output, and
  • -number: specifying that byte, character, or field numbers 1 through number, inclusive, be output.
Since the request in this thread is to print the 1st field and the last 100 fields and Robin's code is setting columns to the number of fields on the first line in the file named input_file, (assuming that the field delimiter in input_file is a <tab> character, that would be:
Code:
cut -f 1,$((columns - 99))- input_file

(the -d delimiter option is not needed since the default delimiter in cut is the <tab> character).

Hi Robin,
The command:
Code:
columns=$(set - $(head -1 input_file | tr " " "_") ; echo $#)  # Count number of columns today.  The tr eliminates spaces, just for counting columns

can be changed to:
Code:
IFS= read -r line < input_file
columns=$(IFS=$tab; set -- $line; echo $#)

using just shell built-ins without needing to exec head and tr. Note also that the standards explicitly stated that:
Code:
set - arg...

produces unspecified results starting in the 2004 revision of the standard; that form was deprecated in the 1992 edition of the standard with the preferred form being:
Code:
set -- arg...

These 2 Users Gave Thanks to Don Cragun For This Post:
RavinderSingh13 rbatte1
# 12  
Old 10-07-2016
Hello Don,

Quote:
Hi Robin,
The command:
Code:
columns=$(set - $(head -1 input_file | tr " " "_") ; echo $#)  # Count number of columns today.  The tr eliminates spaces, just for counting columns

can be changed to:
Code:
IFS= read -r line < input_file
columns=$(IFS=$tab; set -- $line; echo $#)

using just shell built-ins without needing to exec head and tr. Note also that the standards explicitly stated that:
Code:
set - arg...

produces unspecified results starting in the 2004 revision of the standard; that form was deprecated in the 1992 edition of the standard with the preferred form being:
Code:
set -- arg...

Many thanks for the comments (again) and I'm delighted to have a way to save a few processes. I imaging the cost of running what I had in a loop reading a large file and counting columns for each record would be quite expensive. This is very useful.

Regarding the standards change, I was unaware so this is really useful and will hopefully stop me failing in the future.


Kindest regards,
Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Compare 1st column from 2 file and if match print line from 1st file and append column 7 from 2nd

hi I have 2 file with more than 10 columns for both 1st file apple,0,0,0...... orange,1,2,3..... mango,2,4,5..... 2nd file apple,2,3,4,5,6,7... orange,2,3,4,5,6,8... watermerlon,2,3,4,5,6,abc... mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies

2. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

3. Linux

Print the 1st column and the value in 2nd or 3rd column if that is different from the values in 1st

I have file that looks like this, DIP-17571N|refseq:NP_651151 DIP-17460N|refseq:NP_511165|uniprotkb:P45890 DIP-17571N|refseq:NP_651151 DIP-19241N|refseq:NP_524261 DIP-19241N|refseq:NP_524261 DIP-17151N|refseq:NP_524316|uniprotkb:O16797 DIP-19588N|refseq:NP_731165 ... (2 Replies)
Discussion started by: Syeda Sumayya
2 Replies

4. UNIX for Dummies Questions & Answers

Want the UNIX code - I want to sum of the 1st column wherever the first 2nd and 3rd columns r equal

I have the code for the below things.. File1 has the content as below 8859 0 subscriberCreate 18 0 subscriberPaymentMethodChange 1650 0 subscriberProfileUpdate 7668 0 subscriberStatusChange 13 4020100 subscriberProfileUpdate 1 4020129 subscriberStatusChange 2 4020307 subscriberCreate 8831... (5 Replies)
Discussion started by: Mahen
5 Replies

5. Shell Programming and Scripting

Fetching values in CSV file based on column name

input.csv: Field1,Field2,Field3,Field4,Field4 abc ,123 ,xyz ,000 ,pqr mno ,123 ,dfr ,111 ,bbb output: Field2,Field4 123 ,000 123 ,111 how to fetch the values of Field4 where Field2='123' I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Discussion started by: bharathbangalor
10 Replies

6. Shell Programming and Scripting

Fetching columns from .csv file except last column

Hi, i have below list of files so i just want the name of the files in one parameter and not the timestamp. i want only GIDW_Dy_Tm_Seg_Sls_legacy_PL_0_0_ in variable of all files. GIDW_Dy_Tm_Seg_Sls_legacy_PL_0_0_20131001101800.csv GIDW_Dly_Sls_legacy_RO_0_0_20131001172001.csv ... (9 Replies)
Discussion started by: renuk
9 Replies

7. Shell Programming and Scripting

Combine columns from many files but keep them aligned in columns-shorter left column issue

Hello everyone, I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble. I have many files each having two columns and hundreds of rows. first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
Discussion started by: isildur1234
5 Replies

8. Shell Programming and Scripting

Help In fetching the 2nd Peak value of a column

Hi, We have a requirement of calculating the AVG, Peak and 2nd Peak (Say Peak2) of a column in a text file. We are able to get Avg and Peak but we are finding difficulty in getting the 2nd Peak. I tried using the AWK but in vainL. Please find the below Script i have written for fetching AVG and... (3 Replies)
Discussion started by: mahesh Madpathi
3 Replies

9. UNIX for Dummies Questions & Answers

two files.say a and b.both have long columns.i wanna match the column fron 1st file w

ex: a file has : 122323 123456456 125656879 678989965t635 234323432 b has : this is finance no. this is phone no this is extn ajkdgag idjsidj i want the o/p as: 122323 his is finance no. 123456456 this is phone no 123456456 ... (4 Replies)
Discussion started by: TRUPTI
4 Replies

10. Shell Programming and Scripting

How to check Null values in a file column by column if columns are Not NULLs

Hi All, I have a table with 10 columns. Some columns(2nd,4th,5th,7th,8th and 10th) are Not Null columns. I'll get a tab-delimited file and want to check col by col and generate seperate error code for each col eg:102 if 2nd col value is NULL and 104 if 4th col value is NULL so on... I am a... (7 Replies)
Discussion started by: Mandab
7 Replies
Login or Register to Ask a Question