Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Problem in extraction when space is a field delimiter
Top Forums Shell Programming and Scripting Problem in extraction when space is a field delimiter Post 302443559 by AshwaniSharma09 on Monday 9th of August 2010 08:49:04 AM
Old 08-09-2010
Problem in extraction when space is a field delimiter
I have more than 1000 files to parse. Each file contains few lines (number of lines varies) followed by a header line having all column's name (SPOT, NAME etc) and then values for those columns.
**Example File:
Code:
sdgafh
dfhaadfha
sfgaf dhah jkthdj
SPOT  NAME  GENE_NAME CH_MEAN   CHDN_MED  CH2B_MEAN
1         IYPR1         abc                      1.5                 3                      4.5
2         IYPR9         def                       3.6                                         6.3
3         IYPR11        ghi                      2.6                 4                      2.8
4         IYPR13        jkl                       1.6                                         6.7
5         IYPR19        mno                    2.5                 7                      4.3

. . . . . .
. . . . . .
Problems:

1) I need to remove all lines before the header line (for each file).
2) I need to extract 2 columns, let say 2nd (NAME) & 5th column (CHDN_MED). Since there is no values for 2nd & 4th rows for 5th column, it should give a blank space for each in output file. But its giving 6.3 and 6.7 respectively from the 6th column (I am tring with AWK).

The desired output file is :
Code:
IYPR1      3
IYPR9   
IYPR11    4
IYPR13   
IYPR19    7
.                 .
.                 .

**the actual files are with .xls extension and each file has more than 50 columns and 9,000 rows.

Please advise

Thanks
Last edited by Scott; 08-09-2010 at 10:03 AM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extraction of string from Stringlist using delimiter

Hi Experts, I need to extract some set of strings one be one using delimiter. Example: shellscript.sh|unix.sh|script_file.sh i need to extract this shellscript.sh,unix.sh,script_file.sh separately. I tried but couldn't get. Please help me.. Thanks & Regards :), Kanda (3 Replies)
Discussion started by: spkandy
3 Replies

2. Shell Programming and Scripting

delimiter appears in field

The typical line of the input file is as follows, 123|abcde|"xyz|mn"|ghelosa|3455hello| The delimiter is |. I need to change it to another delimiter, say ~. For the above line, the output should be: 123~abcde~xyz|mn~ghelosa~3455hello~ The challenge is when | appears in a field, it... (2 Replies)
Discussion started by: derekxu
2 Replies

3. UNIX for Dummies Questions & Answers

Problem Using Cut With A Space Delimiter

I am trying to extract 'postmaster' from the following string: PenaltyError:=554 5.7.1 Error, send your mail to postmaster@LOCALDOMAIN using the following command: cat /usr/share/assp/assp.cfg | grep ^PenaltyError:= | cut -d '@' -f1 | cut -f8 but it returns: PenaltyError:=554 5.7.1 Error,... (10 Replies)
Discussion started by: cleanden
10 Replies

4. UNIX for Dummies Questions & Answers

Delimiter: Tab or Space?

Hello, Is there a direct command to check if the delimiter in your file is a tab or a space? And how can they be converted from one to another. Thanks, G (4 Replies)
Discussion started by: Gussifinknottle
4 Replies

5. Shell Programming and Scripting

Add field delimiter for the last field

I have a file with three fields and field delimiter '|' like: abc|12:13:45|123 xyz|12:87:32| qwe|54:21:09 In the file the 1st line has proper data -> abc|12:13:45|123 ,the 2nd line doesnt has data for the 3rd field which is okay , the 3rd line doesnt has data for the 3rd field as well the... (5 Replies)
Discussion started by: mehimadri
5 Replies

6. Shell Programming and Scripting

deplace field delimiter

hi here my problem: i have 2 file: 1.tmp 111 222 555 2.tmp 1*TEST1**111*LA 2*TEST2**112*LA 3*TEST3**222*LA 4*TEST4**333*LA 5*TEST5**555*LA (5 Replies)
Discussion started by: saw7
5 Replies

7. Shell Programming and Scripting

Help needed XML Field Extraction

I had an immediate work to sort out the error code and error message which are associated within the log. But here im facing an problem to extract 3 different fields from the XML log can some one please help. I tried using different script including awk & nawk, but not getting the desired output. ... (18 Replies)
Discussion started by: raghunsi
18 Replies

8. Shell Programming and Scripting

Space as a delimiter

not sure if i'm doing this right i'm new tho this but i'm trying to use a space as a delimiter with the cut command my code is size=$( du -k -S -s /home/cmik | cut -d' ' -f1 ) i've also tried -f2 and switching the -d and -f around if that does anything (3 Replies)
Discussion started by: Cmik
3 Replies

9. Shell Programming and Scripting

Need to use delimiter as : and space in awk

Hi , Please suggest me how do I use : (colon and one space) as a delimiter in awk Best regards, Vishal (2 Replies)
Discussion started by: Vishal_dba
2 Replies

10. Shell Programming and Scripting

How can awk ignore the field delimiter like comma inside a field?

We have a csv file as mentioned below and the requirement is to change the date format in file as mentioned below. Current file (file.csv) ---------------------- empname,date_of_join,dept,date_of_resignation ram,08/09/2015,sales,21/06/2016 "akash,sahu",08/10/2015,IT,21/07/2016 ... (6 Replies)
Discussion started by: gopal.biswal
6 Replies

LEARN ABOUT XFREE86

comm

COMM(1) 							   User Commands							   COMM(1)

NAME

       comm - compare two sorted files line by line

SYNOPSIS

       comm [OPTION]... FILE1 FILE2

DESCRIPTION

       Compare sorted files FILE1 and FILE2 line by line.

       When FILE1 or FILE2 (not both) is -, read standard input.

       With  no  options,  produce three-column output.  Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and
       column three contains lines common to both files.

       -1     suppress column 1 (lines unique to FILE1)

       -2     suppress column 2 (lines unique to FILE2)

       -3     suppress column 3 (lines that appear in both files)

       --check-order
	      check that the input is correctly sorted, even if all input lines are pairable

       --nocheck-order
	      do not check that the input is correctly sorted

       --output-delimiter=STR
	      separate columns with STR

       --total
	      output a summary

       -z, --zero-terminated
	      line delimiter is NUL, not newline

       --help display this help and exit

       --version
	      output version information and exit

       Note, comparisons honor the rules specified by 'LC_COLLATE'.

EXAMPLES

       comm -12 file1 file2
	      Print only lines present in both file1 and file2.

       comm -3 file1 file2
	      Print lines in file1 not in file2, and vice versa.

AUTHOR

       Written by Richard M. Stallman and David MacKenzie.

REPORTING BUGS

       GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
       Report comm translation bugs to <http://translationproject.org/team/>

COPYRIGHT

       Copyright (C) 2017 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       join(1), uniq(1)

       Full documentation at: <http://www.gnu.org/software/coreutils/comm>
       or available locally via: info '(coreutils) comm invocation'

GNU coreutils 8.28						   January 2018 							   COMM(1)
All times are GMT -4. The time now is 06:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy