Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How convert space separated list to matched columns? Post 302830539 by Manchesterpaul on Tuesday 9th of July 2013 05:20:24 AM
Old 07-09-2013
How convert space separated list to matched columns?

Hi
I have been racking my (limited) brains to get this to work without success

I have a file output which is a list of lists - ie a single column of data that is separated by space into sub lists below - I need to both split this so that each list is in a separate column (eg tab or semicolon separated) but then also I need to match up the rows in each column. The data looks like this but with about 2000 sub-lists:

AF10_identified_SNPs.vcf:
Code:
CADAFUAG00000594,53,240575,G,A
CADAFUAG00000594,53,242762,A,G
CADAFUAG00000594,53,243450,T,G
CADAFUAG00003722,94,896971,T,C
CADAFUAG00003722,94,898802,A,C
CADAFUAG00003979,102,144910,A,C
CADAFUAG00003979,102,145332,C,T
CADAFUAG00003979,102,145623,T,C
CADAFUAG00004144,93,1548479,A,G
CADAFUAG00004144,93,1548862,T,C
CADAFUAG00004144,93,1548899,T,A
CADAFUAG00004144,93,1549237,A,G
CADAFUAG00004144,93,1549256,G,A
CADAFUAG00004225,101,81335,T,C

AF1163_identified_SNPs.vcf:
Code:
CADAFUAG00000594,53,240575,G,A
CADAFUAG00000594,53,242762,A,G
CADAFUAG00000594,53,243450,T,G
CADAFUAG00003722,94,898802,A,C
CADAFUAG00003979,102,145628,C,T
CADAFUAG00004144,93,1548335,G,A
CADAFUAG00004144,93,1548479,A,G
CADAFUAG00004144,93,1548862,T,C
CADAFUAG00004144,93,1548899,T,A
CADAFUAG00004144,93,1549237,A,G
CADAFUAG00004144,93,1549256,G,A
CADAFUAG00004225,101,81335,T,C

AF210_identified_SNPs.vcf:
Code:
CADAFUAG00000594,53,239935,A,T
CADAFUAG00000594,53,240575,G,A
CADAFUAG00000594,53,241319,C,T
CADAFUAG00000594,53,242762,A,G
CADAFUAG00000594,53,243450,T,G
CADAFUAG00003722,94,897525,G,A
CADAFUAG00003979,102,145623,T,C
CADAFUAG00004144,93,1548479,A,G
CADAFUAG00004144,93,1548658,T,G
CADAFUAG00004144,93,1548862,T,C
CADAFUAG00004144,93,1548899,T,A
CADAFUAG00004144,93,1549256,G,A
CADAFUAG00004144,93,1549271,C,A
CADAFUAG00005594,100,552059,T,C

any help would be much appreciated
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert from CSV to space padded columns (.ksh)

Hello, Could someone please help me to convert a string(s) of comma separated values into space padded columns in .ksh? ex. 10-21-2008,someword,blah,127.0.0.1,8,3 10-21-2008,randomword,ick,128.0.111.128,1,0 converted to 10-21-2008 someword blah 127.0.0.1 8... (6 Replies)
Discussion started by: WhotheWhat
6 Replies

2. Shell Programming and Scripting

how to convert a line to columns, separated by | (pipe)

Hi, Plz help. input line 1;20100403;400|2;20100403;4|3;20290903;400|4;20290903;0|5;20290903;0|9;20100304;0|10;20100304;0|11;20100402;0|18;20100304;0 expected output 1;20100403;400 2;20100403;4 3;20290903;400 4;20290903;0 5;20290903;0 9;20100304;0 10;20100304;0 11;20100402;0... (4 Replies)
Discussion started by: suresh3566
4 Replies

3. Shell Programming and Scripting

Convert a tab separated file using bash

Dear all, I have a file in this format (like a matrix) - A B C .. X A 1 4 2 .. 2 B 2 6 4 .. 8 C 3 5 5 .. 4 . . . ... . X . . ... . and want to convert it into a file with this format: A A = 1 A B = 4 A C = 2 ... A X = 2 B A = 2 B B = 6 etc (2 Replies)
Discussion started by: TheTransporter
2 Replies

4. Shell Programming and Scripting

How to loop through space separated values?

How do I loop thru space separated values in a variable? I hate to use very complicated counter increment logic for this kind of simple problem. Expected result(using ksh) $>echo "aaa bbbb cccc" | <looping code here> var=aaa var=bbbb var=cccc $>echo "aaa bbbb cccc" | while IFS=" "... (12 Replies)
Discussion started by: kchinnam
12 Replies

5. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

6. Shell Programming and Scripting

Convert comma separated file to fix length

Hi, I am converting a comma separated file to fixed field lenght and I am using that: COLUMNS="25 24 67 26 39 63 20 34 35 14 397" ( cat $indir/input_file.dat | \ $AWK -v columns="$COLUMNS" ' BEGIN { FS=","; OFS=""; split(columns, arr, " "); } { for(i=1; i<=NF;... (5 Replies)
Discussion started by: apenkov
5 Replies

7. Shell Programming and Scripting

Matched a pattern from multiple columns

Hi, I need to extract an info in $1 based on a matched pattern in $2,$3,$4, and $5. The sample input file as follows:- ID Pat1 Pat2 Pro1 use1 add41 M M M add87 M M M M add32 ... (16 Replies)
Discussion started by: redse171
16 Replies

8. Shell Programming and Scripting

Need Help on For Loop to pass space separated value as one value

Hi, I am having a file say list1 with a output like below jun 12 18:23 may 20 18:23 Now i want to pass the above two values into for loop,I have written a script like this. #!/bin/bash a=`cat list1` for i in $a do echo "HI $i" done expected output: HI jun 12 18:23 (3 Replies)
Discussion started by: sumanthupar
3 Replies

9. Shell Programming and Scripting

Convert column to quote and comma separated row

Hi, I have a list of tables in a file.txt C_CLAIM C_HLD C_PROVIDER I want the output to be 'C_CLAIM','C_HLD','C_PROVIDER' Currently I'm usin awk and getting output which is almost correct but still has minor defects awk -vORS="','" '{ print $1 }' file.txt The output of... (4 Replies)
Discussion started by: wahi80
4 Replies

10. Shell Programming and Scripting

Convert fixed value fields to comma separated values

Hi All, Hope you are doing Great!!!. Today i have came up with a problem to say exactly it was for performance improvement. I have written code in perl as a solution for this to cut in specific range, but it is taking time to run for files thousands of lines so i am expecting a sed... (9 Replies)
Discussion started by: mad man
9 Replies
LOCATE(1)						    BSD General Commands Manual 						 LOCATE(1)

NAME
locate -- find filenames quickly SYNOPSIS
locate [-0Scims] [-l limit] [-d database] pattern ... DESCRIPTION
The locate program searches a database for all pathnames which match the specified pattern. The database is recomputed periodically (usually weekly or daily), and contains the pathnames of all files which are publicly accessible. Shell globbing and quoting characters (``*'', ``?'', ``'', ``['' and ``]'') may be used in pattern, although they will have to be escaped from the shell. Preceding any character with a backslash (``'') eliminates any special meaning which it may have. The matching differs in that no characters must be matched explicitly, including slashes (``/''). As a special case, a pattern containing no globbing characters (``foo'') is matched as though it were ``*foo*''. Historically, locate only stored characters between 32 and 127. The current implementation store any character except newline (' ') and NUL (''). The 8-bit character support does not waste extra space for plain ASCII file names. Characters less than 32 or greater than 127 are stored in 2 bytes. The following options are available: -0 Print pathnames separated by an ASCII NUL character (character code 0) instead of default NL (newline, character code 10). -S Print some statistics about the database and exit. -c Suppress normal output; instead print a count of matching file names. -d database Search in database instead of the default file name database. Multiple -d options are allowed. Each additional -d option adds the specified database to the list of databases to be searched. The option database may be a colon-separated list of databases. A single colon is a reference to the default database. $ locate -d $HOME/lib/mydb: foo will first search string ``foo'' in $HOME/lib/mydb and then in /var/db/locate.database. $ locate -d $HOME/lib/mydb::/cdrom/locate.database foo will first search string ``foo'' in $HOME/lib/mydb and then in /var/db/locate.database and then in /cdrom/locate.database. $ locate -d db1 -d db2 -d db3 pattern is the same as $ locate -d db1:db2:db3 pattern or $ locate -d db1:db2 -d db3 pattern If - is given as the database name, standard input will be read instead. For example, you can compress your database and use: $ zcat database.gz | locate -d - pattern This might be useful on machines with a fast CPU and little RAM and slow I/O. Note: you can only use one pattern for stdin. -i Ignore case distinctions in both the pattern and the database. -l number Limit output to number of file names and exit. -m Use mmap(2) instead of the stdio(3) library. This is the default behavior and is faster in most cases. -s Use the stdio(3) library instead of mmap(2). ENVIRONMENT
LOCATE_PATH path to the locate database if set and not empty, ignored if the -d option was specified. FILES
/var/db/locate.database locate database /usr/libexec/locate.updatedb Script to update the locate database /etc/periodic/weekly/310.locate Script that starts the database rebuild SEE ALSO
find(1), whereis(1), which(1), fnmatch(3), locate.updatedb(8) Woods, James A., "Finding Files Fast", ;login, 8:1, pp. 8-10, 1983. HISTORY
The locate command first appeared in 4.4BSD. Many new features were added in FreeBSD 2.2. BUGS
The locate program may fail to list some files that are present, or may list files that have been removed from the system. This is because locate only reports files that are present in the database, which is typically only regenerated once a week by the /etc/periodic/weekly/310.locate script. Use find(1) to locate files that are of a more transitory nature. The locate database is typically built by user ``nobody'' and the locate.updatedb(8) utility skips directories which are not readable for user ``nobody'', group ``nobody'', or world. For example, if your HOME directory is not world-readable, none of your files are in the data- base. The locate database is not byte order independent. It is not possible to share the databases between machines with different byte order. The current locate implementation understands databases in host byte order or network byte order if both architectures use the same integer size. So on a FreeBSD/i386 machine (little endian), you can read a locate database which was built on SunOS/sparc machine (big endian, net). The locate utility does not recognize multibyte characters. BSD
August 17, 2006 BSD
All times are GMT -4. The time now is 10:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy