Sponsored Content
Top Forums Shell Programming and Scripting reformat data with a shell script Post 302342117 by durden_tyler on Friday 7th of August 2009 12:09:17 PM
Old 08-07-2009
Here's one way to do it with Perl:

Code:
$
$ cat data.txt
>ALLLY
GGCCCCTCGAGCCTCGAACCGGAACCTCCAAATCCGAGACGCTCTGCTTATGAGGACCTC
GAAATATGCCGGCCAGTGAAAAAATCTTGTGGCTTTGAGGGCTTTTGGTTGGCCAGGGGC
AGTAAAAATCTCGGAGAGCTGACACCAAGTCCTCCCCTGCCACGTAGCAGTGGTAAAGTC
CGAAGCTCAAATTCCGAGAATTGAGCTCTGTTGATTCTTAGAACTGGGGTTCTTAGAAGT
>BLLLK
CTGGTCTCAGTCTGGTACTGAAGTCAGGAATGGCTTAAGGTGAAATCGTGGTCCTCTGGT
GAAGCTCAGCGAAGACCCCCTCGCCTTGTTTATGACAAGAGAACTTCTGGGGGCGGGAGG
AAGAGTCCCTGTTACGATGCTGATCATCATTGAGCTTTTGCTGAGCAGAAAACTCTTTAG
TACTCAAGGTCGAGAGTCTCTGGTGGTCTGCCTGGCACCAGGCACCTTCCTACAACCCTA
GTTTTCCAAAAGGACAAAGCCTGGGGCAGGCGACGTCCTAGCTCGCATTTGAACAGGGCC
GCGGGCCAGCAGAGATGCGCGATGCCCAACTCTTTCCAAGAGCACCTCGCGTCCCGAACC
$
$
$ perl -ne 'chomp; if (/^>/) {s/^>//; print $. != 1 ? "\n":"",$_,"\t"} else {print} END {print "\n"}' data.txt
ALLLY   GGCCCCTCGAGCCTCGAACCGGAACCTCCAAATCCGAGACGCTCTGCTTATGAGGACCTCGAAATATGCCGGCCAGTGAAAAAATCTTGTGGCTTTGAGGGCTTTTGGTTGGCCAGGGGCAGTAAAAATCTCGGAGAGCTGACACCAAGTCCTCCCCTGCCACGTAGCAGTGGTAAAGTCCGAAGCTCAAATTCCGAGAATTGAGCTCTGTTGATTCTTAGAACTGGGGTTCTTAGAAGT
BLLLK   CTGGTCTCAGTCTGGTACTGAAGTCAGGAATGGCTTAAGGTGAAATCGTGGTCCTCTGGTGAAGCTCAGCGAAGACCCCCTCGCCTTGTTTATGACAAGAGAACTTCTGGGGGCGGGAGGAAGAGTCCCTGTTACGATGCTGATCATCATTGAGCTTTTGCTGAGCAGAAAACTCTTTAGTACTCAAGGTCGAGAGTCTCTGGTGGTCTGCCTGGCACCAGGCACCTTCCTACAACCCTAGTTTTCCAAAAGGACAAAGCCTGGGGCAGGCGACGTCCTAGCTCGCATTTGAACAGGGCCGCGGGCCAGCAGAGATGCGCGATGCCCAACTCTTTCCAAGAGCACCTCGCGTCCCGAACC
$
$

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

help reformat data with awk

I am trying to write an awk program to reformat a data table and convert the date to julian time. I have all the individual steps working, but I am having some issues joing them into one program. Can anyone help me out? Here is my code so far: # This is an awk program to convert the dates from... (4 Replies)
Discussion started by: climbak
4 Replies

2. Shell Programming and Scripting

Reformat Data (Perl)

I am new to Perl. I need to reformat a data file as the last part of a script I am working on. I am stuck on this. Here is the current format: CUSTOMER Filename 09/04/07-08:49 CUSTOMER Filename 09/04/07-08:52 CUSTOMER Filename 09/04/07-08:52 CUSTOMER2 Filename 09/04/07-08:49 CUSTOMER2... (3 Replies)
Discussion started by: flood
3 Replies

3. Shell Programming and Scripting

Shell Script to Reformat a flat file

Hi , I have a text file noname.txt containing 1000+ records like this. One of the record I have given below. Input will b e like this BOT: 2010/06/01 00:25:59 21 = "private" Access-Method = 31 NCC = GBR 01 = "340806@osiris.fr.ft" 04 =... (2 Replies)
Discussion started by: smalya
2 Replies

4. Shell Programming and Scripting

Reformat the data of a file.

I have a file which have data like A.txt a 1Jan I am in a1. 1Jan I was born. 2Jan I am here. 3Jan I am in a3. b 1Jan I am in b1. c 2Jan I am in c2. d 2Jan I am in d2. 5jan I am in d5. date in the file might be vary evertime. (9 Replies)
Discussion started by: samkhu
9 Replies

5. Shell Programming and Scripting

Help with reformat data content

input file: hsa-miR-4726-5p Score hsa-miR-483-5p Score hsa-miR-125b-2* Score hsa-miR-4492 hsa-miR-4508 hsa-miR-4486 Score Desired output file: hsa-miR-4726-5p Score hsa-miR-483-5p Score hsa-miR-125b-2* Score hsa-miR-4492 hsa-miR-4508 hsa-miR-4486 Score ... (6 Replies)
Discussion started by: perl_beginner
6 Replies

6. Shell Programming and Scripting

Help with reformat input data

Input file: 58227131 50087390 57339526 40578034 65348841 55614853 64363217 44178559 Desired output file: 58227131 50087390 57339526 40578034 65348841 55614853 64363217 44178559 Command that I try: (4 Replies)
Discussion started by: perl_beginner
4 Replies

7. Shell Programming and Scripting

Reformat MLS Data - Use AWK?

I am helping my wife set up a real estate site and I am starting to integrate MLS listings. We are using a HostGator level 5 VPS running CentOS and have full root and SSH access to the VPS. Thus far I have automated the daily FTP download of listings from our MLS server using a little sh script.... (4 Replies)
Discussion started by: Chicago_Realtor
4 Replies

8. Shell Programming and Scripting

Data reformat and rearrangement problem asking

Input file: dependent general_process dependent general_process regulation general_process - - template component food component binding data_rearrangement binding data_rearrangement specific_activity data_rearrangement - ... (7 Replies)
Discussion started by: cpp_beginner
7 Replies

9. Shell Programming and Scripting

Help with reformat data structure

Input file: bv|111259484|pir||T49736_real_data bv|159484|pir||T9736_data_figure bv|113584|prf|T4736|truth bv|113584|pir||T4736_truth Desired output: bv|111259484|pir|T49736|real_data bv|159484|pir|T9736|data_figure bv|113584|prf|T4736|truth bv|113584|pir|T4736|truth Once the... (8 Replies)
Discussion started by: perl_beginner
8 Replies

10. Shell Programming and Scripting

Help with reformat data set

Input file 4CL1 O24145 CoA1 4CL1 P31684 CoA1 4CL1 Q54P77 CoA_1 73 O36421 Unknown 4CL3 Q9S777 coumarate 4CL3 Q54P79 coumarate 4CL3 QP7932 coumarate Desired output result 4CL1 O24145#P31684 CoA1 4CL1 Q54P77 CoA_1 73 O36421 Unknown 4CL3 Q9S777#Q54P79#QP7932 coumarate I... (5 Replies)
Discussion started by: perl_beginner
5 Replies
Path::Class(3)						User Contributed Perl Documentation					    Path::Class(3)

NAME
Path::Class - Cross-platform path specification manipulation VERSION
version 0.26 SYNOPSIS
use Path::Class; my $dir = dir('foo', 'bar'); # Path::Class::Dir object my $file = file('bob', 'file.txt'); # Path::Class::File object # Stringifies to 'foo/bar' on Unix, 'fooar' on Windows, etc. print "dir: $dir "; # Stringifies to 'bob/file.txt' on Unix, 'bobfile.txt' on Windows print "file: $file "; my $subdir = $dir->subdir('baz'); # foo/bar/baz my $parent = $subdir->parent; # foo/bar my $parent2 = $parent->parent; # foo my $dir2 = $file->dir; # bob # Work with foreign paths use Path::Class qw(foreign_file foreign_dir); my $file = foreign_file('Mac', ':foo:file.txt'); print $file->dir; # :foo: print $file->as_foreign('Win32'); # foofile.txt # Interact with the underlying filesystem: # $dir_handle is an IO::Dir object my $dir_handle = $dir->open or die "Can't read $dir: $!"; # $file_handle is an IO::File object my $file_handle = $file->open($mode) or die "Can't read $file: $!"; DESCRIPTION
"Path::Class" is a module for manipulation of file and directory specifications (strings describing their locations, like '/home/ken/foo.txt' or 'C:WindowsFoo.txt') in a cross-platform manner. It supports pretty much every platform Perl runs on, including Unix, Windows, Mac, VMS, Epoc, Cygwin, OS/2, and NetWare. The well-known module "File::Spec" also provides this service, but it's sort of awkward to use well, so people sometimes avoid it, or use it in a way that won't actually work properly on platforms significantly different than the ones they've tested their code on. In fact, "Path::Class" uses "File::Spec" internally, wrapping all the unsightly details so you can concentrate on your application code. Whereas "File::Spec" provides functions for some common path manipulations, "Path::Class" provides an object-oriented model of the world of path specifications and their underlying semantics. "File::Spec" doesn't create any objects, and its classes represent the different ways in which paths must be manipulated on various platforms (not a very intuitive concept). "Path::Class" creates objects representing files and directories, and provides methods that relate them to each other. For instance, the following "File::Spec" code: my $absolute = File::Spec->file_name_is_absolute( File::Spec->catfile( @dirs, $file ) ); can be written using "Path::Class" as my $absolute = Path::Class::File->new( @dirs, $file )->is_absolute; or even as my $absolute = file( @dirs, $file )->is_absolute; Similar readability improvements should happen all over the place when using "Path::Class". Using "Path::Class" can help solve real problems in your code too - for instance, how many people actually take the "volume" (like "C:" on Windows) into account when writing "File::Spec"-using code? I thought not. But if you use "Path::Class", your file and directory objects will know what volumes they refer to and do the right thing. The guts of the "Path::Class" code live in the "Path::Class::File" and "Path::Class::Dir" modules, so please see those modules' documentation for more details about how to use them. EXPORT The following functions are exported by default. file A synonym for "Path::Class::File->new". dir A synonym for "Path::Class::Dir->new". If you would like to prevent their export, you may explicitly pass an empty list to perl's "use", i.e. "use Path::Class ()". The following are exported only on demand. foreign_file A synonym for "Path::Class::File->new_foreign". foreign_dir A synonym for "Path::Class::Dir->new_foreign". Notes on Cross-Platform Compatibility Although it is much easier to write cross-platform-friendly code with this module than with "File::Spec", there are still some issues to be aware of. o On some platforms, notably VMS and some older versions of DOS (I think), all filenames must have an extension. Thus if you create a file called foo/bar and then ask for a list of files in the directory foo, you may find a file called bar. instead of the bar you were expecting. Thus it might be a good idea to use an extension in the first place. AUTHOR
Ken Williams, KWILLIAMS@cpan.org COPYRIGHT
Copyright (c) Ken Williams. All rights reserved. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Path::Class::Dir, Path::Class::File, File::Spec perl v5.16.2 2013-08-25 Path::Class(3)
All times are GMT -4. The time now is 08:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy