Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Converting tables of row data into columns of tables
Top Forums Shell Programming and Scripting Converting tables of row data into columns of tables Post 302127199 by justthisguy on Monday 16th of July 2007 03:52:37 PM
Old 07-16-2007
Thank y'all! drl, I largely used your example, thank you much for taking the time!

::beer:: >> drl

I've pasted a commented version of the guts of my solution, should anyone else have the same or similar questions.

It's rough edged (I need to sit down and work on handling the indenting of the header in cases other than 3 element data sets), but functional.

In the scenario I posted above, this script (let's name it 'massagetocolumn.sh') would be called as follows:

./massagetocolumn.sh data.file test_data_

Chris Larson
JustThisGuy

Code:
#!/bin/sh

# Exit if any variable is not set.
set -o nounset

# Input file:
DATA_FILE=${1}
echo "Data File: "$DATA_FILE

# Dataset Header Prefix.
HEADER_PREFIX=${2}
echo "Header Prefix: "$HEADER_PREFIX

# Make columns from space-delimited file.
# -e indicates a command, several of which can be included in one
# sed call. s indicates the string to search for, which is prefixed
# by '/'. The replacement string is prefixed by the second '/' and
# closed with a final '/'.
# Following the list of commands is the input file.
# ' > $DATA_FILE.temp" directs the output to an output file,
# in this case with '.temp' added to the filename.
# This file is removed when the script finishes. In this
# script, the 's/[[ ]]*/\t/' is finding all spaces and replacing them
# with TAB (\t). The 's/$HEADER_PREFIX\S*/&\t\t/' is finding all strings beginning
# with '$HEADER_PREFIX' and appending two tabs after each occurrence.
# Oh, and the 'g' tells sed to replace all occurrences, not just the first occurrence
# per line, which is the default behavior.
sed -e 's/[[ ]]*/\t/g' -e 's/$HEADER_PREFIX\S*/&\t\t/' $DATA_FILE > $DATA_FILE.temp

# Cut datasets from input file into separate temporary files, named as xx##.
# The '-k' option leaves the temp files in place in the case of an error.
# The '-s' option silences the default byte counts that csplit offers.
# The '-z' option deletes any output files that are empty.
# csplit cuts the data sets based on the search string, in this case:
# whatever you put as the second argument to the script.
csplit -k -s -z $DATA_FILE.temp /^$HEADER_PREFIX/ "{*}"

# Paste temporary files into output, piped through 'column' to create columns.
# NOTE: there is a TAB inside -s" ".
# The 'paste' command pastes multiple files into one, with the contents of all files side by side.
paste xx* | column -s"      " -t > $DATA_FILE.out

#Remove the temporary files.
rm xx*
rm $DATA_FILE.temp

# Exit
exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

extract data from html tables

hi i need to use unix to extract data from several rows of a table coded in html. I know that rows within a table have the tags <tr> </tr> and so i thought that my first step should be to to delete all of the other html code which is not contained within these tags. i could then use this method... (8 Replies)
Discussion started by: Streetrcr
8 Replies

2. Shell Programming and Scripting

Reading data from multiple tables from Oracle DB

Hi , I want to read the data from 9 tables in oracle DB into 9 different files in the same connection instance (session). I am able to get data from one table to one file with below code : X=`sqlplus -s user/pwd@DB <<eof select col1 from table1; EXIT; eof` echo $X>myfile Can anyone... (2 Replies)
Discussion started by: net
2 Replies

3. UNIX Desktop Questions & Answers

Extracting data from tables and storing in a file

Hi I am trying to write a script to extract information from a database and save that info to a csv file. I am using sql, an oracle database I have no idea how to even begin this. Can somebody please help. (8 Replies)
Discussion started by: ladyAnne
8 Replies

4. Shell Programming and Scripting

Shell script for comparing data of tables

Hi, I have two databases with same tables on different servers.I need to check the data content in each table and if something is missing, should print that. I have a tool which takes the snapshot the table structure,index so on and compares with the other server tables snapshot. Now i need... (1 Reply)
Discussion started by: nessj
1 Replies

5. Shell Programming and Scripting

Help converting row data to columns

I've been trying to figure this out for a while but I'm completely stumped. I have files with data in rows and I need to convert the data to columns. Each record contains four rows with a "field name: value" pair. I would like to convert it to four columns with the field names as column headers... (5 Replies)
Discussion started by: happy_ee
5 Replies

6. Shell Programming and Scripting

Extracting data from tables......

HOw to extracts data from tables in database. Merges them into one output file. This output file is loaded into another tables in database. (1 Reply)
Discussion started by: nari.bommi
1 Replies

7. Shell Programming and Scripting

Getting input data from 2 tables

Hello All, I have one table contains: Table1: 5 10 30 40 60 80 ... Table 2: 10 20 60 80 (7 Replies)
Discussion started by: krsnadasa
7 Replies

8. Shell Programming and Scripting

Append data by looking up 2 tables for multiple files

I want to lookup values from two different tables based on common columns and append. The trick is the column to be looked up is not fixed and varies , so it has to be detected from the header. How can I achieve this at once, for multiple data files, but lookup tables fixed. The two lookup... (5 Replies)
Discussion started by: ritakadm
5 Replies

9. Shell Programming and Scripting

UPDATE COmmand post comparing 2 columns in 2 mysql tables

my queryis : select distinct m.name, item_count, item from master m join client p on m.name=p.name where item_count = 1 and item > 1; But how should I update them? i used update statetment : Update from client Set item =1 where m.name=p.name and item_count=1 AND item>1 Is this wrong? (1 Reply)
Discussion started by: siya@
1 Replies

10. Shell Programming and Scripting

Script writing for tables with binary data

Hi There, I'm trying to write a simple script that will email me when we have an application job in a certain status that needs human intervention. I've used this script for other tables and it works great. However, this one gives me the warning that there is binary data so it might not. ... (2 Replies)
Discussion started by: Colel2
2 Replies

LEARN ABOUT SUSE

pdl::graphics::lut

LUT(3)							User Contributed Perl Documentation						    LUT(3)

NAME

       PDL::Graphics::LUT - provides access to a number of look-up tables

SYNOPSIS

	use PDL::Graphics::PGPLOT;
	use PDL::Graphics::LUT;

	# what tables are available
	my @tables = lut_names();

	# get the reversed colour table 'smooth',
	# with the gamma intensity ramp
	my ( $l, $r, $g, $b ) = lut_data( 'smooth', 1, 'gamma' );

	# use the table idl5 in ctab
	ctab( lut_data('idl5') );

DESCRIPTION

       PDL::Graphics::LUT contains a number of colour look-up tables (in rgb format) and intensity ramps, and provides routines to access this
       data.  The format of the data is suitable for use by "ctab" in PDL::Graphics::PGPLOT.

       Unlike the initial release of the package, the data tables are now stored within the PDL distribution as FITS files (see $tabledir and
       $rampdir), rather than in the module itself.  Changes to these directories will be picked up on the next call to one of the package
       functions.

FUNCTIONS

   lut_names()
       Return, as a list, the names of the available colour tables.

	@tables = lut_names();

   lut_ramps()
       Return, as a list, the names of the available intensity ramps.

	@ramps = lut_ramps();

   lut_data()
       Load in the requested colour table and intensity ramp.

	( $l, $r, $g, $b ) = lut_data( $table, [ $reverse, [ $ramp ] ] );

       Returns the levels and r, g, b components of the colour table $table. If $reverse is 1 (defaults to 0 if not supplied), then the r, g, and
       b components are reversed before being returned.  If not supplied, $ramp defaults to "ramp" (this is a linear intensity ramp).

       The returned values are piddles containing values in the range 0 to 1 inclusive, and are floats.

VARIABLES

   $tabledir
       The directory in which the colour tables (in rgb format) are stored.

   $rampdir
       The directory in which the intensity ramps are stored.

   $suffix
       The suffix for the data files in $tabledir and $rampdir.

FURTHER INFORMATION

       The colour tables were taken from the STARLINK GAIA package, and are provided under the GNU copyleft.  See http://star-www.rl.ac.uk/ and
       http://star-www.dur.ac.uk/~pdraper/ for more details.

AUTHOR

       Doug Burke (djburke@cpan.org), with thanks to Peter Draper/STARLINK for providing the colour-table data, and Christian Soeller and Karl
       Glazebrook for their help.

       All rights reserved. There is no warranty. You are allowed to redistribute this software / documentation under certain conditions. For
       details, see the file COPYING in the PDL distribution. If this file is separated from the PDL distribution, the copyright notice should be
       included in the file.

perl v5.12.1							    2009-10-17								    LUT(3)
All times are GMT -4. The time now is 09:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy