Sponsored Content
Top Forums Shell Programming and Scripting Converting tables of row data into columns of tables Post 302127199 by justthisguy on Monday 16th of July 2007 03:52:37 PM
Old 07-16-2007
Thank y'all! drl, I largely used your example, thank you much for taking the time!

::beer:: >> drl

I've pasted a commented version of the guts of my solution, should anyone else have the same or similar questions.

It's rough edged (I need to sit down and work on handling the indenting of the header in cases other than 3 element data sets), but functional.

In the scenario I posted above, this script (let's name it 'massagetocolumn.sh') would be called as follows:

./massagetocolumn.sh data.file test_data_

Chris Larson
JustThisGuy

Code:
#!/bin/sh

# Exit if any variable is not set.
set -o nounset

# Input file:
DATA_FILE=${1}
echo "Data File: "$DATA_FILE

# Dataset Header Prefix.
HEADER_PREFIX=${2}
echo "Header Prefix: "$HEADER_PREFIX

# Make columns from space-delimited file.
# -e indicates a command, several of which can be included in one
# sed call. s indicates the string to search for, which is prefixed
# by '/'. The replacement string is prefixed by the second '/' and
# closed with a final '/'.
# Following the list of commands is the input file.
# ' > $DATA_FILE.temp" directs the output to an output file,
# in this case with '.temp' added to the filename.
# This file is removed when the script finishes. In this
# script, the 's/[[ ]]*/\t/' is finding all spaces and replacing them
# with TAB (\t). The 's/$HEADER_PREFIX\S*/&\t\t/' is finding all strings beginning
# with '$HEADER_PREFIX' and appending two tabs after each occurrence.
# Oh, and the 'g' tells sed to replace all occurrences, not just the first occurrence
# per line, which is the default behavior.
sed -e 's/[[ ]]*/\t/g' -e 's/$HEADER_PREFIX\S*/&\t\t/' $DATA_FILE > $DATA_FILE.temp

# Cut datasets from input file into separate temporary files, named as xx##.
# The '-k' option leaves the temp files in place in the case of an error.
# The '-s' option silences the default byte counts that csplit offers.
# The '-z' option deletes any output files that are empty.
# csplit cuts the data sets based on the search string, in this case:
# whatever you put as the second argument to the script.
csplit -k -s -z $DATA_FILE.temp /^$HEADER_PREFIX/ "{*}"

# Paste temporary files into output, piped through 'column' to create columns.
# NOTE: there is a TAB inside -s" ".
# The 'paste' command pastes multiple files into one, with the contents of all files side by side.
paste xx* | column -s"      " -t > $DATA_FILE.out

#Remove the temporary files.
rm xx*
rm $DATA_FILE.temp

# Exit
exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

extract data from html tables

hi i need to use unix to extract data from several rows of a table coded in html. I know that rows within a table have the tags <tr> </tr> and so i thought that my first step should be to to delete all of the other html code which is not contained within these tags. i could then use this method... (8 Replies)
Discussion started by: Streetrcr
8 Replies

2. Shell Programming and Scripting

Reading data from multiple tables from Oracle DB

Hi , I want to read the data from 9 tables in oracle DB into 9 different files in the same connection instance (session). I am able to get data from one table to one file with below code : X=`sqlplus -s user/pwd@DB <<eof select col1 from table1; EXIT; eof` echo $X>myfile Can anyone... (2 Replies)
Discussion started by: net
2 Replies

3. UNIX Desktop Questions & Answers

Extracting data from tables and storing in a file

Hi I am trying to write a script to extract information from a database and save that info to a csv file. I am using sql, an oracle database I have no idea how to even begin this. Can somebody please help. (8 Replies)
Discussion started by: ladyAnne
8 Replies

4. Shell Programming and Scripting

Shell script for comparing data of tables

Hi, I have two databases with same tables on different servers.I need to check the data content in each table and if something is missing, should print that. I have a tool which takes the snapshot the table structure,index so on and compares with the other server tables snapshot. Now i need... (1 Reply)
Discussion started by: nessj
1 Replies

5. Shell Programming and Scripting

Help converting row data to columns

I've been trying to figure this out for a while but I'm completely stumped. I have files with data in rows and I need to convert the data to columns. Each record contains four rows with a "field name: value" pair. I would like to convert it to four columns with the field names as column headers... (5 Replies)
Discussion started by: happy_ee
5 Replies

6. Shell Programming and Scripting

Extracting data from tables......

HOw to extracts data from tables in database. Merges them into one output file. This output file is loaded into another tables in database. (1 Reply)
Discussion started by: nari.bommi
1 Replies

7. Shell Programming and Scripting

Getting input data from 2 tables

Hello All, I have one table contains: Table1: 5 10 30 40 60 80 ... Table 2: 10 20 60 80 (7 Replies)
Discussion started by: krsnadasa
7 Replies

8. Shell Programming and Scripting

Append data by looking up 2 tables for multiple files

I want to lookup values from two different tables based on common columns and append. The trick is the column to be looked up is not fixed and varies , so it has to be detected from the header. How can I achieve this at once, for multiple data files, but lookup tables fixed. The two lookup... (5 Replies)
Discussion started by: ritakadm
5 Replies

9. Shell Programming and Scripting

UPDATE COmmand post comparing 2 columns in 2 mysql tables

my queryis : select distinct m.name, item_count, item from master m join client p on m.name=p.name where item_count = 1 and item > 1; But how should I update them? i used update statetment : Update from client Set item =1 where m.name=p.name and item_count=1 AND item>1 Is this wrong? (1 Reply)
Discussion started by: siya@
1 Replies

10. Shell Programming and Scripting

Script writing for tables with binary data

Hi There, I'm trying to write a simple script that will email me when we have an application job in a certain status that needs human intervention. I've used this script for other tables and it works great. However, this one gives me the warning that there is binary data so it might not. ... (2 Replies)
Discussion started by: Colel2
2 Replies
PASTE(1)						    BSD General Commands Manual 						  PASTE(1)

NAME
paste -- merge corresponding or subsequent lines of files SYNOPSIS
paste [-s] [-d list] file ... DESCRIPTION
The paste utility concatenates the corresponding lines of the given input files, replacing all but the last file's newline characters with a single tab character, and writes the resulting lines to standard output. If end-of-file is reached on an input file while other input files still contain data, the file is treated as if it were an endless source of empty lines. The options are as follows: -d list Use one or more of the provided characters to replace the newline characters instead of the default tab. The characters in list are used circularly, i.e., when list is exhausted the first character from list is reused. This continues until a line from the last input file (in default operation) or the last line in each file (using the -s option) is displayed, at which time paste begins selecting characters from the beginning of list again. The following special characters can also be used in list: newline character tab character \ backslash character Empty string (not a null character). Any other character preceded by a backslash is equivalent to the character itself. -s Concatenate all of the lines of each separate input file in command line order. The newline character of every line except the last line in each input file is replaced with the tab character, unless otherwise specified by the -d option. If '-' is specified for one or more of the input files, the standard input is used; standard input is read one line at a time, circularly, for each instance of '-'. EXIT STATUS
The paste utility exits 0 on success, and >0 if an error occurs. EXAMPLES
List the files in the current directory in three columns: ls | paste - - - Combine pairs of lines from a file into single lines: paste -s -d ' ' myfile Number the lines in a file, similar to nl(1): sed = myfile | paste -s -d ' ' - - Create a colon-separated list of directories named bin, suitable for use in the PATH environment variable: find / -name bin -type d | paste -s -d : - SEE ALSO
cut(1), lam(1) STANDARDS
The paste utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. HISTORY
A paste command appeared in Version 32V AT&T UNIX. BSD
June 25, 2004 BSD
All times are GMT -4. The time now is 05:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy