Sponsored Content
Top Forums Shell Programming and Scripting Converting tables of row data into columns of tables Post 302127199 by justthisguy on Monday 16th of July 2007 03:52:37 PM
Old 07-16-2007
Thank y'all! drl, I largely used your example, thank you much for taking the time!

::beer:: >> drl

I've pasted a commented version of the guts of my solution, should anyone else have the same or similar questions.

It's rough edged (I need to sit down and work on handling the indenting of the header in cases other than 3 element data sets), but functional.

In the scenario I posted above, this script (let's name it 'massagetocolumn.sh') would be called as follows:

./massagetocolumn.sh data.file test_data_

Chris Larson
JustThisGuy

Code:
#!/bin/sh

# Exit if any variable is not set.
set -o nounset

# Input file:
DATA_FILE=${1}
echo "Data File: "$DATA_FILE

# Dataset Header Prefix.
HEADER_PREFIX=${2}
echo "Header Prefix: "$HEADER_PREFIX

# Make columns from space-delimited file.
# -e indicates a command, several of which can be included in one
# sed call. s indicates the string to search for, which is prefixed
# by '/'. The replacement string is prefixed by the second '/' and
# closed with a final '/'.
# Following the list of commands is the input file.
# ' > $DATA_FILE.temp" directs the output to an output file,
# in this case with '.temp' added to the filename.
# This file is removed when the script finishes. In this
# script, the 's/[[ ]]*/\t/' is finding all spaces and replacing them
# with TAB (\t). The 's/$HEADER_PREFIX\S*/&\t\t/' is finding all strings beginning
# with '$HEADER_PREFIX' and appending two tabs after each occurrence.
# Oh, and the 'g' tells sed to replace all occurrences, not just the first occurrence
# per line, which is the default behavior.
sed -e 's/[[ ]]*/\t/g' -e 's/$HEADER_PREFIX\S*/&\t\t/' $DATA_FILE > $DATA_FILE.temp

# Cut datasets from input file into separate temporary files, named as xx##.
# The '-k' option leaves the temp files in place in the case of an error.
# The '-s' option silences the default byte counts that csplit offers.
# The '-z' option deletes any output files that are empty.
# csplit cuts the data sets based on the search string, in this case:
# whatever you put as the second argument to the script.
csplit -k -s -z $DATA_FILE.temp /^$HEADER_PREFIX/ "{*}"

# Paste temporary files into output, piped through 'column' to create columns.
# NOTE: there is a TAB inside -s" ".
# The 'paste' command pastes multiple files into one, with the contents of all files side by side.
paste xx* | column -s"      " -t > $DATA_FILE.out

#Remove the temporary files.
rm xx*
rm $DATA_FILE.temp

# Exit
exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

extract data from html tables

hi i need to use unix to extract data from several rows of a table coded in html. I know that rows within a table have the tags <tr> </tr> and so i thought that my first step should be to to delete all of the other html code which is not contained within these tags. i could then use this method... (8 Replies)
Discussion started by: Streetrcr
8 Replies

2. Shell Programming and Scripting

Reading data from multiple tables from Oracle DB

Hi , I want to read the data from 9 tables in oracle DB into 9 different files in the same connection instance (session). I am able to get data from one table to one file with below code : X=`sqlplus -s user/pwd@DB <<eof select col1 from table1; EXIT; eof` echo $X>myfile Can anyone... (2 Replies)
Discussion started by: net
2 Replies

3. UNIX Desktop Questions & Answers

Extracting data from tables and storing in a file

Hi I am trying to write a script to extract information from a database and save that info to a csv file. I am using sql, an oracle database I have no idea how to even begin this. Can somebody please help. (8 Replies)
Discussion started by: ladyAnne
8 Replies

4. Shell Programming and Scripting

Shell script for comparing data of tables

Hi, I have two databases with same tables on different servers.I need to check the data content in each table and if something is missing, should print that. I have a tool which takes the snapshot the table structure,index so on and compares with the other server tables snapshot. Now i need... (1 Reply)
Discussion started by: nessj
1 Replies

5. Shell Programming and Scripting

Help converting row data to columns

I've been trying to figure this out for a while but I'm completely stumped. I have files with data in rows and I need to convert the data to columns. Each record contains four rows with a "field name: value" pair. I would like to convert it to four columns with the field names as column headers... (5 Replies)
Discussion started by: happy_ee
5 Replies

6. Shell Programming and Scripting

Extracting data from tables......

HOw to extracts data from tables in database. Merges them into one output file. This output file is loaded into another tables in database. (1 Reply)
Discussion started by: nari.bommi
1 Replies

7. Shell Programming and Scripting

Getting input data from 2 tables

Hello All, I have one table contains: Table1: 5 10 30 40 60 80 ... Table 2: 10 20 60 80 (7 Replies)
Discussion started by: krsnadasa
7 Replies

8. Shell Programming and Scripting

Append data by looking up 2 tables for multiple files

I want to lookup values from two different tables based on common columns and append. The trick is the column to be looked up is not fixed and varies , so it has to be detected from the header. How can I achieve this at once, for multiple data files, but lookup tables fixed. The two lookup... (5 Replies)
Discussion started by: ritakadm
5 Replies

9. Shell Programming and Scripting

UPDATE COmmand post comparing 2 columns in 2 mysql tables

my queryis : select distinct m.name, item_count, item from master m join client p on m.name=p.name where item_count = 1 and item > 1; But how should I update them? i used update statetment : Update from client Set item =1 where m.name=p.name and item_count=1 AND item>1 Is this wrong? (1 Reply)
Discussion started by: siya@
1 Replies

10. Shell Programming and Scripting

Script writing for tables with binary data

Hi There, I'm trying to write a simple script that will email me when we have an application job in a certain status that needs human intervention. I've used this script for other tables and it works great. However, this one gives me the warning that there is binary data so it might not. ... (2 Replies)
Discussion started by: Colel2
2 Replies
MKTEMP(1)						    BSD General Commands Manual 						 MKTEMP(1)

NAME
mktemp -- make temporary file name (unique) SYNOPSIS
mktemp [-d] [-q] [-u] template DESCRIPTION
The mktemp utility takes the given file name template and overwrites a portion of it to create a file name. This file name is unique and suitable for use by the application. The template may be any file name with at least 6 of 'Xs' appended to it, for example /tmp/temp.XXXXXX. The trailing 'Xs' are replaced with the current process number and/or a unique letter combination. The number of unique file names mktemp can return depends on the number of 'Xs' provided; six 'Xs' will result in mktemp testing roughly 26 ** 6 combinations. If mktemp can successfully generate a unique file name, the file is created with mode 0600 (unless the -u flag is given) and the filename is printed to standard output. OPTIONS
The available options are as follows: -d Make a directory instead of a file. -q Fail silently if an error occurs. This is useful if a script does not want error output to go to standard error. -u Operate in ``unsafe'' mode. The temp file will be unlinked before mktemp exits. This is slightly better than mktemp(3) but still introduces a race condition. Use of this option is not encouraged. RETURN VALUES
The mktemp utility exits with a value of 0 on success, and 1 on failure. EXAMPLES
The following sh(1) fragment illustrates a simple use of mktemp where the script should quit if it cannot get a safe temporary file. TMPFILE=`mktemp /tmp/$0.XXXXXX` || exit 1 echo "program output" >> $TMPFILE In this case, we want the script to catch the error itself. TMPFILE=`mktemp -q /tmp/$0.XXXXXX` if [ $? -ne 0 ]; then echo "$0: Can't create temp file, exiting..." exit 1 fi Note that one can also check to see that $TMPFILE is zero length instead of checking $?. This would allow the check to be done later one in the script (since $? would get clobbered by the next shell command). SEE ALSO
mkstemp(3), mktemp(3) HISTORY
The mktemp utility appeared in OpenBSD. BSD
November, 20, 1996 BSD
All times are GMT -4. The time now is 05:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy