Sponsored Content
Top Forums Shell Programming and Scripting Converting tables of row data into columns of tables Post 302127199 by justthisguy on Monday 16th of July 2007 03:52:37 PM
Old 07-16-2007
Thank y'all! drl, I largely used your example, thank you much for taking the time!

::beer:: >> drl

I've pasted a commented version of the guts of my solution, should anyone else have the same or similar questions.

It's rough edged (I need to sit down and work on handling the indenting of the header in cases other than 3 element data sets), but functional.

In the scenario I posted above, this script (let's name it 'massagetocolumn.sh') would be called as follows:

./massagetocolumn.sh data.file test_data_

Chris Larson
JustThisGuy

Code:
#!/bin/sh

# Exit if any variable is not set.
set -o nounset

# Input file:
DATA_FILE=${1}
echo "Data File: "$DATA_FILE

# Dataset Header Prefix.
HEADER_PREFIX=${2}
echo "Header Prefix: "$HEADER_PREFIX

# Make columns from space-delimited file.
# -e indicates a command, several of which can be included in one
# sed call. s indicates the string to search for, which is prefixed
# by '/'. The replacement string is prefixed by the second '/' and
# closed with a final '/'.
# Following the list of commands is the input file.
# ' > $DATA_FILE.temp" directs the output to an output file,
# in this case with '.temp' added to the filename.
# This file is removed when the script finishes. In this
# script, the 's/[[ ]]*/\t/' is finding all spaces and replacing them
# with TAB (\t). The 's/$HEADER_PREFIX\S*/&\t\t/' is finding all strings beginning
# with '$HEADER_PREFIX' and appending two tabs after each occurrence.
# Oh, and the 'g' tells sed to replace all occurrences, not just the first occurrence
# per line, which is the default behavior.
sed -e 's/[[ ]]*/\t/g' -e 's/$HEADER_PREFIX\S*/&\t\t/' $DATA_FILE > $DATA_FILE.temp

# Cut datasets from input file into separate temporary files, named as xx##.
# The '-k' option leaves the temp files in place in the case of an error.
# The '-s' option silences the default byte counts that csplit offers.
# The '-z' option deletes any output files that are empty.
# csplit cuts the data sets based on the search string, in this case:
# whatever you put as the second argument to the script.
csplit -k -s -z $DATA_FILE.temp /^$HEADER_PREFIX/ "{*}"

# Paste temporary files into output, piped through 'column' to create columns.
# NOTE: there is a TAB inside -s" ".
# The 'paste' command pastes multiple files into one, with the contents of all files side by side.
paste xx* | column -s"      " -t > $DATA_FILE.out

#Remove the temporary files.
rm xx*
rm $DATA_FILE.temp

# Exit
exit 0

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

extract data from html tables

hi i need to use unix to extract data from several rows of a table coded in html. I know that rows within a table have the tags <tr> </tr> and so i thought that my first step should be to to delete all of the other html code which is not contained within these tags. i could then use this method... (8 Replies)
Discussion started by: Streetrcr
8 Replies

2. Shell Programming and Scripting

Reading data from multiple tables from Oracle DB

Hi , I want to read the data from 9 tables in oracle DB into 9 different files in the same connection instance (session). I am able to get data from one table to one file with below code : X=`sqlplus -s user/pwd@DB <<eof select col1 from table1; EXIT; eof` echo $X>myfile Can anyone... (2 Replies)
Discussion started by: net
2 Replies

3. UNIX Desktop Questions & Answers

Extracting data from tables and storing in a file

Hi I am trying to write a script to extract information from a database and save that info to a csv file. I am using sql, an oracle database I have no idea how to even begin this. Can somebody please help. (8 Replies)
Discussion started by: ladyAnne
8 Replies

4. Shell Programming and Scripting

Shell script for comparing data of tables

Hi, I have two databases with same tables on different servers.I need to check the data content in each table and if something is missing, should print that. I have a tool which takes the snapshot the table structure,index so on and compares with the other server tables snapshot. Now i need... (1 Reply)
Discussion started by: nessj
1 Replies

5. Shell Programming and Scripting

Help converting row data to columns

I've been trying to figure this out for a while but I'm completely stumped. I have files with data in rows and I need to convert the data to columns. Each record contains four rows with a "field name: value" pair. I would like to convert it to four columns with the field names as column headers... (5 Replies)
Discussion started by: happy_ee
5 Replies

6. Shell Programming and Scripting

Extracting data from tables......

HOw to extracts data from tables in database. Merges them into one output file. This output file is loaded into another tables in database. (1 Reply)
Discussion started by: nari.bommi
1 Replies

7. Shell Programming and Scripting

Getting input data from 2 tables

Hello All, I have one table contains: Table1: 5 10 30 40 60 80 ... Table 2: 10 20 60 80 (7 Replies)
Discussion started by: krsnadasa
7 Replies

8. Shell Programming and Scripting

Append data by looking up 2 tables for multiple files

I want to lookup values from two different tables based on common columns and append. The trick is the column to be looked up is not fixed and varies , so it has to be detected from the header. How can I achieve this at once, for multiple data files, but lookup tables fixed. The two lookup... (5 Replies)
Discussion started by: ritakadm
5 Replies

9. Shell Programming and Scripting

UPDATE COmmand post comparing 2 columns in 2 mysql tables

my queryis : select distinct m.name, item_count, item from master m join client p on m.name=p.name where item_count = 1 and item > 1; But how should I update them? i used update statetment : Update from client Set item =1 where m.name=p.name and item_count=1 AND item>1 Is this wrong? (1 Reply)
Discussion started by: siya@
1 Replies

10. Shell Programming and Scripting

Script writing for tables with binary data

Hi There, I'm trying to write a simple script that will email me when we have an application job in a certain status that needs human intervention. I've used this script for other tables and it works great. However, this one gives me the warning that there is binary data so it might not. ... (2 Replies)
Discussion started by: Colel2
2 Replies
CSPLIT(1)						    BSD General Commands Manual 						 CSPLIT(1)

NAME
csplit -- split files based on context SYNOPSIS
csplit [-ks] [-f prefix] [-n number] file args ... DESCRIPTION
The csplit utility splits file into pieces using the patterns args. If file is a dash ('-'), csplit reads from standard input. The options are as follows: -f prefix Give created files names beginning with prefix. The default is ``xx''. -k Do not remove output files if an error occurs or a HUP, INT or TERM signal is received. -n number Use number of decimal digits after the prefix to form the file name. The default is 2. -s Do not write the size of each output file to standard output as it is created. The args operands may be a combination of the following patterns: /regexp/[[+|-]offset] Create a file containing the input from the current line to (but not including) the next line matching the given basic regular expression. An optional offset from the line that matched may be specified. %regexp%[[+|-]offset] Same as above but a file is not created for the output. line_no Create containing the input from the current line to (but not including) the specified line number. {num} Repeat the previous pattern the specified number of times. If it follows a line number pattern, a new file will be created for each line_no lines, num times. The first line of the file is line number 1 for historic reasons. After all the patterns have been processed, the remaining input data (if there is any) will be written to a new file. Requesting to split at a line before the current line number or past the end of the file will result in an error. ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of csplit as described in environ(7). EXIT STATUS
The csplit utility exits 0 on success, and >0 if an error occurs. EXAMPLES
Split the mdoc(7) file foo.1 into one file for each section (up to 20): csplit -k foo.1 '%^.Sh%' '/^.Sh/' '{20}' Split standard input after the first 99 lines and every 100 lines thereafter: csplit -k - 100 '{19}' SEE ALSO
sed(1), split(1), re_format(7) STANDARDS
The csplit utility conforms to IEEE Std 1003.1-2001 (``POSIX.1''). HISTORY
A csplit command appeared in PWB UNIX. BUGS
Input lines are limited to LINE_MAX (2048) bytes in length. BSD
January 26, 2005 BSD
All times are GMT -4. The time now is 02:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy