Sponsored Content
Top Forums Shell Programming and Scripting Shell script for .Txt to .csv conversion with data processing Post 302930488 by ongoto on Monday 5th of January 2015 10:42:56 AM
Old 01-05-2015
A program which extracts the values you want to see shouldn't be hard to do, but you will have to edit the parameters.txt file yourself and provide the values you want to see, and omit what you dont want to see. The script can only extract the values you offer. It can't do the editing. In order for scripts to work, the data types and format have to be consistent.

You need to use single words with no spaces, or use numeric values, in the values column:
HSN AsperPlan # would be good (camel case)

Code:
# The following types can be a problem for Linux:
CS 1 for non MBC
CHAP 6 for MBC 
CHAP 11 for BCCH on OL
etc.
# Better would be CS_1_for_non_MBC, etc. (or camel case)

If you want to use 100 in place of Ranges, then you will have to make those changes yourself, and give us (or yourself) a modified parameters.txt file to work from.

The field names in parameters.txt that are not found in CNAI_DUMP_RAW1.txt should be omited from parameters.txt.

---------- Post updated 01-05-15 at 07:42 AM ---------- Previous update was 01-04-15 at 11:17 PM ----------

First attempt in bash. Awk would be much faster I'm sure.
Code:
#!/bin/bash
###############################################################################
# Use these commands to strip carriage returns from input files
# sed -i 's|\r$||g' CNAI_DUMP_RAW1.txt
# sed -i 's|\r$||g' parameters.txt
###############################################################################
dumpfil="./CNAI_DUMP_RAW1.txt"
parmsfil="./parameters.txt"
outfil="./csv.out"
> $outfil

hdr=($(head -n 1 $dumpfil))
lnum=0
while read raw
do
    lnum=$(( $lnum + 1 ))
    [ $lnum -lt 3 ] && continue     # skip header and row separators
    while read parm1 parm2
    do
        fnum=0  # lookup actual values
        for fld in ${hdr[*]};
        do
            fnum=$(( $fnum + 1 ))
            if [[ $fld == ${parm1,,} ]]; then   # field headers are lower case
                break
            fi
        done
        cell=$(echo $raw | cut -d ' ' -f 4 2>&1)
        act_val=$(echo $raw | cut -d ' ' -f $fnum 2>&1)
        printf "%s,%s,%s,%s\n" "$parm1" "$parm2" "$cell" "$act_val" | tee -a $outfil
    done < $parmsfil
done < $dumpfil

# output
# ------
# CCMIN,RANGES,RBR201A,100
# CCHPWR,33,RBR201A,33
# CCHPWR,30,RBR201A,33
# ATT,YES,RBR201A,YES
# CB,NO,RBR201A,NO
# CBQ,HIGH,RBR201A,HIGH
# ...
# ...
# etc.

# real    2m8.955s
# user    1m59.455s
# sys   0m12.274s


Last edited by ongoto; 01-05-2015 at 11:39 AM..
This User Gave Thanks to ongoto For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to format a .CSV data

Hi There I needed to write a Unix shell script which will pick up the data from a .CSV file and reformat it as per the requirement and write it to another .CSV file. Currently I am in the proess of Data Import to "Remedy System" (A one kind of incident mangement Application) and this... (8 Replies)
Discussion started by: Uday1982
8 Replies

2. Shell Programming and Scripting

shell script to insert data from gps.txt to mysql database

Hi, I have gps receiver, by using gpsd data i can read gps log data to my database(my sql). Steps: 1. telenet localhost 2947 > gps.txt (press enter) 2. r (press enter) //then i will get the data like below in gps.txt file Trying 127.0.0.1... Connected to localhost.... (1 Reply)
Discussion started by: gudivada213
1 Replies

3. UNIX for Advanced & Expert Users

shell script to format .CSV data

Hi all, I have written a shell script to search a specified directory (e.g. /home/user) for a list of specific words (shown as ${TMPDIR}/wordlist below). The script works well enough, but I was wondering if there was a way to display the line number that the word is found on? Thanks! cat... (1 Reply)
Discussion started by: tmcmurtr
1 Replies

4. Shell Programming and Scripting

Shell script for CSV conversion

thanks for allowing me join your forum i have an output of linux command "who" which provides following details..... CURRENT USER/ACCT INFO 17:31:36 up 4:49, 4 users, load average: 0.03, 0.04, 0.00 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT root :0 - 12:59 ?xdm? 4:54 0.02s /bin/sh /usr/bi... (1 Reply)
Discussion started by: ayyappancheta
1 Replies

5. Shell Programming and Scripting

Conversion of below Tabs Tex file into CSV format file : shell script needed

Request if some one could provide me shell script that converts the below "input file" to "CSV format file" given Name Domain Contact Phone Email Location ----------------------- ------------------------------------------------ ------- ----- ---------------------------------... (7 Replies)
Discussion started by: sreenath1037
7 Replies

6. Shell Programming and Scripting

Conversion of xhtml data into csv format using dump utility

Hi Unix Gurus, I tried to convert the attached xhtml table content into csv file using unix shell script (lynx -dump filename) and got the below results: Title ID Owner Priority Estimate Project Change Date Changed By Complexity Create Date Created By Detail Estimate Total De tail... (6 Replies)
Discussion started by: bi.infa
6 Replies

7. Shell Programming and Scripting

Copy data to CSV file from txt output

Hi Everyone , Below is output from a TL1 , I want just the NE Name: and beside that the Temperature and the voltages in a csv file , Is this possible? > act-user:AB1S2a:ArshadFO:493::**********; AB1S2a 2016-02-07 10:13:24 M 493 COMPLD "ArshadFO:2016-02-07 10-04-55,0" ;... (11 Replies)
Discussion started by: adgjmpt
11 Replies

8. Shell Programming and Scripting

Help with Shell Scrip in Masking particular columns in .csv file or .txt file using shell script

Hello Unix Shell Script Experts, I have a script that would mask the columns in .csv file or .txt file. First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder. Two parameters are passed ... (5 Replies)
Discussion started by: Mahesh G
5 Replies

9. Shell Programming and Scripting

How to get the shell script to read the .txt file as an input/data?

i have written my shell script in notepad however i am struggling to pass the data file to be read to the script the data file is of .txt format. My target is to run the shell script from the terminal and pass 3 arguments e.g. polg@DESKTOP-BVPDC5C:~/CS1420/coursework$ bash valsplit.sh input.txt... (11 Replies)
Discussion started by: Gurdza32
11 Replies
All times are GMT -4. The time now is 05:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy