Sponsored Content
Top Forums Shell Programming and Scripting How to parse parts of 1 column into two separate columns? Post 302928034 by craigwg on Monday 8th of December 2014 11:59:37 AM
Old 12-08-2014
How to parse parts of 1 column into two separate columns?

I have a shell script that is currently transferring a csv file from a server into a Teradata database table. One of the 30 or so columns is called "destination_url". In that URL there are parameters, and it is possible for those parameters to be repeated because of referring companies copying referred URLs from one another. So, for example one of these urls might look like this:

http://othercompany.com/media/redir.php?prof=403&camp=3&affcode=kw1266732&cid={creative}&networkType={ifContent:content}&url[]=http://www.mycompany.com/search?keywords=shelf&taxonomy=dep32&cid=128513&kid=169896850098703&TRACK=PSGGL

My task is to take the cid value and kid value and pull them out and place them in two columns. The two new columns will be called cid_value and kid_value. I should point out that the source destination_url will remain unchanged. I just need the two values from it.

The problem is that the urls SOMETIMES have more than one cid and/or kid. So usually I need to pull the first instance of the cid and kid. cid and kid will always be numeric. cid will always have 7 digits. kid can have anywhere from 6 to 14 digits.

I know this is possible to do in shell but I have NO idea how to do this. Can someone help? I'm new to shell scripting. Feeling totally lost and without help. I know it will be using sed and awk and I'm trying to read about them but nothing is clicking yet.

Last edited by craigwg; 12-08-2014 at 01:25 PM.. Reason: Clarity
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting columns from a matrix and storing each column in a separate file

Hi All, I have a huge matrix file consisting some some millions rows and 6000 columns. The contents are just floating point numbers in the matrix. I want to extract each column (i.e. 6000 of them) and store each column in a separate file. For example, 1.dat will consist of elements from column... (4 Replies)
Discussion started by: shoaibjameel123
4 Replies

2. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Hey guys, Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have: (a_b_c.txt) WB34_2_SLA8.txt WB34_1_SLA8.txt WB34_1_DB10.txt WB34_2_DB10.txt WB34_1_SLA8.txt WB34_2_SLA8.txt 77_1_SLA8.txt... (1 Reply)
Discussion started by: Breentax
1 Replies

3. UNIX for Advanced & Expert Users

Parse (delimited string) key-value pairs in a column into separate lines

Hi experts, e.g. i/p data looks like 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified|| o/p data should like - row1: 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747|unidentified ... (1 Reply)
Discussion started by: sumoka
1 Replies

4. UNIX for Advanced & Expert Users

Parse key-value pair into separate rows

Hi, I'm getting key-value pairs in a string as follows - 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified I need output as follows - row1:... (5 Replies)
Discussion started by: sumoka
5 Replies

5. Shell Programming and Scripting

Print every 5 4th column values as separate row with different first column

Hi, I have the following file, chr1 100 200 20 chr1 201 300 22 chr1 220 345 23 chr1 230 456 33.5 chr1 243 567 90 chr1 345 600 20 chr1 430 619 21.78 chr1 870 910 112.3 chr1 914 920 12 chr1 930 999 13 My output would be peak1 20 22 23 33.5 90 peak2 20 21.78 112.3 12 13 Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

Need help with awk statement to break nth column in csv file into 3 separate columns

Hello Members, I have a csv file in the format below. Need help with awk statement to break nth column into 3 separate columns and export the changes to new file. input file --> file.csv cat file.csv|less "product/fruit/mango","location/asia/india","type/alphonso" need output in... (2 Replies)
Discussion started by: awk-admirer
2 Replies

7. UNIX for Dummies Questions & Answers

How to generate one long column by merging two separate two columns in a single file?

Dear all, I have a simple question. I have a file like below (separated by tab): col1 col2 col3 col4 col5 col6 col7 21 66745 rs1234 21 rs5678 23334 0.89 21 66745 rs2334 21 rs9978 23334 0.89 21 66745 ... (4 Replies)
Discussion started by: forevertl
4 Replies

8. Shell Programming and Scripting

Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All, I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns. So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies

9. Shell Programming and Scripting

Separate a hash variable into 2 parts in Perl

Dear Perl users/experts, Could somebody help me how to solve my problem, I have a hash variable that I want to convert into dot file (graphviz). I know how to convert it to dot file but I need some modification on the output of the hash variable before convert it to dot file. Eeach key of... (1 Reply)
Discussion started by: askari
1 Replies

10. Shell Programming and Scripting

Improve awk code that has three separate parts

I have a very inefficient awk below that I need some help improving. Basically, there are three parts, that ideally, could be combined into one search and one output file. Thank you :). Part 1: Check if the user inputted string contains + or - in it and if it does the input is writting to a... (4 Replies)
Discussion started by: cmccabe
4 Replies
SQLLINE:(1)							   User Commands						       SQLLINE:(1)

NAME
sqlline - JDBC command-line utility for issuing SQL SYNOPSIS
sqlline [ option... ] DESCRIPTION
-u <database url> the JDBC URL to connect to -n <username> the username to connect as -p <password> the password to connect as -d <driver class> the driver class to use --color=[true/false] control whether color is used for display --showHeader=[true/false] show column names in query results --headerInterval=ROWS; the interval between which heades are displayed --fastConnect=[true/false] skip building table/column list for tab-completion --autoCommit=[true/false] enable/disable automatic transaction commit --verbose=[true/false] show verbose error messages and debug info --showWarnings=[true/false] display connection warnings --force=[true/false] continue running script even after errors --maxWidth=MAXWIDTH the maximum width of the terminal --maxColumnWidth=MAXCOLWIDTH the maximum width to use when displaying columns --silent=[true/false] be more silent --autosave=[true/false] automatically save preferences --outputformat=[table/vertical/csv/tsv] format mode for result display --isolation=LEVEL set the transaction isolation level --help display this message Usage: java sqlline.SqlLine -u <database url> the JDBC URL to connect to -n <username> the username to connect as -p <password> the password to connect as -d <driver class> the driver class to use --color=[true/false] control whether color is used for display --showHeader=[true/false] show column names in query results --headerInterval=ROWS; the interval between which heades are displayed --fastConnect=[true/false] skip building table/column list for tab-completion --autoCommit=[true/false] enable/disable automatic transaction commit --verbose=[true/false] show verbose error messages and debug info --showWarnings=[true/false] display connection warnings --force=[true/false] continue running script even after errors --maxWidth=MAXWIDTH the maximum width of the terminal --maxColumnWidth=MAXCOLWIDTH the maximum width to use when displaying columns --silent=[true/false] be more silent --autosave=[true/false] automatically save preferences --outputformat=[table/vertical/csv/tsv] format mode for result display --isolation=LEVEL set the transaction isolation level --help display this message -u <database url> the JDBC URL to connect to -n <username> the username to connect as -p <password> the password to connect as -d <driver class> the driver class to use --color=[true/false] control whether color is used for display --showHeader=[true/false] show column names in query results --headerInterval=ROWS; the interval between which heades are displayed --fastConnect=[true/false] skip building table/column list for tab-completion --autoCommit=[true/false] enable/disable automatic transaction commit --verbose=[true/false] show verbose error messages and debug info --showWarnings=[true/false] display connection warnings --force=[true/false] continue running script even after errors --maxWidth=MAXWIDTH the maximum width of the terminal --maxColumnWidth=MAXCOLWIDTH the maximum width to use when displaying columns --silent=[true/false] be more silent --autosave=[true/false] automatically save preferences --outputformat=[table/vertical/csv/tsv] format mode for result display --isolation=LEVEL set the transaction isolation level --help display this message sqlline August 2008 SQLLINE:(1)
All times are GMT -4. The time now is 06:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy