Sponsored Content
Top Forums Shell Programming and Scripting How to parse parts of 1 column into two separate columns? Post 302928034 by craigwg on Monday 8th of December 2014 11:59:37 AM
Old 12-08-2014
How to parse parts of 1 column into two separate columns?

I have a shell script that is currently transferring a csv file from a server into a Teradata database table. One of the 30 or so columns is called "destination_url". In that URL there are parameters, and it is possible for those parameters to be repeated because of referring companies copying referred URLs from one another. So, for example one of these urls might look like this:

http://othercompany.com/media/redir.php?prof=403&camp=3&affcode=kw1266732&cid={creative}&networkType={ifContent:content}&url[]=http://www.mycompany.com/search?keywords=shelf&taxonomy=dep32&cid=128513&kid=169896850098703&TRACK=PSGGL

My task is to take the cid value and kid value and pull them out and place them in two columns. The two new columns will be called cid_value and kid_value. I should point out that the source destination_url will remain unchanged. I just need the two values from it.

The problem is that the urls SOMETIMES have more than one cid and/or kid. So usually I need to pull the first instance of the cid and kid. cid and kid will always be numeric. cid will always have 7 digits. kid can have anywhere from 6 to 14 digits.

I know this is possible to do in shell but I have NO idea how to do this. Can someone help? I'm new to shell scripting. Feeling totally lost and without help. I know it will be using sed and awk and I'm trying to read about them but nothing is clicking yet.

Last edited by craigwg; 12-08-2014 at 01:25 PM.. Reason: Clarity
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting columns from a matrix and storing each column in a separate file

Hi All, I have a huge matrix file consisting some some millions rows and 6000 columns. The contents are just floating point numbers in the matrix. I want to extract each column (i.e. 6000 of them) and store each column in a separate file. For example, 1.dat will consist of elements from column... (4 Replies)
Discussion started by: shoaibjameel123
4 Replies

2. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Hey guys, Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have: (a_b_c.txt) WB34_2_SLA8.txt WB34_1_SLA8.txt WB34_1_DB10.txt WB34_2_DB10.txt WB34_1_SLA8.txt WB34_2_SLA8.txt 77_1_SLA8.txt... (1 Reply)
Discussion started by: Breentax
1 Replies

3. UNIX for Advanced & Expert Users

Parse (delimited string) key-value pairs in a column into separate lines

Hi experts, e.g. i/p data looks like 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified|| o/p data should like - row1: 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747|unidentified ... (1 Reply)
Discussion started by: sumoka
1 Replies

4. UNIX for Advanced & Expert Users

Parse key-value pair into separate rows

Hi, I'm getting key-value pairs in a string as follows - 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified I need output as follows - row1:... (5 Replies)
Discussion started by: sumoka
5 Replies

5. Shell Programming and Scripting

Print every 5 4th column values as separate row with different first column

Hi, I have the following file, chr1 100 200 20 chr1 201 300 22 chr1 220 345 23 chr1 230 456 33.5 chr1 243 567 90 chr1 345 600 20 chr1 430 619 21.78 chr1 870 910 112.3 chr1 914 920 12 chr1 930 999 13 My output would be peak1 20 22 23 33.5 90 peak2 20 21.78 112.3 12 13 Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

Need help with awk statement to break nth column in csv file into 3 separate columns

Hello Members, I have a csv file in the format below. Need help with awk statement to break nth column into 3 separate columns and export the changes to new file. input file --> file.csv cat file.csv|less "product/fruit/mango","location/asia/india","type/alphonso" need output in... (2 Replies)
Discussion started by: awk-admirer
2 Replies

7. UNIX for Dummies Questions & Answers

How to generate one long column by merging two separate two columns in a single file?

Dear all, I have a simple question. I have a file like below (separated by tab): col1 col2 col3 col4 col5 col6 col7 21 66745 rs1234 21 rs5678 23334 0.89 21 66745 rs2334 21 rs9978 23334 0.89 21 66745 ... (4 Replies)
Discussion started by: forevertl
4 Replies

8. Shell Programming and Scripting

Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All, I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns. So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies

9. Shell Programming and Scripting

Separate a hash variable into 2 parts in Perl

Dear Perl users/experts, Could somebody help me how to solve my problem, I have a hash variable that I want to convert into dot file (graphviz). I know how to convert it to dot file but I need some modification on the output of the hash variable before convert it to dot file. Eeach key of... (1 Reply)
Discussion started by: askari
1 Replies

10. Shell Programming and Scripting

Improve awk code that has three separate parts

I have a very inefficient awk below that I need some help improving. Basically, there are three parts, that ideally, could be combined into one search and one output file. Thank you :). Part 1: Check if the user inputted string contains + or - in it and if it does the input is writting to a... (4 Replies)
Discussion started by: cmccabe
4 Replies
PTHREAD_GETCPUCLOCKID(3)				     Linux Programmer's Manual					  PTHREAD_GETCPUCLOCKID(3)

NAME
pthread_getcpuclockid - retrieve ID of a thread's CPU time clock SYNOPSIS
#include <pthread.h> #include <time.h> int pthread_getcpuclockid(pthread_t thread, clockid_t *clock_id); Compile and link with -pthread. DESCRIPTION
The pthread_getcpuclockid() function returns the clock ID for the CPU time clock of the thread thread. RETURN VALUE
On success, this function returns 0; on error, it returns a nonzero error number. ERRORS
ENOENT Per-thread CPU time clocks are not supported by the system. ESRCH No thread with the ID thread could be found. VERSIONS
This function is available in glibc since version 2.2. CONFORMING TO
POSIX.1-2001. NOTES
When thread refers to the calling thread, this function returns an identifier that refers to the same clock manipulated by clock_gettime(2) and clock_settime(2) when given the clock ID CLOCK_THREAD_CPUTIME_ID. EXAMPLE
The program below creates a thread and then uses clock_gettime(2) to retrieve the total process CPU time, and the per-thread CPU time con- sumed by the two threads. The following shell session shows an example run: $ ./a.out Main thread sleeping Subthread starting infinite loop Main thread consuming some CPU time... Process total CPU time: 1.368 Main thread CPU time: 0.376 Subthread CPU time: 0.992 Program source /* Link with "-lrt" */ #include <time.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <pthread.h> #include <string.h> #include <errno.h> #define handle_error(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0) #define handle_error_en(en, msg) do { errno = en; perror(msg); exit(EXIT_FAILURE); } while (0) static void * thread_start(void *arg) { printf("Subthread starting infinite loop "); for (;;) continue; } static void pclock(char *msg, clockid_t cid) { struct timespec ts; printf("%s", msg); if (clock_gettime(cid, &ts) == -1) handle_error("clock_gettime"); printf("%4ld.%03ld ", ts.tv_sec, ts.tv_nsec / 1000000); } int main(int argc, char *argv[]) { pthread_t thread; clockid_t cid; int j, s; s = pthread_create(&thread, NULL, thread_start, NULL); if (s != 0) handle_error_en(s, "pthread_create"); printf("Main thread sleeping "); sleep(1); printf("Main thread consuming some CPU time... "); for (j = 0; j < 2000000; j++) getppid(); pclock("Process total CPU time: ", CLOCK_PROCESS_CPUTIME_ID); s = pthread_getcpuclockid(pthread_self(), &cid); if (s != 0) handle_error_en(s, "pthread_getcpuclockid"); pclock("Main thread CPU time: ", cid); /* The preceding 4 lines of code could have been replaced by: pclock("Main thread CPU time: ", CLOCK_THREAD_CPUTIME_ID); */ s = pthread_getcpuclockid(thread, &cid); if (s != 0) handle_error_en(s, "pthread_getcpuclockid"); pclock("Subthread CPU time: 1 ", cid); exit(EXIT_SUCCESS); /* Terminates both threads */ } SEE ALSO
clock_gettime(2), clock_settime(2), timer_create(2), clock_getcpuclockid(3), pthread_self(3), pthreads(7), time(7) COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2009-02-08 PTHREAD_GETCPUCLOCKID(3)
All times are GMT -4. The time now is 07:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy