12-08-2014
How to parse parts of 1 column into two separate columns?
I have a shell script that is currently transferring a csv file from a server into a Teradata database table. One of the 30 or so columns is called "destination_url". In that URL there are parameters, and it is possible for those parameters to be repeated because of referring companies copying referred URLs from one another. So, for example one of these urls might look like this:
http://othercompany.com/media/redir.php?prof=403&camp=3&affcode=kw1266732&cid={creative}&networkType={ifContent:content}&url[]=http://www.mycompany.com/search?keywords=shelf&taxonomy=dep32&cid=128513&kid=169896850098703&TRACK=PSGGL
My task is to take the cid value and kid value and pull them out and place them in two columns. The two new columns will be called cid_value and kid_value. I should point out that the source destination_url will remain unchanged. I just need the two values from it.
The problem is that the urls SOMETIMES have more than one cid and/or kid. So usually I need to pull the first instance of the cid and kid. cid and kid will always be numeric. cid will always have 7 digits. kid can have anywhere from 6 to 14 digits.
I know this is possible to do in shell but I have NO idea how to do this. Can someone help? I'm new to shell scripting. Feeling totally lost and without help. I know it will be using sed and awk and I'm trying to read about them but nothing is clicking yet.
Last edited by craigwg; 12-08-2014 at 01:25 PM..
Reason: Clarity
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I have a huge matrix file consisting some some millions rows and 6000 columns. The contents are just floating point numbers in the matrix. I want to extract each column (i.e. 6000 of them) and store each column in a separate file.
For example, 1.dat will consist of elements from column... (4 Replies)
Discussion started by: shoaibjameel123
4 Replies
2. Shell Programming and Scripting
Hey guys,
Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have:
(a_b_c.txt)
WB34_2_SLA8.txt
WB34_1_SLA8.txt
WB34_1_DB10.txt
WB34_2_DB10.txt
WB34_1_SLA8.txt
WB34_2_SLA8.txt
77_1_SLA8.txt... (1 Reply)
Discussion started by: Breentax
1 Replies
3. UNIX for Advanced & Expert Users
Hi experts,
e.g.
i/p data looks like
0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified||
o/p data should like -
row1: 0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747|unidentified
... (1 Reply)
Discussion started by: sumoka
1 Replies
4. UNIX for Advanced & Expert Users
Hi,
I'm getting key-value pairs in a string as follows -
0000xm7zcNDIkP888vRqGv93xA7:176n00qql||9700005405552747,9700005405717924,9700005405733788|unidentified,unidentified,unidentified
I need output as follows -
row1:... (5 Replies)
Discussion started by: sumoka
5 Replies
5. Shell Programming and Scripting
Hi,
I have the following file,
chr1 100 200 20
chr1 201 300 22
chr1 220 345 23
chr1 230 456 33.5
chr1 243 567 90
chr1 345 600 20
chr1 430 619 21.78
chr1 870 910 112.3
chr1 914 920 12
chr1 930 999 13
My output would be
peak1 20 22 23 33.5 90
peak2 20 21.78 112.3 12 13
Here the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies
6. Shell Programming and Scripting
Hello Members,
I have a csv file in the format below. Need help with awk statement to break nth column into 3 separate columns and export the changes to new file.
input file --> file.csv
cat file.csv|less
"product/fruit/mango","location/asia/india","type/alphonso"
need output in... (2 Replies)
Discussion started by: awk-admirer
2 Replies
7. UNIX for Dummies Questions & Answers
Dear all,
I have a simple question. I have a file like below (separated by tab):
col1 col2 col3 col4 col5 col6 col7
21 66745 rs1234 21 rs5678 23334 0.89
21 66745 rs2334 21 rs9978 23334 0.89
21 66745 ... (4 Replies)
Discussion started by: forevertl
4 Replies
8. Shell Programming and Scripting
Hello All,
I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns.
So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies
9. Shell Programming and Scripting
Dear Perl users/experts,
Could somebody help me how to solve my problem, I have a hash variable that I want to convert into dot file (graphviz).
I know how to convert it to dot file but I need some modification on the output of the hash variable before convert it to dot file.
Eeach key of... (1 Reply)
Discussion started by: askari
1 Replies
10. Shell Programming and Scripting
I have a very inefficient awk below that I need some help improving. Basically, there are three parts, that ideally, could be combined into one search and one output file. Thank you :).
Part 1:
Check if the user inputted string contains + or - in it and if it does the input is writting to a... (4 Replies)
Discussion started by: cmccabe
4 Replies
LEARN ABOUT SUSE
pthread_getcpuclockid
PTHREAD_GETCPUCLOCKID(3) Linux Programmer's Manual PTHREAD_GETCPUCLOCKID(3)
NAME
pthread_getcpuclockid - retrieve ID of a thread's CPU time clock
SYNOPSIS
#include <pthread.h>
#include <time.h>
int pthread_getcpuclockid(pthread_t thread, clockid_t *clock_id);
Compile and link with -pthread.
DESCRIPTION
The pthread_getcpuclockid() function returns the clock ID for the CPU time clock of the thread thread.
RETURN VALUE
On success, this function returns 0; on error, it returns a nonzero error number.
ERRORS
ENOENT Per-thread CPU time clocks are not supported by the system.
ESRCH No thread with the ID thread could be found.
VERSIONS
This function is available in glibc since version 2.2.
CONFORMING TO
POSIX.1-2001.
NOTES
When thread refers to the calling thread, this function returns an identifier that refers to the same clock manipulated by clock_gettime(2)
and clock_settime(2) when given the clock ID CLOCK_THREAD_CPUTIME_ID.
EXAMPLE
The program below creates a thread and then uses clock_gettime(2) to retrieve the total process CPU time, and the per-thread CPU time con-
sumed by the two threads. The following shell session shows an example run:
$ ./a.out
Main thread sleeping
Subthread starting infinite loop
Main thread consuming some CPU time...
Process total CPU time: 1.368
Main thread CPU time: 0.376
Subthread CPU time: 0.992
Program source
/* Link with "-lrt" */
#include <time.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>
#include <errno.h>
#define handle_error(msg)
do { perror(msg); exit(EXIT_FAILURE); } while (0)
#define handle_error_en(en, msg)
do { errno = en; perror(msg); exit(EXIT_FAILURE); } while (0)
static void *
thread_start(void *arg)
{
printf("Subthread starting infinite loop
");
for (;;)
continue;
}
static void
pclock(char *msg, clockid_t cid)
{
struct timespec ts;
printf("%s", msg);
if (clock_gettime(cid, &ts) == -1)
handle_error("clock_gettime");
printf("%4ld.%03ld
", ts.tv_sec, ts.tv_nsec / 1000000);
}
int
main(int argc, char *argv[])
{
pthread_t thread;
clockid_t cid;
int j, s;
s = pthread_create(&thread, NULL, thread_start, NULL);
if (s != 0)
handle_error_en(s, "pthread_create");
printf("Main thread sleeping
");
sleep(1);
printf("Main thread consuming some CPU time...
");
for (j = 0; j < 2000000; j++)
getppid();
pclock("Process total CPU time: ", CLOCK_PROCESS_CPUTIME_ID);
s = pthread_getcpuclockid(pthread_self(), &cid);
if (s != 0)
handle_error_en(s, "pthread_getcpuclockid");
pclock("Main thread CPU time: ", cid);
/* The preceding 4 lines of code could have been replaced by:
pclock("Main thread CPU time: ", CLOCK_THREAD_CPUTIME_ID); */
s = pthread_getcpuclockid(thread, &cid);
if (s != 0)
handle_error_en(s, "pthread_getcpuclockid");
pclock("Subthread CPU time: 1 ", cid);
exit(EXIT_SUCCESS); /* Terminates both threads */
}
SEE ALSO
clock_gettime(2), clock_settime(2), timer_create(2), clock_getcpuclockid(3), pthread_self(3), pthreads(7), time(7)
COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can
be found at http://www.kernel.org/doc/man-pages/.
Linux 2009-02-08 PTHREAD_GETCPUCLOCKID(3)