Sponsored Content
Top Forums Shell Programming and Scripting Concatenation lines based on first field of the lines Post 302987656 by svks1985 on Tuesday 13th of December 2016 08:23:41 PM
Old 12-13-2016
Concatenation lines based on first field of the lines

Hello All,

This is to request some assistance on the issue that I encountered until recently.
Problem is:
I have a pipe delimited file in which some lines/records are broken. Now, I have to join/concatenate broken lines in the file to form actual record to make sure that the count of records before and after processing the file stays the same.

Code:
Sample data looks like this:
113321|107|E|1|828|20|4032832|EL POETA|VILLALOBOS MIJARES PABLO
NEPTALI RICARDO ELIECER
SABREZ|CA|2000|10000|10600|201407201412
113321|107|E|1|828|20|3924814|ME HACE TANTO BIEN|GUERRERO DE LA PENA
MUNOZ CARLOS ISSAC|CA|1666|10000|8800|201407201412
113321|107|E|1|828|20|4055313|PEPE|ALVAREZ GONZALEZ
ANDERSON MIGUEL|CA|2500|10000|13200|201407201412
113321|107|E|1|828|20|4034084|SIN TI|VILLALOBOS MIJARES PABLO
NEPTALI RICARDO ELIECER|CA|1000|10000|5300|201407201412

Code:
Expected output would be like this:
113321|107|E|1|828|20|4032832|EL POETA|VILLALOBOS MIJARES PABLO NEPTALI RICARDO ELIECER SABREZ|CA|2000|10000|10600|201407201412
113321|107|E|1|828|20|3924814|ME HACE TANTO BIEN|GUERRERO DE LA PENA MUNOZ CARLOS ISSAC|CA|1666|10000|8800|201407201412
113321|107|E|1|828|20|4055313|PEPE|ALVAREZ GONZALEZ ANDERSON MIGUEL|CA|2500|10000|13200|201407201412
113321|107|E|1|828|20|4034084|SIN TI|VILLALOBOS MIJARES PABLO NEPTALI RICARDO ELIECER|CA|1000|10000|5300|201407201412

Code:
Code that I have tried so far:
awk -v var="$pattern" '/var"\n"/{printf "\n" $0;next}{printf $0}' file.txt
$pattern is variable that I am passing as 113321

Any assistance would be greatlly appreciated

Last edited by svks1985; 12-13-2016 at 09:31 PM.. Reason: Adding code tags and snippet
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

add lines automatically based on a field on another file

hello I have a number of lines that need to be added at the end of a file each time I add a field in another file (let's name it file2) file2 has this format: filed1:field2:path1:path2:path3:path... Whenever I add a path field, I should add to file1 these lines: <Location path1>... (0 Replies)
Discussion started by: melanie_pfefer
0 Replies

2. Shell Programming and Scripting

Awk concatenation in different lines

Hi All I have the data as id-number 01 name-id x0 input-id x0 output-id x0 name-id x0 input-id x0 output-id x0 name-id x0 input-id x0 output-id x0 id-number 02 name-id x0 input-id x0 output-id x0 name-id x0 input-id x0 output-id x0 name-id x0 input-id x0 output-id x0 . . I... (4 Replies)
Discussion started by: posner
4 Replies

3. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

4. Shell Programming and Scripting

Combine multiple lines in file based on specific field

Hi, I have an issue to combine multiple lines of a file. I have records as below. Fields are delimited by TAB. Each lines are ending with a new line char (\n) Input -------- ABC 123456 abcde 987 890456 7890 xyz ght gtuv ABC 5tyin 1234 789 ghty kuio ABC ghty jind 1234 678 ght ... (8 Replies)
Discussion started by: ratheesh2011
8 Replies

5. UNIX for Dummies Questions & Answers

join 2 lines based on 1st field

hi i have a file with the following lines 2303:13593:137135 16 abc1 26213806....... 1234:45675:123456 16 bbc1 9813806....... 2303:13593:137135 17 bna1 26566444.... 1234:45675:123456 18 nnb1 98123456....... i want to join the lines having common 1st field i,e., ... (1 Reply)
Discussion started by: anurupa777
1 Replies

6. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

7. Shell Programming and Scripting

Issue in Concatenation/Joining of lines in a dynamically generated file

Hi, I have a file containing many records delimited by pipe (|). Each record should contain 17 columnns/fields. there are some fields having fields less than 17.So i am extracting those records to a file using the below command awk 'BEGIN {FS="|"} NF !=17 {print}' feedfile.txt... (8 Replies)
Discussion started by: TomG
8 Replies

8. Shell Programming and Scripting

awk joining multiple lines based on field count

Hi Folks, I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help. INPUT hname01 windows appnamec1eda_p1, ... (5 Replies)
Discussion started by: shunya
5 Replies

9. Shell Programming and Scripting

awk to print lines based on text in field and value in two additional fields

In the awk below I am trying to print the entire line, along with the header row, if $2 is SNV or MNV or INDEL. If that condition is met or is true, and $3 is less than or equal to 0.05, then in $7 the sub pattern :GMAF= is found and the value after the = sign is checked. If that value is less than... (0 Replies)
Discussion started by: cmccabe
0 Replies

10. UNIX for Beginners Questions & Answers

Print lines based upon unique values in Nth field

For some reason I am having difficulty performing what should be a fairly easy task. I would like to print lines of a file that have a unique value in the first field. For example, I have a large data-set with the following excerpt: PS003,001 MZMWR/ L-DWD// * PS003,001... (4 Replies)
Discussion started by: jvoot
4 Replies
SPRIO(1)							  SLURM commands							  SPRIO(1)

NAME
sprio - view the factors that comprise a job's scheduling priority SYNOPSIS
sprio [OPTIONS...] DESCRIPTION
sprio is used to view the components of a job's scheduling priority when the multi-factor priority plugin is installed. sprio is a read- only utility that extracts information from the multi-factor priority plugin. By default, sprio returns information for all pending jobs. Options exist to display specific jobs by job ID and user name. OPTIONS
-h, --noheader Do not print a header on the output. --help Print a help message describing all options sprio. -j <job_id_list>, --jobs=<job_id_list> Requests a comma separated list of job ids to display. Defaults to all jobs. -l, --long Report more of the available information for the selected jobs. -M, --clusters=<string> The cluster to issue commands to. Only one cluster name may be specified. -n, --norm Display the normalized priority factors for the selected jobs. -o <output_format>, --format=<output_format> Specify the information to be displayed, its size and position (right or left justified). The default formats when all factors have been assigned non-zero weights are default "%.7i %.10Y %.10A %.10F %.10J %.10P %.10Q" -l, --long "%.7i %.8u %.10Y %.10A %.10F %.10J %.10P %.10Q %.6N" The format of each field is "%[.][size]type". size is the minimum field size. If no size is specified, whatever is needed to print the information will be used. . indicates the output should be left justified. By default, output is right justified. Valid type specifications include: %a Normalized age priority %A Weighted age priority %f Normalized fair-share priority %F Weighted fair-share priority %i Job ID %j Normalized job size priority %J Weighted job size priority %N Nice adjustment %p Normalized partition priority %P Weighted partition priority %q Normalized quality of service priority %Q Weighted quality of service priority %u User name for a job %Y Job priority %y Normalized job priority -u <user_list>, --user=<user_list> Request jobs from a comma separated list of users. The list can consist of user names or user id numbers. --usage Print a brief help message listing the sprio options. -v, --verbose Report details of sprios actions. -V , --version Print version information and exit. -w , --weights Display the configured weights for each factor. This is for information purposes only. Actual job data is suppressed. ENVIRONMENT VARIABLES
If no corresponding commandline option is specified, sprio will use the value of the following environment variables. SLURM_CLUSTERS Same as --clusters EXAMPLES
Print the list of all pending jobs with their weighted priorities > sprio JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65539 62664 0 51664 1000 10000 0 65540 62663 0 51663 1000 10000 0 65541 62662 0 51662 1000 10000 0 Print the list of all pending jobs with their normalized priorities > sprio -n JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65539 0.00001459 0.0007180 0.5166470 1.0000000 1.0000000 0.0000000 65540 0.00001459 0.0007180 0.5166370 1.0000000 1.0000000 0.0000000 65541 0.00001458 0.0007180 0.5166270 1.0000000 1.0000000 0.0000000 Print the job priorities for specific jobs > sprio --jobs=65548,65547 JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65547 62078 0 51078 1000 10000 0 65548 62077 0 51077 1000 10000 0 Print the job priorities for jobs of specific users > sprio --users=fred,sally JOBID USER PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS 65548 fred 62079 1 51077 1000 10000 0 65549 sally 62080 1 51078 1000 10000 0 Print the configured weights for each priority component > sprio -w JOBID PRIORITY AGE FAIRSHARE JOBSIZE PARTITION QOS Weights 1000 100000 1000 10000 1 COPYING
Copyright (C) 2009 Lawrence Livermore National Security. Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). CODE-OCEC-09-009. All rights reserved. This file is part of SLURM, a resource management program. For details, see <http://www.schedmd.com/slurmdocs/>. SLURM is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. SLURM is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. SEE ALSO
squeue(1), sshare(1) sprio 2.3 May 2011 SPRIO(1)
All times are GMT -4. The time now is 05:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy