Sponsored Content
Top Forums Shell Programming and Scripting Bash/shell merge similar lines Post 302966905 by vgersh99 on Wednesday 17th of February 2016 06:55:35 PM
Old 02-17-2016
something along these lines - a bit succinct on explanation, but... :
Code:
awk '
  # idx is an index constructed by concatenating first 3 fields in the line
  {idx=$1 FS $2 FS $3} 

  # a - an array indexed by 'idx'.
  # if idx is in a, add forth field to it; if not, assign forth field as the first entry in array a
  {a[idx]=(idx in a)?a[idx] OFS $4:$4} 

  # after all records/lines have been process...
  # iterate through all the indecies in array a, print out the value of the index and
  # the corresponding value stored in the array with the current index.
  END {for(i in a) print i FS a[i]}' OFS=, myInputFileGoesHere


Last edited by vgersh99; 02-17-2016 at 08:39 PM..
This User Gave Thanks to vgersh99 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Urgent : Merge similar lines

Hi, I have a file like this. please notice that ./usr/orders1/order_new_2627 appears more than once, thus needs to be merged. I would like to merge the lines where the first column match so the output should be like this: Please help (2 Replies)
Discussion started by: rakeshou
2 Replies

2. Shell Programming and Scripting

Help with comparing 2 lines in 2 different file in shell bash

Hi guys i need help with comparing lines in 2 separate files. Both files contain the same amount of lines and i need to output the difference into the 2nd file. The 1st file is always correct. 1st file (Expected.e): Tuesday, 11 August 2009 Wednesday, 12 August 2009 Thursday, 13 August 2009... (2 Replies)
Discussion started by: kcrap
2 Replies

3. Shell Programming and Scripting

merge similar rows

I have a large file (10M lines) that contains two columns: a frequency and a string, ex: 3 aaaaa 4 bbbbb 2 ccccc 5 aaaaa 1 ddddd 4 ccccc I need to merge the lines whose string part is the same, while updating the frequency. The output should look like this: 8 aaaaa 4 bbbbb 5 ccccc... (2 Replies)
Discussion started by: tootles564
2 Replies

4. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies

5. UNIX for Dummies Questions & Answers

Merge two files with two columns being similar

Hi everyone. How can I merge two files, where each file has 2 columns and the first columns in both files are similar? I want all in a file of 4 columns; join command removes the duplicate columns. 1 Dave 2 Mark 3 Paul 1 Apple 2 Orange 3 Grapes to get it like this in the 3rd file:... (9 Replies)
Discussion started by: Atrisa
9 Replies

6. UNIX for Dummies Questions & Answers

merge lines within a file that start with a similar pattern

Hello! i have a text file.. which contains the data as follows i want to merge the declarations lines pertaining to one datatype in to a single line as follows i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies

7. Shell Programming and Scripting

Help with merge two file based on similar column content

Input file 1: A1BG A1BG A1BG A1CF A1CF BCAS BCAS A2LD1 A2M A2M HAT . . Input file 2: A1BG All A1CF TEMP (5 Replies)
Discussion started by: perl_beginner
5 Replies

8. Shell Programming and Scripting

Merge two files with similar column entries

Hi , I have few files which contains user name and data transfer rate in MBs and this data is collected for year and for each month report is saved in 12 different files I have to merge all the files to prepare the final report Files are as below Similarly I have 10 more files ... (5 Replies)
Discussion started by: pratapsingh
5 Replies

9. Shell Programming and Scripting

Shell script to merge and delete lines

POLY_STATS { EqName 103_tri Id 123 act_polyCount 1 act_polyValue 0 } POLY_STATS { EqName 103_tri Id 123 pass_polyCount 2 pass_polyValue 0 } POLY_STATS { EqName 103 Id 123 mes_polyCount 2 mes_polyValue 0 (5 Replies)
Discussion started by: Jag02
5 Replies

10. UNIX for Beginners Questions & Answers

Merge multi-lines into one single line using shell script or Linux command

Hi, Can anyone help me for merge the following multi-line log which beginning with a " and line ending with ": into one line. *****Original Log***** 087;2008-12-06;084403;"mc;;SYHLR6AP1D\LNZW;AD-703;1;12475;SYHLR6AP1B;1.1.1.1;0000000062;HGPDI:MSISDN=12345678,APNID=1,EQOSID=365;... (3 Replies)
Discussion started by: rajeshlinux2010
3 Replies
CUDA Runtime API Specific Functions(3)			     Hardware Locality (hwloc)			    CUDA Runtime API Specific Functions(3)

NAME
CUDA Runtime API Specific Functions - Functions static inline int hwloc_cudart_get_device_pci_ids (hwloc_topology_t topology , int idx, int *domain, int *bus, int *dev) static inline int hwloc_cudart_get_device_cpuset (hwloc_topology_t topology , int idx, hwloc_cpuset_t set) static inline hwloc_obj_t hwloc_cudart_get_device_pcidev (hwloc_topology_t topology, int idx) static inline hwloc_obj_t hwloc_cudart_get_device_osdev_by_index (hwloc_topology_t topology, unsigned idx) Detailed Description Function Documentation static inline int hwloc_cudart_get_device_cpuset (hwloc_topology_t topology, intidx, hwloc_cpuset_tset) [static] Get the CPU set of logical processors that are physically close to device idx. Return the CPU set describing the locality of the CUDA device whose index is idx. Topology topology and device idx must match the local machine. I/O devices detection and the CUDA component are not needed in the topology. The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_cudart_get_device_osdev_by_index(). This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset. static inline hwloc_obj_t hwloc_cudart_get_device_osdev_by_index (hwloc_topology_ttopology, unsignedidx) [static] Get the hwloc OS device object corresponding to the CUDA device whose index is idx. Return the OS device object describing the CUDA device whose index is idx. Return NULL if there is none. The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the CUDA component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_cudart_get_device_cpuset(). Note: The corresponding PCI device object can be obtained by looking at the OS device parent object. This function is identical to hwloc_cuda_get_device_osdev_by_index(). static inline int hwloc_cudart_get_device_pci_ids (hwloc_topology_t topology, intidx, int *domain, int *bus, int *dev) [static] Return the domain, bus and device IDs of the CUDA device whose index is idx. Device index idx must match the local machine. static inline hwloc_obj_t hwloc_cudart_get_device_pcidev (hwloc_topology_ttopology, intidx) [static] Get the hwloc PCI device object corresponding to the CUDA device whose index is idx. Return the PCI device object describing the CUDA device whose index is idx. Return NULL if there is none. Topology topology and device idx must match the local machine. I/O devices detection must be enabled in topology topology. The CUDA component is not needed in the topology. Author Generated automatically by Doxygen for Hardware Locality (hwloc) from the source code. Version 1.7 Sun Apr 7 2013 CUDA Runtime API Specific Functions(3)
All times are GMT -4. The time now is 05:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy