Sponsored Content
Top Forums Shell Programming and Scripting Merge two rows using awk or python Post 302348321 by grossgermany on Friday 28th of August 2009 02:25:06 AM
Old 08-28-2009
Merge two rows using awk or python

Hi,
Suppose I have a space delimited file like this:

Serial# 1970 1971 1972 1973 1974
193532 21 2 X X X
200201 20 30 X X 40
200201 X X 13 15 X
393666 66 3 X X 5
393666 77 X X X X

First, I want to check the serial#, if any two lines have the same serial#,(in this case line 2+3, and line 4+5 qualify), then merge these two lines by replacing X with the value of the other line.

Also, when there is a conflict, in this case line 4+5 have the first column as 66,77(rather than having X in either line or in both lines), then do not merge even though they have the same serial#, but flag both lines with FLAGGED on the CONFLICT_FLAG column.

The result would be:

Serial# 1970 1971 1972 1973 1974 CONFLICT_FLAG
193532 21 2 X X X
200201 20 30 13 15 40
393666 66 3 X X 5 FLAGGED
393666 77 X X X X FLAGGED

Is it possible to do this in either python or awk? Thank you.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge rows based on a common column

Hi guys, Please guide me if you have a solution to this problem. I have tried paste -s but it's not giving the desired output. I have a file with the following content- A123 box1 B345 bat2 C431 my_id A123 service C431 box1 A123 my_id I need two different outputs- OUTPUT1 A123... (6 Replies)
Discussion started by: smriti_shridhar
6 Replies

2. Shell Programming and Scripting

How to merge rows into columns ????

Hi guz I want to merge multiple rows into a multiple columns based on the first column. The file has symbol // I want to break the symbool // and I nedd exactlynew column at that point the output will be like this please guyz help in this isssue!!!!! merging rows into columns ... (4 Replies)
Discussion started by: bogu0001
4 Replies

3. Shell Programming and Scripting

merge similar rows

I have a large file (10M lines) that contains two columns: a frequency and a string, ex: 3 aaaaa 4 bbbbb 2 ccccc 5 aaaaa 1 ddddd 4 ccccc I need to merge the lines whose string part is the same, while updating the frequency. The output should look like this: 8 aaaaa 4 bbbbb 5 ccccc... (2 Replies)
Discussion started by: tootles564
2 Replies

4. UNIX for Dummies Questions & Answers

Merge rows with common column

Dear all I have big file with two columns A_AA960715 GO:0006952 A_AA960715 GO:0008152 A_AA960715 GO:0016491 A_AA960715 GO:0007165 A_AA960715 GO:0005618 A_AA960716 GO:0006952 A_AA960716 GO:0005618 A_AA960716... (15 Replies)
Discussion started by: AAWT
15 Replies

5. UNIX for Dummies Questions & Answers

Merge rows in bid data file

Dear all, Please help me ,,,, if I have input file like this A_AA960715 leucine-rich repeat-containing protein GO:0006952 defense response P A_AA960715 leucine-rich repeat-containing protein GO:0008152 metabolic process P A_AA960715 leucine-rich... (5 Replies)
Discussion started by: AAWT
5 Replies

6. Shell Programming and Scripting

Merge files based on both common and uncommon rows

Hi, I have two files A (2190 rows) and file B (1100 rows). I want to merge the contents of two files based on common field, also I need the unmatched rows from file A file A: ABC XYZ PQR file B: >LMN|chr1:11000-12456: >ABC|chr15:176578-187678: >PQR|chr3:14567-15866: output... (3 Replies)
Discussion started by: Diya123
3 Replies

7. Shell Programming and Scripting

Merge matching rows

Hello, I need this output. thank you very much. input: Code: ***table***wood ***snack***top ***table***garfield ***big***zen ***table***cars output: Code: ***table***wood2345garfield2345cars ***snack***top ***big***zen (7 Replies)
Discussion started by: tara123
7 Replies

8. Shell Programming and Scripting

Merge files and remove duplicated rows

In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows. The file name in the folder will look like e.q: MissingData_2014-08-25_09-30-18.txt MissingData_2014-08-25_09-30-14.txt MissingData_2014-08-26_09-30-12.txt The content... (9 Replies)
Discussion started by: Bergans
9 Replies

9. UNIX for Dummies Questions & Answers

Merge rows into one if first 2 columns match

Hi, I wanted to merge the content and below is input and required output info. Input: /hello,a,r /hello,a,L /hello,a,X /hi,b,v /hi,b,c O/p: /hello,a,r:L:X /hi,v,:v:c Use code tags, thanks. (6 Replies)
Discussion started by: ankitas
6 Replies

10. Programming

[python] merge pdf files.

First off I am very new to python but not to scripting I have done a lot of bash scripting. I need to create a python script for work that will combine multiple pdf files into one pdf file and archive both the combined file and the original pdf files. So we receive zip files from a client... (6 Replies)
Discussion started by: SaltCityScripts
6 Replies
DH_PYTHON(1)							     Debhelper							      DH_PYTHON(1)

NAME
dh_python - calculates Python dependencies and adds postinst and prerm Python scripts (deprecated) SYNOPSIS
dh_python [debhelperoptions] [-n] [-V version] [moduledirs...] DESCRIPTION
Note: This program is deprecated. You should use dh_python2 instead. This program will do nothing if debian/pycompat or a Python-Version control file field exists. dh_python is a debhelper program that is responsible for generating the ${python:Depends} substitutions and adding them to substvars files. It will also add a postinst and a prerm script if required. The program will look at Python scripts and modules in your package, and will use this information to generate a dependency on python, with the current major version, or on pythonX.Y if your scripts or modules need a specific python version. The dependency will be substituted into your package's control file wherever you place the token ${python:Depends}. If some modules need to be byte-compiled at install time, appropriate postinst and prerm scripts will be generated. If already byte- compiled modules are found, they are removed. If you use this program, your package should build-depend on python. OPTIONS
module dirs If your package installs Python modules in non-standard directories, you can make dh_python check those directories by passing their names on the command line. By default, it will check /usr/lib/site-python, /usr/lib/$PACKAGE, /usr/share/$PACKAGE, /usr/lib/games/$PACKAGE, /usr/share/games/$PACKAGE and /usr/lib/python?.?/site-packages. Note: only /usr/lib/site-python, /usr/lib/python?.?/site-packages and the extra names on the command line are searched for binary (.so) modules. -V version If the .py files your package ships are meant to be used by a specific pythonX.Y version, you can use this option to specify the desired version, such as 2.3. Do not use if you ship modules in /usr/lib/site-python. -n, --noscripts Do not modify postinst/prerm scripts. CONFORMS TO
Debian policy, version 3.5.7 Python policy, version 0.3.7 SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Josselin Mouette <joss@debian.org> most ideas stolen from Brendan O'Dea <bod@debian.org> 9.20120909 2011-12-06 DH_PYTHON(1)
All times are GMT -4. The time now is 06:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy