Sponsored Content
Top Forums Shell Programming and Scripting [Solved] Data error need to fix Post 302886011 by gina.lizar on Wednesday 29th of January 2014 11:36:44 AM
Old 01-29-2014
[Solved] Data error need to fix

Hi Guys,

I`m having a strange problem with my data set. Whenever there is a transition to another value is col1, the corresponding 3rd col goes to the next line. This is a huge file, so need to fix in a script. The file is tab delimited.

Here is what is happening when transitioning from DS11.02060 to DS11.02061, the value 1150.76 is in the next line. Happens for every transition. The file is sorted by col 1.

Code:
DS11.02060    DS11.28178    562.907
DS11.02060    DS11.28179
    1150.76
DS11.02061    DS11.02063    455.141
DS11.02061    DS11.02064    476.496
DS11.02062    DS11.28178    562.957
DS11.02062    DS11.28179
    1170.76
DS11.02063   DS11.02063    235.141
DS11.02063    DS11.02064    445.499

Expected output 

DS11.02060    DS11.28178    562.907
DS11.02060    DS11.28179    1150.76
DS11.02061    DS11.02063    455.141
DS11.02061    DS11.02064    476.496
DS11.02062    DS11.28178    562.957
DS11.02062    DS11.28179    1170.76
DS11.02063   DS11.02063    235.141
DS11.02063    DS11.02064    445.499

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk / shell - Fix broken lines and data

Gurus, I am struggling with a issue and thought I could use some of your expertise. Need Help with this I have a flat file that has millions of records 24|john|account ~ info |56| 25|kuo|account ~ journal |58| 27|kim|account ~ journal |59| 28|San|account ~ journal |60|... (3 Replies)
Discussion started by: rimss
3 Replies

2. UNIX for Dummies Questions & Answers

Format/Fix Timestamp Data in a File.

Hello Experts, I have a timestamp(6) column in a .csv data file , format of the data is as below:- ETCT,P,Elec, Inc.,abc,11/5/2010 4:16:09.000000 PM,Y,Y,Y I want the timestamp column to be properly formatted like 11/05/2010 04:16:09.000000 PM Currently the "0" is missing with... (3 Replies)
Discussion started by: mtlrsk
3 Replies

3. Shell Programming and Scripting

[Solved] Extracting data from a determined file

Hi guys, How you doing? Need some help with this .I got a file with this kind of information SURVIVABLE PROCESSORS Name Type IP Address Reg Act Translations Net Updated Rgn Gateway001 ... (8 Replies)
Discussion started by: REX:)
8 Replies

4. Shell Programming and Scripting

[Solved] Messaging data into required report

Hello to all; hope someone can assist me in getting the required output that my manager is expecting. I have been able to generate this code which does the comparison of the files and creates the file called diff_fuss_file.txt $ vi fussrpt.pl #!/usr/bin/perl #cd /tmp #rm output.txt ... (2 Replies)
Discussion started by: gvolpini
2 Replies

5. Shell Programming and Scripting

[SOLVED] Converting data from one format to the other

Hi All, I need to convert an exel spreadsheet into a SAS dataset, and the following format change is needed. Please help, this is too complex for a biologist. Let me describe the input. 1st row is generation.1st column in keyword 'generation', starting 2nd column there are 5... (9 Replies)
Discussion started by: newbie83
9 Replies

6. Shell Programming and Scripting

[Solved] Converting the data into matrix with 0's and 1's

I have a file that contains 2 columns tag,pos cat input_file tag pos atg 10 ata 16 agt 15 agg 19 atg 17 agg 14 I have used following command to sort the file based on second column sort -k 2 input_file tag pos atg 10 agg 14 agt 15 ata 16 agg 19 atg 17 (2 Replies)
Discussion started by: raj_k
2 Replies

7. Shell Programming and Scripting

[Solved] Compare column data in all the rows

Hi.. In the below sorted input file.. I am comparing the first 3 columns of data one by one row and it is a pipeline delimitter file.. AA|BB|CC|line1 AA|BB|CC|ine4 AA|BB|CC|line2 BB|CC|DD|line3 BB|CC|DD|line5 If first 3 columns of data matches with any record in the file the... (4 Replies)
Discussion started by: NareshN
4 Replies

8. Shell Programming and Scripting

[Solved] Testing Data Type of User Input

hi all i'm new in shell scripting and now i'm working on project and i wanna make a function take input from user and test it's datatype then if correct write it to file i wounder what is the best way to do this ? can i use awk ? edit by bakunin: Please give your threads a meaningful... (5 Replies)
Discussion started by: mohamed91
5 Replies

9. Shell Programming and Scripting

[Solved] Data manipulation

Hallo Team, I need your help. I have a file that has two colums. See sample below: 105550 0.28 105550 0.24 125550 0.28 125550 0.24 215650 0.28 215650 0.24 315550 0.28 315550 0.24 335550 0.28 335550 0.24 40555 0.21 40555 0.17 415550 0.21 415550 0.17 43555 0.21 43555 0.17 (5 Replies)
Discussion started by: kekanap
5 Replies

10. UNIX for Beginners Questions & Answers

How to fix and manage data in rescue mode?

Hello, I am running ubuntu 14.04 My server has problems again. I need to manage system files in rescue mode, backup some files, edit /etc/crontab and rc.local files etc. root@rescue:~# cat /proc/mdstat Personalities : md2 : active raid1 sda2 sdb2 4193216 blocks md3 :... (4 Replies)
Discussion started by: baris35
4 Replies
COL(1)							    BSD General Commands Manual 						    COL(1)

NAME
col -- filter reverse line feeds from input SYNOPSIS
col [-bfhpx] [-l num] DESCRIPTION
The col utility filters out reverse (and half reverse) line feeds so that the output is in the correct order with only forward and half for- ward line feeds, and replaces white-space characters with tabs where possible. This can be useful in processing the output of nroff(1) and tbl(1). The col utility reads from the standard input and writes to the standard output. The options are as follows: -b Do not output any backspaces, printing only the last character written to each column position. -f Forward half line feeds are permitted (``fine'' mode). Normally characters printed on a half line boundary are printed on the fol- lowing line. -h Do not output multiple spaces instead of tabs (default). -l num Buffer at least num lines in memory. By default, 128 lines are buffered. -p Force unknown control sequences to be passed through unchanged. Normally, col will filter out any control sequences from the input other than those recognized and interpreted by itself, which are listed below. -x Output multiple spaces instead of tabs. The control sequences for carriage motion that col understands and their decimal values are listed in the following table: ESC-7 reverse line feed (escape then 7) ESC-8 half reverse line feed (escape then 8) ESC-9 half forward line feed (escape then 9) backspace moves back one column (8); ignored in the first column carriage return (13) newline forward line feed (10); also does carriage return shift in shift to normal character set (15) shift out shift to alternate character set (14) space moves forward one column (32) tab moves forward to next tab stop (9) vertical tab reverse line feed (11) All unrecognized control characters and escape sequences are discarded. The col utility keeps track of the character set as characters are read and makes sure the character set is correct when they are output. If the input attempts to back up to the last flushed line, col will display a warning message. ENVIRONMENT
The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of col as described in environ(7). EXIT STATUS
The col utility exits 0 on success, and >0 if an error occurs. SEE ALSO
colcrt(1), expand(1), nroff(1), tbl(1) STANDARDS
The col utility conforms to Version 2 of the Single UNIX Specification (``SUSv2''). HISTORY
A col command appeared in Version 6 AT&T UNIX. BSD
August 4, 2004 BSD
All times are GMT -4. The time now is 03:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy