Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Problems concatenating data using UNIX?

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Problems concatenating data using UNIX?
# 1  
Old 09-17-2014
Problems concatenating data using UNIX?
Hello,
Can somebody help me to solve this inconsistent data issue. I have a pipe delimiter file and one of the column is a comment. I am trying to concatenate into one single sentence. For instance, I have a file actually with 2 records but the way it considers the first record is as different rows and hence loads into the table as different records. In the sample below, 123 and 789 are two different records.


Code:
123|efg|"Try to solve.
This is a unix script. Trying to concatenate.

It seems to be simple but not as simple once I start scripting.
Sincerly,
XYZ
ABC"|5|3|6 months
789|def|"Better way to solve this issue."|4|6|7 years

This is how it has to look.
Code:
abc|efg|"Try to solve. This is a unix script. Trying to concatenate. It seems to be simple but not as simple once I start scripting. Sincerly, XYZ ABC"|5|3|6 months
789|def|"Better way to solve this issue."|4|6|7 years

I first tried to remove the blank space using
Code:
sed '/^$/d' t1.txt > t2.txt

Then I couldn't figure out how to remove spaces after a period and concatenate the next sentence.

I would really appreciate any input on resolving this.
Last edited by Scrutinizer; 09-17-2014 at 06:52 PM.. Reason: CODE tags
# 2  
Old 09-17-2014
Try:
Code:
awk -F\| '{while(NF<6 && getline p) $0=$0 " "p}1' file

# 3  
Old 09-17-2014
Thank you! Do I have to change anything in the code as I am getting an error?

Code:
$ cat t1
123|efg|"Try to solve.
This is a unix script. Trying to concatenate.

It seems to be simple but not as simple once I start scripting.
Sincerly,
XYZ
ABC"|5|3|6 months
789|def|"Better way to solve this issue."|4|6|7 years


I tried the following
Code:
awk -F\| '{while(NF<6 && getline p) $0=$0 " "p}1' t1
awk: syntax error near line 1
awk: bailing out near line 1


Thanks for your help!
Last edited by Don Cragun; 09-18-2014 at 12:23 AM.. Reason: Add CODE tags.
# 4  
Old 09-18-2014
When asking for help writing code, it is always a good idea to tell us what OS and shell you're using. Some tools work differently on different systems.

In this case, I would guess that you're using a Solaris/SunOS system. On Solaris/SunOS systems, you need to change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk because /usr/bin/awk on Solaris/SunOS systems gives you an awk that is compatible with an earlier (1980's vintage) version of the awk programming language that did not support many of the features of the language that are available in newer versions of awk.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Red Hat

Problems with EURO symbol in a data flat file

Hi, I'm creating a flat file with various deimiters in Linux RHEL. the content for the flat file will be extracted from an oracle database. delimiters also stored in a seperate table in oracle databasae. when i use extended ASCII characters like cedilla or EURO (€) symbols, the actual character is... (9 Replies)
Discussion started by: Fakru.y
9 Replies

2. Shell Programming and Scripting

Problems with fetching data from Oracle database

Here's a database query which looks up the NAME column of PRODUCT table SELECT NAME FROM PRODUCT ; And this query retrieves me the following output SUGAR COCOA HONEY WHEAT CABBAGE CAULI FLOWER Please note the last record CAULI FLOWER contains TWO blank spaces between the two words. ... (4 Replies)
Discussion started by: kumarjt
4 Replies

3. Shell Programming and Scripting

Concatenating the lines of a data

I have a data of 1 lac lines with the following format abcde,1,2,3,4, ,ee ,ff ,gg ,hh ,mm abcde,3,4,5,6, ,we ,qw ,as ,zx ,cf abcde,1,5,6,7, ,dd ,aa ,er .... .... (6 Replies)
Discussion started by: aravindj80
6 Replies

4. Shell Programming and Scripting

Execution Problems with scan and change file data content

Input file >Read_1 XXXXXXXXXXSDFXXXXXDS ASDRXXXXXTGAGTXXXXXT TGTGATXXXXXAXXXXGXXA . . Desired output file >Read_1 XXXXXXXXXXXXXXXXXXDS ASDRXXXXXTGAGTXXXXXT TGTGATXXXXXXXXXXXXXA . . (5 Replies)
Discussion started by: patrick87
5 Replies

5. Shell Programming and Scripting

Excution Problems with loading huge data content and convert it

Hi, I got long list of referred file content: CGTGCFTGCGTFREDG PEOGDKGJDGKLJGKL DFGDSFIODUFIODSUF FSDOFJSODIFJSIODFJ DSFSDFDFSDOFJFOSF SDFOSDJFOJFPPIPIOP . . . Input file content: >sample_1 SDFDSKLFKDSLSDFSDFDFGDSFIODUFIODSUFSDDSFDSSDFDSFAS (14 Replies)
Discussion started by: patrick87
14 Replies

6. UNIX for Dummies Questions & Answers

Validating XSL sheet data in Unix Data file

Dear All, Need your help. In my day to day activities I have to validate/search Excel Sheet data (eg.say Application No. 0066782345) data into the Unix environment file whether the same data is present in that file or not. There are hundreds of records coming in excel file and I am doing grep... (1 Reply)
Discussion started by: ravijunghare
1 Replies

7. Shell Programming and Scripting

problems with a script that outputs data to a file

First of all, im a total newbie to the point that i do not know what are the terms to search for my problem. I did however spend the rest of the day today trying to figure out what is wrong with my bash script. ive always thought that the best way to learn is to tackle a problem heads on. but at... (1 Reply)
Discussion started by: joeribut
1 Replies

8. UNIX for Advanced & Expert Users

how to read the data from an excel sheet and use those data as variable in the unix c

I have 3 columns in an excel sheet. c1 c2 c3 EIP_ACCOUNT SMALL_TS_01 select A.* from acc; All the above 3 col shoud be passed a variable in the unix code. 1.How to read an excel file 2.How to pass these data as variable to the unic script (1 Reply)
Discussion started by: Anne Grace
1 Replies

9. Shell Programming and Scripting

concatenating static string to records in data file

I just need to add a static ID to each output record so the users will be able to tell which group records in combined flatfiles come from I have the static ID in a bourne variable. I tried awk '{print "${GroupID}" $0}' infile > outfile But I ended up with the string ${GroupID} instead of... (5 Replies)
Discussion started by: gillbates
5 Replies
Login or Register to Ask a Question