Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

SED or AWK "append line to the previous line"

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting SED or AWK "append line to the previous line"
# 15  
Old 05-17-2009
Give an example of your input file and the desired output within code tags (select the code with the mouse and click on # above the edit box).
# 16  
Old 05-17-2009
cat file.text

1234;test;test;test;
;test;test;test;test
beta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
test



should be

1234;test;test;test;;test;test;test;testbeta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;test
# 17  
Old 05-17-2009
Quote:
Originally Posted by research3
cat file.text

1234;test;test;test;
;test;test;test;test
beta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
test



should be

1234;test;test;test;;test;test;test;testbeta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;test
Should be something like this:

Code:
awk 'NR==1{s=$0;next}
!/^[0-9]/{s=s$0;next}
{print s;s=$0}
END{if(s)print s}' file

# 18  
Old 05-17-2009
Quote:
Originally Posted by research3
Hi all,

At first I want to thank you for your help. I really appreciate it!!
I surprised how many different kinds of possibilities there are to solve these issues.

It reminds me of an article from the pinguin magazin in middle Europe, a going to write something about this later.

I've tested your solutions and I found out the following different things :

DEVTAKH -- your solution is working fine for a small amount of data, but if the csv file has more than 200 lines a lot of lines are lost!
Maybe your string matches with some of the characters in my csv file.

$ cat test.csv | wc -l
552
$ cat test.csv | grep "^;" | wc -l
1
$ time sed -n '/^;.*/!{$!{N;s/\(.*\)\n\n*\(;.*\)*/\1\2/g;P;D};p}' test.csv | wc -l
276

real 0m11.108s
user 0m11.045s
sys 0m0.014s

DURDEN TYLER -- PANYAM --- GHOSTDOG74
your solution is working well, too thanks.



[I]$time awk 'BEGIN {i=0;} { if (substr($0,1,1)==";") {a[i]=a[i]""$0} else {a[++i]=$0} } END { for (i in a) print a;}' test.csv | wc -l
551

real 0m0.106s
user 0m0.091s
sys 0m0.015s


$ time ./test_perl.sh test.csv | wc -l
550

real 0m0.348s
user 0m0.335s
sys 0m0.012s

$ time ./test.py test.csv | wc -l
549

real 0m0.310s
user 0m0.038s
sys 0m0.023s

About the article I just want to say that:

There was an assignment of tasks to work on a text file that was about 1GB.
The assignment was to convert the text file a shortest time.
All languages and scripts were allowed „from python awk to java“.
If you are interested in this I will write more about it.
Let me know.
This will work given the fact that the next line will be the only line that has the concatenation part.

Code:
sed -n '/^;.*/!{$!{N;/.*\n;.*/{s/\(.*\)\n\(;.*\)*/\1\2/g;p;d};/.*\n;.*/!{P;D}};p}' file

# 19  
Old 05-17-2009
-- Bingo --

The code:

Code:
awk 'NR==1{s=$0;next} /^[A-Z]|^;/{s=s$0;next} {print s;s=$0} END{if(s)print s}' file.text


1234;test;test;test;;test;test;test;testbeta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;test
# 20  
Old 05-17-2009
or

Code:
sed -n '/^[a-z]/!{$!{N;/.*\n;.*/{s/\(.*\)\n\(;.*\)*/\1\2/g;p;d};/.*\n;.*/!{P;D}};p}' file.text

1234;test;test;test;;test;test;test;test
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
# 21  
Old 05-17-2009
Quote:
Originally Posted by research3
-- Bingo --

The code:

Code:
awk 'NR==1{s=$0;next} /^[A-Z]|^;/{s=s$0;next} {print s;s=$0} END{if(s)print s}' file.text


1234;test;test;test;;test;test;test;testbeta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;test
you sure its correct?
Code:
# more file
1234;test;test;test;
;test;test;test;test
beta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
test
;test;test;test;test
charlie;text
test
;test;test;test;test
dog;text
1234567789;test;alpha;beta

# awk 'NR==1{s=$0;next} /^[A-Z]|^;/{s=s$0;next} {print s;s=$0} END{if(s)print s}' file
1234;test;test;test;;test;test;test;test
beta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
test;test;test;test;test
charlie;text
test;test;test;test;test
dog;text
1234567789;test;alpha;beta

sed too:
Code:
# sed -n '/^[a-z]/!{$!{N;/.*\n;.*/{s/\(.*\)\n\(;.*\)*/\1\2/g;p;d};/.*\n;.*/!{P;D}};p}' file
1234;test;test;test;;test;test;test;test
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;
;test;test;test;test
;test;test;test;test

Code:
# awk 'BEGIN{s=""}/^[0-9]/ && NR==1 || !/^[0-9]/{ s=s $0}/^[0-9]/ && NR>1{ s=s "\n" $0}END{print  s}' file
1234;test;test;test;;test;test;test;testbeta;text
01234;test;alpha;beta
47888;test;test;test;test
88899;test;test;test;test;test;test;test;testcharlie;texttest;test;test;test;testdog;text
1234567789;test;alpha;beta
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Failure: if grep "$Var" "$line" inside while read line loop

Hi everybody, I am new at Unix/Bourne shell scripting and with my youngest experiences, I will not become very old with it :o My code: #!/bin/sh set -e set -u export IFS= optl="Optl" LOCSTORCLI="/opt/lsi/storcli/storcli" ($LOCSTORCLI /c0 /vall show | grep RAID | cut -d " "... (5 Replies)
Discussion started by: Subsonic66
5 Replies

2. Shell Programming and Scripting

Append a line "while read line"

Hello everyone, I want to process a data file using bash, I would like add a new line RIGHT after the time when i know "another variable met a value" Like this: while read line do ... if ; then #Append a new line "===" to input.txt file after $line position #break fi done <... (11 Replies)
Discussion started by: fongthai
11 Replies

3. Shell Programming and Scripting

Move a line containg "char" above line containing "xchar"

Okay, so I have a rather large text file and will have to process many more and this will save me hours of work. I'm not very good at scripting, so bear with me please. Working on Linux RHEL I've been able to filter and edit and clean up using sed, but I have a problem with moving lines. ... (9 Replies)
Discussion started by: rex007can
9 Replies

4. Shell Programming and Scripting

Is it possible to use sed to handle the line contains BOTH "AA" and "BB"

if yes, can some expert give an example Lei (2 Replies)
Discussion started by: yanglei_fage
2 Replies

5. Shell Programming and Scripting

AWK for multiple line records RS="^" FS="#"

I have to pull multiple line records with ^ as the record separator(RS)... # should be my field separator (FS)... Sample record is: ^-60#ORA-00060: deadlock detected while waiting for resource ORA-00001: unique constraint (SARADM.TCKNUM_PK) violated#PROC:AVAILABLE_FOR_GETNXTTIC#02/27/2012... (7 Replies)
Discussion started by: Vidhyaprakash
7 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. Shell Programming and Scripting

append "awk command" to the end of each line

hi; this is my qqq.mos: l ./gcsw 86.0.0.1 'lt all;l+;lset SectorPort=860 Tilt 861;l-' l ./gcsw 86.0.0.2 'lt all;l+;lset SectorPort=862 Tilt 863;l-' l ./gcsw 86.0.0.3 'lt all;l+;lset SectorPort=864 Tilt 865;l-' ... i want to append;l nawk 'NR==14 && $NF!="Set."{print "l ./gcsw "r"... (4 Replies)
Discussion started by: gc_sw
4 Replies

8. Shell Programming and Scripting

sed append "\n" to end of every line in file

I know it sounds simple, but I want to e-mail the last 6 lines of a log file, which I have tailed into logresults.txt. I'm using echo -e "Subject:server results\nFrom:server log <user@domain.com>\n"`cat logresults.txt` | sendmail -t user@domain.com which works, but the body of the e-mail has... (4 Replies)
Discussion started by: unclecameron
4 Replies

9. Shell Programming and Scripting

each line as 20digit long append zero "0" in front

i have an ip file like 121 1213412 34345353 long file want to made each line as 20digit long append zero "0" in front as 121 become 00000000000000000121 (1 Reply)
Discussion started by: RahulJoshi
1 Replies

10. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question