Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Spacing off when files combined using awk or cat

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Spacing off when files combined using awk or cat
# 8  
Old 01-04-2016
The awk command I suggested in post #6 should fix it.

Code:
awk 'NF' *.txt > example.txt

# 9  
Old 01-04-2016
Those lines in example.txt are NOT empty, they contain sequences of <TAB>s, and, most important, DOS line terminators: <CR>.
# 10  
Old 01-04-2016
So would the below work? Thank you Smilie

Code:
for f in /home/cmccabe/Desktop/folder/*.txt ; do
     bname=`basename $f`
     pref=${bname%%.txt}
    sed 's/\r//' | | sed -E 's,\\t|\\r|\\n,,g' $f > /home/cmccabe/Desktop/new/${pref}_unix.txt
done

Last edited by cmccabe; 01-04-2016 at 06:00 PM.. Reason: added edit of new code
# 11  
Old 01-04-2016
As a rule I do not download any attachment from any forum. If it is true that you have lines containing tabs and spaces, and that the character return is present and you would like to remove those, please, try the following:

Code:
perl -ne 's/\r//; print unless /^(\W+)?$/' *.txt > example.txt

That would sanitized it for both instances.
# 12  
Old 01-05-2016
I apologize for the long post, I am trying to avoid attachments. If I merge file1,2,3 into one example.txt it appears that things do not copy over correctly. Then the awk does not result in the desired output. If I manually copy and paste the output is fine, but I have too many files to do that. cat doesn't seem to work either and I'm not sure what else to try. Thank you Smilie.

file1.txt
Code:
name	31	Index	Chromosomal Position	Gene	Inheritance
		122	2106725	TSC2	AD
		124	2115481	TSC2	AD
		121	2105400	TSC2	AD
		82	135782221	TSC1	AD
		81	135782026	TSC1	AD
		126	2138218	TSC2	AD
		123	2113107	TSC2	AD
		125	2126142	TSC2	AD

file2.txt
Code:
name2	12	Index	Chromosomal Position	Gene	Inheritance
		1	43396568	SLC2A1	AD, AR

file3.txt
Code:
name3	20	Index	Chromosomal Position	Gene	Inheritance
		188	2135240	TSC1	AD
		179	2103379	TSC1 AD
		191	2137899	TSC2	AD
		181	2110617	TSC2	AD
		190	2137857	TSC2	AD
		189	2137806	TSC2	AD
		186	2133798	TSC2	AD
		187	2135074	TSC2	AD
		180	2105400	TSC2	AD
		183	2122822	TSC2	AD
		192	2138218	TSC2	AD
		185	2125937	TSC2	AD
		184	2125788	TSC2	AD
		193	2138269	TSC2	AD
		182	2112981	TSC2	AD

Code:
perl -ne 's/\r//; print unless /^(\W+)?$/' *.txt > example.txt

example.txt
Code:
name	31	Index	Chromosomal Position	Gene	Inheritance
		122	2106725	TSC2	AD
		124	2115481	TSC2	AD
		121	2105400	TSC2	AD
		82	135782221	TSC1	AD
		81	135782026	TSC1	AD
		126	2138218	TSC2	AD
		123	2113107	TSC2	AD
		125	2126142	TSC2	ADname2	12	Index	Chromosomal Position	Gene	Inheritance
		1	43396568	SLC2A1	AD, ARname3	20	Index	Chromosomal Position	Gene	Inheritance
		188	2135240	TSC1	AD
		179	2103379	TSC1 AD
		191	2137899	TSC2	AD
		181	2110617	TSC2	AD
		190	2137857	TSC2	AD
		189	2137806	TSC2	AD
		186	2133798	TSC2	AD
		187	2135074	TSC2	AD
		180	2105400	TSC2	AD
		183	2122822	TSC2	AD
		192	2138218	TSC2	AD
		185	2125937	TSC2	AD
		184	2125788	TSC2	AD
		193	2138269	TSC2	AD
		182	2112981	TSC2	AD

awk command to match specific name and copy header row where the match was found:

Code:
awk '
NF == 7         {HD = $0 RS}
$3 == "TSC1"    {printf "%s%s\n", HD, $0
                 HD = ""
                }
' example.txt > TSC1.txt

TSC1.txt (current output)
Code:
name	31	Index	Chromosomal Position	Gene	Inheritance
		82	135782221	TSC1	AD
		81	135782026	TSC1	AD
		188	2135240	TSC1	AD
		179	2103379	TSC1 AD

Desired output
Code:
name	31	Index	Chromosomal Position	Gene	Inheritance
		82	135782221	TSC1	AD
		81	135782026	TSC1	AD
name3	20	Index	Chromosomal Position	Gene	Inheritance
		188	2135240	TSC1	AD
		179	2103379	TSC1	AD

# 13  
Old 01-05-2016
It appears that your files file{1,2}.txt do not terminate in a new line.
Please, give it a try at:
Code:
perl -nle 's/\r//; print unless /^(\W+)?$/' file{1..3}.txt > example.txt

The -l will ensure that a new line is added after each print
Last edited by Aia; 01-05-2016 at 05:45 PM.. Reason: mispell
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Lost redirecting stderr & stdout to 3 files - one each plus combined

Hi folks I need/want to redirect output (stdout, stderr) from an exec call to separate files. One for stderr only and two(!) different (!) ones for the combined output of stderr and stdout. After some research and testing i got this so far : (( exec ${command} ${command_parameters} 3>&1... (6 Replies)
Discussion started by: MDominok
6 Replies

2. Shell Programming and Scripting

Combined sed+awk for lookup csv file

have written a combined sed+awk to perform a lookup operation which works but looking to enhance it. looking to match a record using any of the comma separated values + return selected fields from the record - including the field header. so: cat foo make,model,engine,trim,value... (6 Replies)
Discussion started by: jack.bauer
6 Replies

3. Shell Programming and Scripting

awk combined with an IF

Hi everybody! I try to printout a csv-file with the exeption of cell $1 and $4. what i tried so far: awk '{for(i = 1; i<=NF; i++);if(i == 1 || i == 4);else print($i)}' file.csv ..any ideas how it work and why my example fails? Thanks in advance! IMPe (3 Replies)
Discussion started by: IMPe
3 Replies

4. UNIX for Dummies Questions & Answers

Grep and cat combined

Hello, i need to search one word (snp1) from many files and copy the content of the columns of this word in new file. example: file 1: SNP BP CHR P snp1 1 3 0.01 snp2 2 2 0.05 . . file 2: SNP BP CHR P snp1 1 3 0.06 snp2 2 2 0.3 output... (6 Replies)
Discussion started by: biopsy
6 Replies

5. Shell Programming and Scripting

awk or sed: change the color of a column w/o screwing up column spacing

Hey folks. I wrote a little awk script that summarizes /proc/net/dev info and then pipes it to the nix column command to set up column spacing appropriately. Here's some example output: Iface RxMBytes RxPackets RxErrs RxDrop TxMBytes TxPackets TxErrs TxDrop bond0 9 83830... (3 Replies)
Discussion started by: ryran
3 Replies

6. UNIX for Dummies Questions & Answers

awk for scientific notion and decimal combined data

Dea all, I have a question. I have a column of numbers with scientific notion and decimal combined data. I want to print it only if the number <0.05 using awk. however the very small numbers with scientific notion is not selected. Do any one know how to solve it? Thanks! example as below: ... (4 Replies)
Discussion started by: forevertl
4 Replies

7. Shell Programming and Scripting

Cat 114 files using grep or awk to pull muliple fields

Files xxxxxxx.txt ------------------------------------------------------------------------------------------------------------------------------------ Req.By: xxxxxxx WABUSH MINES - xxxxxx MINE (1001) Page: 1 Run on: 12/14/09... (4 Replies)
Discussion started by: sctxms
4 Replies

8. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

9. UNIX for Dummies Questions & Answers

Combined many txt files by columns

Hi, I want to combined 500 .txt files (each file only has one line) to a single txt file (A.txt). for example, A1.txt with 12 23 34 45 A2.txt with 56 67 78 89 ...etc, so I have A1.txt, A2.txt... , A499.txt, A500.txt (1 Reply)
Discussion started by: AMBER
1 Replies

10. Shell Programming and Scripting

Awk not printing the last combined column

nawk -F "|" 'FNR==NR {a=$2 OFS $3 OFS $4 OFS $5 OFS $6;next}\ {if ($5 in a)print $1,"test",$5,a, $2,$3,$4 OFS OFS OFS OFS OFS OFS OFS OFS $2-$3-$4 ; \ else print $1,"Database",$5 OFS OFS OFS OFS OFS OFS $2,$3,$4 OFS OFS OFS OFS OFS OFS OFS OFS $2-$3-$4 }' OFS="|" \ file1 file2 > file3 This... (5 Replies)
Discussion started by: pinnacle
5 Replies
Login or Register to Ask a Question