How to make paste -d second file print down while looping?


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to make paste -d second file print down while looping?
# 15  
Old 08-30-2019
Quote:
Originally Posted by bigvito19
I'm trying to get the file on the right to print or generated down the file on the left without the left file printing too. I don't want the left file to print at all. I also want to merge both lines as it prints down the file. Or if there is a way to print files in opposite directions creating new lines. I don't want repeated lines.

The files I am now testing are 2.5gbs each, will try to test 40+gbs later.

I'm using xfce xubuntu 4.12

My awk version is 1.3.3
I guess I wasn't clear enough in my request. Please explain what diagnostics or erroneous output you have received from awk that make you believe awk is unable to process large files on your system. I don't know of any reason (other than lack of memory) that would keep awk on xfce xubuntu from processing large files.
# 16  
Old 08-30-2019
Quote:
Originally Posted by Don Cragun
I guess I wasn't clear enough in my request. Please explain what diagnostics or erroneous output you have received from awk that make you believe awk is unable to process large files on your system. I don't know of any reason (other than lack of memory) that would keep awk on xfce xubuntu from processing large files.
it just said line 11: 7605 Killed

I can open a file with a python script or with the paste cmd but with bigger files seems like I can't use the awk.
# 17  
Old 08-30-2019
What awk script were you running when you got the diagnostic: line 11: 7605 killed?

How big were the two files you were having awk read into memory when you got that diagnostic.

How much memory is installed on your system?

If you were redirecting the output from your awk script to a file, what was the size of that output file and how much free space was there on the filesystem to which you were writing that file when awk was killed?
# 18  
Old 08-30-2019
Quote:
Originally Posted by Don Cragun
What awk script were you running when you got the diagnostic: line 11: 7605 killed?

How big were the two files you were having awk read into memory when you got that diagnostic.

How much memory is installed on your system?

If you were redirecting the output from your awk script to a file, what was the size of that output file and how much free space was there on the filesystem to which you were writing that file when awk was killed?


Code:
 awk '
NR==FNR {a[c++]=$0; next;}
{b[++d]=$0;}
END { for (i=0; i<c; i++) {
         for (j=0; j<c; j++) print a[j], b[(j-i)<0?(c+j-i):(j-i)];
         print _;
      }
}' a b


I was using the above code I got from the first page of this post.

The files I tried to test was 172MB each

I got enough memory to run big files.

I was directing the output to the terminal not to a file.
Moderator's Comments:
Mod Comment Please use CODE tags whenever you post sample input, sample output, or code segments.

Last edited by Don Cragun; 08-30-2019 at 08:18 PM..
# 19  
Old 08-30-2019
Quote:
Originally Posted by bigvito19
Code:
 awk '
NR==FNR {a[c++]=$0; next;}
{b[++d]=$0;}
END { for (i=0; i<c; i++) {
         for (j=0; j<c; j++) print a[j], b[(j-i)<0?(c+j-i):(j-i)];
         print _;
      }
}' a b


I was using the above code I got from the first page of this post.

The files I tried to test was 172MB each

I got enough memory to run big files.

I was directing the output to the terminal not to a file.
Moderator's Comments:
Mod Comment Please use CODE tags whenever you post sample input, sample output, or code segments.
OK. My mistake. The diagnostic was not from awk, it was from your shell script.

What are the contents of file containing your shell script?

Was there any output from awk before you got that diagnostic, and, if there was, how much?
# 20  
Old 08-30-2019
Quote:
Originally Posted by Don Cragun
OK. My mistake. The diagnostic was not from awk, it was from your shell script.

What are the contents of file containing your shell script?

Was there any output from awk before you got that diagnostic, and, if there was, how much?

I only put 2 files in the shell script, I had put 2 very small files to test it out and I was getting outputs from that. Then when I tried 2 bigger files, it didn't have any output after that then it said killed.
# 21  
Old 08-30-2019
Quote:
Originally Posted by bigvito19
I only put 2 files in the shell script, I had put 2 very small files to test it out and I was getting outputs from that. Then when I tried 2 bigger files, it didn't have any output after that then it said killed.
OK. Since you won't answer my questions, there is nothing I can do to help.

I leave it up to you to figure out what about line 11 in your shell script doesn't work with big files.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to make second file print down the whole first file?

Using Python I have 2 text files (big files over 1gb) that opens side by side on the same line in terminal, but I want the file on the right to print down the other file while the file on the left is stationary or displayed all at once. I want to print text file 2 through all of text file 1. ... (6 Replies)
Discussion started by: bigvito19
6 Replies

2. Shell Programming and Scripting

How to cut a pipe delimited file and paste it with another file to form a comma separated outputfile

Hello ppl I have a requirement to split (cut in unix) a file (A.txt) which is a pipe delimited file into A1.txt and A2.txt Now I have to join (paste in unix) this A2.txt with external file A3.txt to form output file A4.txt which should be CSV (comma separated file) so that third party can... (25 Replies)
Discussion started by: etldev
25 Replies

3. Shell Programming and Scripting

Looping structure to make up for lack of bash GOTO

Hello, I am re-processing some files when a specific condition is met. The condition is read from the filename. Since files may need to be re-processed a number of times before they no longer meet the condition, I need to know when to stop re-processing. I am having trouble visualizing the... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

4. Shell Programming and Scripting

Looping inside directories based on a file which contains file directory list

Hi All, Please help. I have got a file which contains a listing of a file and some directories after it, one by one. I am calling this file xyz.txt here file1 dir1 dir2 dir3 dir4 file2 dir5 dir6 dir7 dir8 file3 dir9 dir10 dir11 dir12 (6 Replies)
Discussion started by: Piyush Jakra
6 Replies

5. Shell Programming and Scripting

FILE_ID extraction from file name and save it in CSV file after looping through each folders

FILE_ID extraction from file name and save it in CSV file after looping through each folders My files are located in UNIX Server, i want to extract file_id and file_name from each file .and save it in a CSV file. How do I do that? I have folders in unix environment, directory structure is... (15 Replies)
Discussion started by: princetd001
15 Replies

6. Shell Programming and Scripting

Make pwd print escape character

I decided I wanted to have the cd command print my full working directory after each cd command, so I put this cw command in .bashrc as a function. cw () { cd "${1}" pwd }While this works I would like pwd to print escapes when a space in a directory name exists. This would... (7 Replies)
Discussion started by: jelloir
7 Replies

7. Shell Programming and Scripting

How to make a long print string to shotcut form in perl?

print "1.readfromfile\n2.add_ex1(4,5)\n3.add_ex2(11,5)\n4.add_ex3(9,3)\n5.add_ex4(91,4)\n"; How to do it in this form: print "1.readfromfile\n 2.add_ex1(4,5)\n 3.add_ex2(11,5)\n 4.add_ex3(9,3)\n 5.add_ex4(91,4)\n"; (3 Replies)
Discussion started by: cola
3 Replies

8. Programming

makeutility: how to get the make-file name inside of the make-file?

How I can get the current make-file name in a make-file So, if I run make with specified file:make -f target.mak is it possible to have the 'target' inside of the that 'target.mak' from the file name? (2 Replies)
Discussion started by: alex_5161
2 Replies

9. Shell Programming and Scripting

Paste content of a file to another file and make it as columned

Pls help me on this. I have to 2 files like shown below: File 1 TAIJM AXPKIM BEMGW File 2 PXMPA JYGE IMJP What i want to do is to paste both file to a new file on thir format: File 3 TAIJM PXMPA AXPKIM JYGE BEMGW IMJP I tried cat and print, but it doesn't work. Cn... (6 Replies)
Discussion started by: kingpeejay
6 Replies

10. UNIX for Dummies Questions & Answers

How to print something in make utility

Hi want to know the syntax of printing something (value or variable) in GNU make utility. I give this in the Makefile: echo "Hi" OR @echo "Hi" But I only get error this when I run make (at the line where I have echo): Makefile:9: *** missing separator. Stop. Whats the problem? How can... (2 Replies)
Discussion started by: radiatejava
2 Replies
Login or Register to Ask a Question