Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Character limitation in Join command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Character limitation in Join command
# 1  
Old 10-08-2009
Character limitation in Join command
Hi,
I'm trying to compare and join two files using join command. One of my files length is 3000. Join is not working for this file. It truncates character after 1600 lines. Anyone know a solution for this?

the command i used is

Comparing the 1st column of the first file and 3rd column of third file. The out put doesnt have characters after 1600 if matched.

Code:
join -t@ -1 1 -2 3 $cross_reference_file $input_file > $input_temp

# 2  
Old 10-08-2009
Do you mean record length - the number of characters in a line?
Code:
getconf LINE_MAX

Shows 2048 on a lot of UNIX systems. This is the maximum line size you can read and process. 3000 is larger than 2048.

Also join will drop non-matching records if that is what you are asking.
Quote:
There is one line in the output for each pair of lines in
file1 and file2 that have identical join fields.
In this case if only 1600 match you get 1600 lines of output.

I cannot tell which question you are asking.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed 255 Character Limitation

Hello, I am using sed command to place a comma dynamically in certain positions. When the position variable exceeds 255 characters, it errors out. I would appreciate if someone can point in the correct direction sed 's/^\(.\{'"$pos"'\}\)./\1,/' ragha.txt > ragha3.txt If $pos > 255,... (8 Replies)
Discussion started by: ragha81
8 Replies

2. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

3. Shell Programming and Scripting

Limitation on rm command

Hi all, does any one know ,if there is any limitation on rm command limitation referes here as a size . Ex:when my script try to rum rm command which have size of nearly 20-22 GB ..CPU load gets high ? if anyone know the relation of CPU load and limitation of rm command . (8 Replies)
Discussion started by: niteshagrawal06
8 Replies

4. Shell Programming and Scripting

Remove newline character or join the broken record

Hi, I have a very huge file, around 1GB of data. I want to remove the newline characters in the file but not preceded by the original end delimiter {} sample data will look like this 1234567 abcd{} 1234sssss as67 abcd{} 12dsad3dad 4sdad567 abcdsadd{} this should look like this... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies

5. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

6. UNIX for Advanced & Expert Users

Find command -size option limitation ?

Hi All, I ran code in test environment to find the files more than 1TB given below is a snippet from code: FILE_SYSTEM=/home/arun MAX_FILE_LIMIT=1099511627776 find $FILE_SYSTEM -type f -size +"$MAX_FILE_LIMIT"c -ls -xdev 2>/dev/null | while read fname do echo "File larger than... (3 Replies)
Discussion started by: Arunprasad
3 Replies

7. Shell Programming and Scripting

join (pls help on join command)

Hi, I am a new learner of join command. Some result really make me confused. Please kindly help me. input: file1: LEO oracle engineer 210375 P.Jones Office Runner ID897 L.Clip Personl Chief ID982 S.Round UNIX admin ID6 file2: Dept2C ID897 6 years Dept5Z ID982 1 year Dept3S ID6 2... (1 Reply)
Discussion started by: summer_cherry
1 Replies

8. Shell Programming and Scripting

Size limitation in Tar command

Hi to every body there, I am new this forum and this is my first post. I am a new user of Unix, is there any size limitation of files while creating tar file. Thanks in advance (4 Replies)
Discussion started by: Manvar Khan
4 Replies

9. Shell Programming and Scripting

Limitation of ls command

Hi, Iam using an alias to get the file count from one directory using normal ls command like ls file*|wc -l.If my file increases more than 35,000 ,my alias is not working.It shows that arg list too long. is that can be limitation of ls or problem in alias? I would appreciate if anyone can... (2 Replies)
Discussion started by: cskumar
2 Replies

10. HP-UX

HP-UX 11i - File Size Limitation And Number Of Folders Limitation

Hi All, Can anyone please clarify me the following questions: 1. Is there any file size limitation in HP-UX 11i, that I can able to create upto certain size of file (say 2 GB) and not more then that???? 2. At max. how many files we can able to keep inside a folder???? 3. How many... (2 Replies)
Discussion started by: sundeep_mohanty
2 Replies
Login or Register to Ask a Question