Sponsored Content
Full Discussion: a problem with large files
Top Forums Shell Programming and Scripting a problem with large files Post 302436507 by m_wassal on Monday 12th of July 2010 07:58:45 AM
Old 07-12-2010
Quote:
Originally Posted by Scrutinizer
Or since 2,3 million lines will have to be deleted, this may be necessary:
Code:
awk 'NR==1{getline x<f}NR==x{print;getline x<f}' f=file file1 > file2

assuming that &quot;file&quot; is sorted numerically. otherwise sort -n first.

Last edited by m_wassal; 07-12-2010 at 09:15 AM.. Reason: forgot writing the contents
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Problem in processing a very large file.

Hi Friends, Getting an error while processing a very large file using an sqlloader........ The file is larger than 2 GB. Now need to change the compiler to 64-bit so that the file can be processed. Is there any command for the same. Thanks in advance. (1 Reply)
Discussion started by: Rohini Vijay
1 Replies

2. Shell Programming and Scripting

problem with 0 byte and large files

how to remove all zero byte files in a particular directory and also files that are morew than 1GB. pLEASE let me know (3 Replies)
Discussion started by: dsravan
3 Replies

3. UNIX for Advanced & Expert Users

Large file FTP problem

We are experiencing a problem on a lengthy data transfer by FTP through a firewall. Since there are two ports in use on a ftp transfer (data and control), one sits idle while the other's transfering data. The idle port (control) will get timed out and the data transfer won't know that it's... (3 Replies)
Discussion started by: rprajendran
3 Replies

4. UNIX for Dummies Questions & Answers

Problem using find with prune on large number of files

Hi all; I'm having a problem when want to list a large number of files in current directory using find together with the prune option. First i used this command but it list all the files including those in sub directories: find . -name "*.dat" | xargs ls -ltr Then i modified the command... (2 Replies)
Discussion started by: ashikin_8119
2 Replies

5. UNIX for Dummies Questions & Answers

Large file problem

I have a large file, around 570 gb that I want to copy to tape. However, my tape drive will load only up to 500 gb. I don't have enough space on disk to compress it before copying to tape. Can I compress and tar to tape in one command without writing a compressed disk file? Any suggestions... (8 Replies)
Discussion started by: iancrozier
8 Replies

6. UNIX for Dummies Questions & Answers

Large Problem with nautilus

Hi, I am a torrent-maniak and I use Transmission. All things were good but Nautilus begun to show problem while I was runnning Transmission.Its situation was becoming worse and worse. Now, when I boot I can hardly open a nautilus window and browse my files.It will "stack" in seconds for sure! I... (2 Replies)
Discussion started by: hakermania
2 Replies

7. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

8. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

9. Shell Programming and Scripting

A Large Percent Problem

Hello everyone, I have two matrices at same sizes. I need to re-calculate the numbers in matrix A according to the percentages in martix B it is like matrix A is 10.00 20.00 30.00 40.00 60.00 70.00 80.00 90.00 20.00 30.00 80.00 50.00 martix B is 00.08 00.05 ... (2 Replies)
Discussion started by: miriammiriam
2 Replies
GETLINE(3)						     Linux Programmer's Manual							GETLINE(3)

NAME
getline, getdelim - delimited string input SYNOPSIS
#define _GNU_SOURCE #include <stdio.h> ssize_t getline(char **lineptr, size_t *n, FILE *stream); ssize_t getdelim(char **lineptr, size_t *n, int delim, FILE *stream); DESCRIPTION
getline() reads an entire line, storing the address of the buffer containing the text into *lineptr. The buffer is null-terminated and includes the newline character, if a newline delimiter was found. If *lineptr is NULL, the getline() routine will allocate a buffer for containing the line, which must be freed by the user program. Alter- natively, before calling getline(), *lineptr can contain a pointer to a malloc()-allocated buffer *n bytes in size. If the buffer is not large enough to hold the line read in, getline() resizes the buffer to fit with realloc(), updating *lineptr and *n as necessary. In either case, on a successful call, *lineptr and *n will be updated to reflect the buffer address and size respectively. getdelim() works like getline(), except a line delimiter other than newline can be specified as the delimiter argument. As with getline(), a delimiter character is not added if one was not present in the input before end of file was reached. RETURN VALUE
On success, getline() and getdelim() return the number of characters read, including the delimiter character, but not including the termi- nating null character. This value can be used to handle embedded null characters in the line read. Both functions return -1 on failure to read a line (including end of file condition). ERRORS
EINVAL Bad parameters (n or lineptr is NULL, or stream is not valid). EXAMPLE
#define _GNU_SOURCE #include <stdio.h> #include <stdlib.h> int main(void) { FILE * fp; char * line = NULL; size_t len = 0; ssize_t read; fp = fopen("/etc/motd", "r"); if (fp == NULL) exit(EXIT_FAILURE); while ((read = getline(&line, &len, fp)) != -1) { printf("Retrieved line of length %zu : ", read); printf("%s", line); } if (line) free(line); return EXIT_SUCCESS; } CONFORMING TO
Both getline() and getdelim() are GNU extensions. They are available since libc 4.6.27. SEE ALSO
read(2), fopen(3), fread(3), gets(3), fgets(3), scanf(3) GNU
2001-10-07 GETLINE(3)
All times are GMT -4. The time now is 04:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy