Sponsored Content
Top Forums Shell Programming and Scripting Matching 10 Million file records with 10 Million in other file Post 302655439 by methyl on Wednesday 13th of June 2012 07:29:01 AM
Old 06-13-2012
Any comment about the order of the data in the files and whether there is a one-for-one match between the two files (in which case the paste command might be suitable?

Edit: Posts crossed. I can see that neither file is in any particular order and that your sample does not show a one-for-one match.

It's going to be necessary to sort both files. Does the order of the final output data matter?

Last edited by methyl; 06-13-2012 at 08:35 AM..
 

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data from large file 80+ million records

Hello, I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file. What will be the besat and fastest way to extract the ne file. sample file format :--... (2 Replies)
Discussion started by: learner16s
2 Replies

2. Shell Programming and Scripting

sort a file which has 3.7 million records

hi, I'm trying to sort a file which has 3.7 million records an gettign the following error...any help is appreciated... sort: Write error while merging. Thanks (6 Replies)
Discussion started by: greenworld
6 Replies

3. What is on Your Mind?

Pick a Number Between 0 and 20 for 1 Million Bits

Here is an easy game! I wrote a number between 0 and 20 (that can include 0 and 20) on a piece of paper. I am staring at it now, imagining the number so you can read my mind ;) Reply once, and only once, with a number from 0 to 20 and the first person to guess it wins 1,000,000 Bits. ... (24 Replies)
Discussion started by: Neo
24 Replies

4. Shell Programming and Scripting

Tail 86000 lines from 1.2 million line file?

I have a log file that is about 1.2 million lines long and about 300MB. we need a way to clean up this file and only keep the last few thousand lines. if i use tail command we run our of memory as the file is too big. I do have a key word to match on. example, we want to keep every line... (8 Replies)
Discussion started by: robsonde
8 Replies

5. UNIX for Dummies Questions & Answers

Pls. help with script to remove million files

Hi, one of the server, log directory was never cleaned up. We have so many files. I want to remove all the files that starts with dfr* but I get error message when I use the *. rm qfr* bash: /usr/bin/rm: Arg list too long I am trying to write this script but not working. ... (4 Replies)
Discussion started by: samnyc
4 Replies

6. UNIX for Dummies Questions & Answers

Deleting a million of files ..

Hi, Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies

7. UNIX for Dummies Questions & Answers

Add 1 million columns

Hi, here is my problem: I've got a file with 6 columns (file1): a b c d e f a b c d e f a b c d e f a b c d e f I need to add 1 million columns to this file, each column needs to be a zero. Here is how the result file (file2) should look like (for the sake of the example, I've only... (7 Replies)
Discussion started by: zajtat
7 Replies
pilot-addresses(1)					      General Commands Manual						pilot-addresses(1)

NAME
pilot-addresses - read and write address book databases to and from a Palm handheld device, such as those made from Palm, Handspring, Han- dera, TRGPro, Sony or other Palm Compatible Handheld PDA device SYNOPSIS
pilot-addresses -p <port> [-c category ] [-d category ] [-r file | -w file ] (Note that some options are not shown above) DESCRIPTION
pilot-addresses allows the user to read all entries in the Palm address book database, write new entries into the database, and delete a category or delete all entries in the database. TARGET DEVICE
The default serial device used to communicate with a Palm is /dev/pilot. If the environment variable $PILOTPORT is set, its value will override the default. A serial device specified on the command-line will be used regardless of any $PILOTPORT setting. OPTIONS
Several options exist, including... -p --port <port>, Use device file port to communicate with the Palm handheld device. If this is not specified, will look for the $PILOTPORT environ- ment variable. If both are not found, will fall back to /dev/pilot. -h --help Display help synopsis for pilot-addresses -v --version Display version of pilot-addresses -a Augments fields in address book records with additional information. The augmented information is placed before and separated from the field with a semi-colon, (;). Augmented information includes: category_name - placed in front of each record or ["Work" | "Home" | "Fax" | "Other" | "E-mail" | "Main" | "Pager" | "Mobile" ] - placed in front of each phone number field. Empty fields are not augmented. -c category Install records to category category by default. Normally pilot-addresses uses Unfiled as the default category. This option is over- ridden by the category specified in the record with the -a option. -d category Delete all records in the specified category before installing new records. -D Delete all address book records in all categories. Obviously, be very careful with this one. -e Escape all special characters with a backslash. This enables you to read and write entries with newline characters in a field or note. -q Causes pilot-addresses to be quiet and not prompt you to press the HotSync button. -r file Reads records from file and install them to the Palm address book database. (Use the -w file to get a template file for input records.) -t delim Include category in each record, use the delimiter specified to separate all fields of a record. Delimiters are specified as fol- lows: 3=tab, 2=;, 1=,. This overrides the default delimiter of comma between fields and semi-colon between a field's augmented information. (Please note that this may generate confusing results when used with the -a option.) -T Write a header line with field titles as the first line of the data file. -w file Get all address book records from the Palm address book database and writes them into file USAGE
The program will connect to a target device and port, prompt the user to HotSync, and perform the requested read or write operation speci- fied by the user. EXAMPLES
To write all address records in a Palm to the file addrbook.csv: pilot-addresses -w addrbook.csv or pilot-addresses -p /dev/irnine -w addrbook.csv To read the address book records in the file addrbook.csv and install them on a Palm: pilot-addresses -r addrbook.csv To read the address book records in the file addrbook.csv and place them into the Palm address book database category Special after first deleting all current records in the Special category on the palm: pilot-addresses -c Special -d Special -r addrbook.csv SEE ALSO
pilot-link(7) KNOWN BUGS
pilot-addresses has no known bugs. REPORTING BUGS
Report bugs at http://bugs.pilot-link.org/ AUTHOR
pilot-addresses originally written by Kenneth Albanowski, manual page was written by Robert Wittig <bob.wittig@gt.org>. Free Software Foundation Palm Computing Device Tools pilot-addresses(1)
All times are GMT -4. The time now is 09:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy