Sponsored Content
Top Forums Shell Programming and Scripting parsing data from a big file using keys from another smaller file Post 302511395 by Corona688 on Wednesday 6th of April 2011 05:35:16 PM
Old 04-06-2011
Code:
< datafile awk '
BEGIN { FS="\t"
        # get the very first key.
        getline key < "keyfile" }
{
        # If the data's ahead in order, read keys until you catch up
        # but don't read keys past EOF.
        while(key && (key < $1))
                getline key < "keyfile"

        if(key && (key == $1))
                print;
}'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

3. Shell Programming and Scripting

perl help to split big verilog file into smaller ones for each module

Hi I have a big verilog file with multiple modules. Each module begin with the code word 'module <module-name>(ports,...)' and end with the 'endmodule' keyword. Could you please suggest the best way to split each of these modules into multiple files? Thank you for the help. Example of... (7 Replies)
Discussion started by: return_user
7 Replies

4. Shell Programming and Scripting

How to cut some data from big file

How to cut data from big file my file around 30 gb I tried "head -50022172 filename > newfile.txt ,and tail -5454283 newfile.txt. It's slowy. afer that I tried sed -n '46467831,50022172p' filename > newfile.txt ,also slow Please recommend me , faster command to cut some data from... (4 Replies)
Discussion started by: almanto
4 Replies

5. Shell Programming and Scripting

Helping in parsing subset of text from a big results file

Hi All, I need some help to effectively parse out a subset of results from a big results file. Below is an example of the text file. Each block that I need to parse starts with "reading sequence file 10.codon" (next block starts with another number) and ends with **p-Value(s)**. I have given... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

6. Shell Programming and Scripting

Sort a big data file

Hello, I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field. I'm trying to sort with "sort" command and it brings me 6 minutes. I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Discussion started by: rubber08
2 Replies

7. Shell Programming and Scripting

Segment a big file into smaller ones

Greeting to all. I have big text file that I would like to segment into many smaller files. Each file should be maximum 10 000 lines. The file is called time.txt. after the execution of the file I would like to have. time_01.txt, time_02, txt, ...,time_n.txt Can anybody help. Br. (2 Replies)
Discussion started by: flash80
2 Replies

8. Shell Programming and Scripting

parsing characters and number from a big file with brackets

I have a big file with many brackets () in it from which I need to parse number characters and numbers. Below is an example of my file 14 (((A__0:0.02,B__1:0.3)0:0.04,C__0:0.025)2:0.01),(D__0:0.00978,E__2:0.01031)1:0.00362; 15... (1 Reply)
Discussion started by: Lucky Ali
1 Replies

9. Shell Programming and Scripting

Parsing data using keys from one file

I have 2 text files where I need to parse data from file 2 using the data from file 1. Below are my sample files File 1 (tab delimited) 257 350 670 845 725 1025 767 820 ... .... .... file 2 (tab delimited) 220..450 TA AB650 ABCED 520..850 GA AB720 ABCDE 700..1100 TC AB820 ABCDE... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

10. Shell Programming and Scripting

Extract data according to keys from filename mentioned in file

Hello experts, I want to join a file with files whosE names are mentioned in one of the columns of the same file. File 1 t1,a,b,file number 1 t1,a,c,file number 1 t2,c,d,file number 2 t2,c,e,file number 2 t2,c,f,file number 2 t2,c,g,file number 2 t3,e,f,file number 3 file number 1... (3 Replies)
Discussion started by: ritakadm
3 Replies
SSH-COPY-ID(1)						    BSD General Commands Manual 					    SSH-COPY-ID(1)

NAME
ssh-copy-id -- copy public keys to a remote host SYNOPSIS
ssh-copy-id [-lv] [-i keyfile] [-o option] [-p port] [user@]hostname DESCRIPTION
The ssh-copy-id utility copies public keys to a remote host's ~/.ssh/authorized_keys file (creating the file and directory, if required). The following options are available: -i file Copy the public key contained in file. This option can be specified multiple times and can be combined with the -l option. If a private key is specified and a public key is found then the public key will be used. -l Copy the keys currently held by ssh-agent(1). This is the default if the -i option was not specified. -o ssh-option Pass this option directly to ssh(1). This option can be specified multiple times. -p port Connect to the specified port on the remote host instead of the default. -v Pass -v to ssh(1). The remaining arguments are a list of remote hosts to connect to, each one optionally qualified by a user name. EXIT STATUS
The ssh-copy-id utility exits 0 on success, and >0 if an error occurs. EXAMPLES
To send a specific key to multiple hosts: $ ssh-copy-id -i /path/to/keyfile.pub user@host1 user@host2 user@host3 HISTORY
The ssh-copy-id utility was written by Eitan Adler <eadler@FreeBSD.org> as a drop-in replacement for an existing utility included with OpenSSH. BSD
February 28, 2014 BSD
All times are GMT -4. The time now is 10:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy