need help using "join" Post: 302163755

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers need help using "join" Post 302163755 by H2OBoodle on Saturday 2nd of February 2008 07:12:41 AM

02-02-2008

Registered User

Well my assumptions on what you were trying to do were bad, so my awk solution didn't pan out for me. However I gave it a shot with a really short python script, and I think I may have what you need.

To be honest, you still didn't give me a clear idea of what you wanted your output to look like, so here's what I assumed. If I'm wrong, then sorry, this is my last shot.

Using the first four lines of your data, I think you want this output:
0060122450000 2000550
0060122450001 2000632
0060122450002 2001374
0060122450006 2004006

The first number is from file_a with the first four chars chopped off. The second number has the first value stripped off.

If this is correct, then here's a super simple python script to get that for you:

script name: foo.py

# open the data files
fa = open('file_a','rb')
fb = open('file_b','rb')

#Go through the files line by line stripping out just the parts
# that you want to keep. Also strip the newlines from the end.

for line in fa:
bita = line[4:].strip('\n')
tmpb = fb.readline().strip('\n')
# You could add a chk here to ensure tmpb matches bita. You'd have to do some additional chopping though. With millions of records, I'd do it.
bitb = tmpb[-7:]
print bita,bitb

# close the files
fa.close()
fb.close()

Run the script like this:
shellPrompt$ python foo.py

The script makes the assumption that your data files are matched up correctly, with the entries matching position-wise all the way through. If they're not, then this won't work without modifications. Your data will be wrong if the values are shifted.

And finally, there are probably more elegant python or shell techniques of doing this, but this works.

Good luck.

Last edited by H2OBoodle; 02-02-2008 at 08:18 AM.. Reason: give more info on how to run the script, warn on data corruption if data is not aligned.

H2OBoodle

View Public Profile for H2OBoodle

Find all posts by H2OBoodle

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha

2. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and...

3. Shell Programming and Scripting

Remove ":" and join lines in outline file

I have a vim outliner file like this: Title title 2 :Testing now :testing 2 :testing 3 title 3 :testing :ttt :ttg Is there a way to use a script or command to remove...

4. Web Development

Perl join two files by "common" column

Hello; I am posting to get any help on my code that I have been struggling for some time. The project is to join two files each with 80k~180k rows. I want to merge them together by the shared common column. The problem of the shared column is partially matching, not exactly the same. File1:...

5. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone...

6. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2...

7. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing �To� e-mail address and column 3 contains �cc� e-mail address to include with same email. Sample input file, email.txt Below is an sample code where...

8. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing...

9. Shell Programming and Scripting

Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk

Hello, This post is already here but want to do this with another way Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: 1|123|jojo 1|NULL|bibi...

LEARN ABOUT DEBIAN

idle3

IDLE(1) 						      General Commands Manual							   IDLE(1)

NAME

       IDLE - An Integrated DeveLopment Environment for Python

SYNTAX

       idle [ -dins ] [ -t title ] [ file ...]

       idle [ -dins ] [ -t title ] ( -c cmd | -r file ) [ arg ...]

       idle [ -dins ] [ -t title ] - [ arg ...]

DESCRIPTION

       This  manual page documents briefly the idle command.  This manual page was written for Debian because the original program does not have a
       manual page.  For more information, refer to IDLE's help menu.

       IDLE is an Integrated DeveLopment Environment for Python. IDLE is based on Tkinter, Python's bindings to the Tk widget  set.  Features  are
       100%  pure  Python, multi-windows with multiple undo and Python colorizing, a Python shell window subclass, a debugger. IDLE is cross-plat-
       form, i.e. it works on all platforms where Tk is installed.

OPTIONS

       -h     Print this help message and exit.

       -n     Run IDLE without a subprocess (see Help/IDLE Help for details).

       The following options will override the IDLE 'settings' configuration:

       -e     Open an edit window.

       -i     Open a shell window.

       The following options imply -i and will open a shell:

       -c cmd Run the command in a shell, or

       -r file
	      Run script from file.

       -d     Enable the debugger.

       -s     Run $IDLESTARTUP or $PYTHONSTARTUP before anything else.

       -t title
	      Set title of shell window.

       A default edit window will be bypassed when -c, -r, or - are used.

       [arg]* and [file]* are passed to the command (-c) or script (-r) in sys.argv[1:].

EXAMPLES

       idle   Open an edit window or shell depending on IDLE's configuration.

       idle foo.py foobar.py
	      Edit the files, also open a shell if configured to start with shell.

       idle -est "Baz" foo.py
	      Run $IDLESTARTUP or $PYTHONSTARTUP, edit foo.py, and open a shell window with the title "Baz".

       idle -c "import sys; print sys.argv" "foo"
	      Open a shell window and run the command, passing "-c" in sys.argv[0] and "foo" in sys.argv[1].

       idle -d -s -r foo.py "Hello World"
	      Open a shell window, run a startup script, enable the debugger, and run foo.py, passing "foo.py" in sys.argv[0] and "Hello World" in
	      sys.argv[1].

       echo "import sys; print sys.argv" | idle - "foobar"
	      Open a shell window, run the script piped in, passing '' in sys.argv[0] and "foobar" in sys.argv[1].

SEE ALSO

       python(1).

AUTHORS

       Various.

								 21 September 2004							   IDLE(1)