Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How join works and the specific parameters to my problem?
Homework and Emergencies Homework & Coursework Questions How join works and the specific parameters to my problem? Post 302412499 by Lechnology on Tuesday 13th of April 2010 01:38:12 AM
Old 04-13-2010
How join works and the specific parameters to my problem?
1. The problem statement, all variables and given/known data:
I have two files created from extracting data off of two CSV files, one containing class enrollment on a specific quarter and the other containing grades for that specific quarter. The Enrollment file generated contains course name, instructor and catalog number and a few other columns. The Grade file generated contains GPA and catalog number and a few other columns.

Both files have over 5k rows. I need to join the two files using the catalog number as the key field.

The Enrollment CSV file had some entries with Catalog No. missing, these were, in fact, subtotal entries and they were filtered out after the file was created.

2. Relevant commands, code, scripts, algorithms:
awk was used to generated the files using "|" as the field separator.

Example:
Course file
...
Catalog No. | ART 101 | instructor | Intro to Art
...

Grade file
...
Catalog No. | ART 101 | GPA
...

I need to use join to create a merged file of the two based on the Catalog No.

Example:
Merge file
...
Catalog No. | ART 101 | Instructor | Intro to Art | GPA
...

The original CSV files have the courses grouped by department (i.e. ART, CRIM, ENGR, CS, etc.)

3. The attempts at a solution (include all code and scripts):
I've attempted two approaches, one where both files are sorted based on Catalog No. and one without sorting.

Every relevant join parameters were used in an attempt to merge the two files but each time, I get either an empty merged file or it just contains one or the other file in it.

join -t"|" -o'1.1 1.2 1.3 2.1 2.3' Courses.txt Grades.txt > Merged.txt

empty

join -t"|" -o'1.1 1.2 1.3 2.1 2.3' Courses.txt Grades.txt > Merged.txt

Gave me: 10000|GEOL|150|EARTH REVEALED|Professor||||

The |||| were where the GPA and other stuff from Grades are supposed to be.

join -t"|" -1 1 -2 1 Courses.txt Grades.txt > Merged.txt
empty

join -t"|" -a1 Courses.txt Grades.txt > Merged.txt
Just Courses entry, none from Grades. Same thing with -a2.

-j was also used, as well as.

Keep in mind I sorted the files so that the catalog numbers would match line to line.

All the examples I've seen regarding how join works has the key fields in both files in order and correspond almost line-to-line so this sort of suggest that join does not search the entire file for matches.

California State University, Los Angeles, California, USA. Albert Cervantes, CS 345

You guys got strange requirements. Gee, where's Los Angeles? In Canada?[COLOR="#738fbf"]
Last edited by Lechnology; 04-14-2010 at 03:32 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Plink problem....only works if passwd is in the script

I have a one line bat script run off a XP machine that tar's and compresses some files from a Sol 8 box. It goes something like this (a bit simplified).... plink -pw <passwd> user@host "tar -cvf - -C / tmp/a_file | compress " > a_file.tar.Z So this works....and it's worked many times. But now... (3 Replies)
Discussion started by: Yinzer955i
3 Replies

2. Shell Programming and Scripting

A join problem?

Hi everybody, I am hoping somebody here will be either be able to solve my troubles or at least give me a push in the right direction :) . I am developing a shell script to read in 4 different files worth of data that each contain a list of: username firstname secondname group score I... (2 Replies)
Discussion started by: jamjamjammie
2 Replies

3. Programming

Problem with Pipes => Only works first pipe

Hi! I'm having problems with pipes... I need comunnications with childs processes and parents, but only one child can comunnicate with parent (first child), others childs can't. A brief of code: if(pipe(client1r)<0){ perror("pipe"); } ... (1 Reply)
Discussion started by: serpens11
1 Replies

4. UNIX for Dummies Questions & Answers

problem with join

So I want to join two files that have a lot of rows The file named gen1 has 2 columns: head gen1 1008567 0.4026931012 1119535 0.7088912314 1120590 0.7093805634 1145994 0.7287952590 1148140 0.7313924434 1155173 0.7359550430 1188481 0.7598914553 1201155 0.7663406553 1206921... (2 Replies)
Discussion started by: peanuts48
2 Replies

5. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

6. Shell Programming and Scripting

how to retrieve specific parameters using a xml tag

Hi, I have the following code in my xml file: <aaaRule loginIdPattern=".*" orgIdPattern=".*" deny="false" /> <aaaRuleGroup name="dpaas"> <aaaRule loginIdPattern=".*" orgIdPattern=".*" deny="false" /> I want to retrieve orgIdPattern and loginIdPattern parameter value based on... (2 Replies)
Discussion started by: mjavalkar
2 Replies

7. UNIX for Dummies Questions & Answers

Join Lines at a specific point

Hi I'm a beginner, and i've been having trouble joining two lines. I need to convert this file 1097ALABAMA Mobile County METHOMYL INSE CTICIDES 6 1.6200000E+00 1.8000001E+00 1003ALABAMA Baldwin County ... (5 Replies)
Discussion started by: kf_1434
5 Replies

8. Solaris

How to Limit ftp access parameters for specific users?

Dear friends, :) I create new user useradd -g other -d /export/home/sltftp -m -s /bin/bash -c "SLT user account for TMA ftp backup" sltftp now i need do restrict thees chmod delete overwrite rename from this user:(for all the files in the server ,sltftp user can only able to download... (4 Replies)
Discussion started by: darakas
4 Replies

9. Shell Programming and Scripting

Replacing whole string starting with specific works

Hi guys, So what I am trying to accomplish is to replace a whole string starting with some designated string. eg: When even I find a string starting with : eai.endpoint.url= replace the entire line with: eai.endpoint.url=http://www.endpoint.com/API Righ now I am trying to... (4 Replies)
Discussion started by: Junaid Subhani
4 Replies

10. Red Hat

How to join Linux pc to active directory in specific ou with authconfig?

I use authconfig command to join linux computers to AD How do I specify which OU they go in? I llook at authconfig --help but not see antyhing\ Can I use --ldapbasedn=<dn> to select what OU it should go iN (0 Replies)
Discussion started by: red888
0 Replies

LEARN ABOUT DEBIAN

pyntor-selfrun

pyntor-selfrun(1)						      Pyntor							 pyntor-selfrun(1)

NAME

       pyntor-selfrun - creation of self-extracting presentation archives

SYNOPSIS

       pyntor-selfrun [OPTIONS] presentation-archive|presentation-directory

DESCRIPTION

       Pyntor  is a presentation tool which can display slides and effects of various formats.	The tool pyntor-selfrun allows one to create self-
       extracting archives containing both the presentation and Pyntor itself, so that it can be run on  computers  where  Pyntor  itself  is  not
       installed.  It also ensures independence of the corresponding Pyntor version, as future changes of the application do not affect the previ-
       ously created presentations.  The way pyntor-selfrun works is that it takes the presentation-archive  in  question,  packed  as	a  tarball
       (which  might be named *.pyntor), and the release tarball of Pyntor which must be present somewhere. Using a template file, it then creates
       the self-running and self-extracting script for distribution.

       In case the presentation-archive does not exist yet, pyntor-selfrun can create it automatically from a presentation-directory. This is use-
       ful even when not creating self-extracting archives, see the -a option.

OPTIONS

       -t, --template=templatefile
	      Uses  a  template  different  from  that one which is shipped with Pyntor and used by default. The template is a script in Python or
	      another scripting language, which contains the variables %PYNTOR% and %ARCHIVE% which are replaced with the base64-encoded  contents
	      of the two files given as arguments to pyntor-selfrun.  This option is not recommended for most cases.

       -a, --archive
	      Creates a pyntor presentation-archive from a directory which contains a script file, local data files and optionally some local com-
	      ponents.	This is a convenience operation, since presentation archives are just tarballs, but in the future  some  checks  might	be
	      done here.

       -r, --release=sourcetarball
	      Specifies  where	to find Pyntor itself to include it into the self-extracting archive. The source tarball should be a released pyn-
	      tor-*.tar.gz file.

       -h, --help
	      Displays a summary of all available command line options.

BUGS

       In a future version, pyntor-selfrun should allow to create *.pyntor archives automatically by examining a script file, including only those
       files of Pyntor which it really needs.

AUTHORS

       Josef Spillner <josef@coolprojects.org>

SEE ALSO

       pyntor(1), pyntor-components(1)

Cool Projects								0.6							 pyntor-selfrun(1)
All times are GMT -4. The time now is 01:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy