Sponsored Content
Top Forums Shell Programming and Scripting Help Manipulating Large Csv File Post 302450107 by xxwohxx on Wednesday 1st of September 2010 04:26:02 PM
Old 09-01-2010
Quote:
Originally Posted by radoulov
Code:
awk 'END {
  print p, c2, c3, x
  }
NF == 3 && p && p != $1 {
  print p, c2, c3, x
  c2 = c3 = x
  }
{ 
  r2 = $(NF - 1); r3 = $NF
  c2 = c2 ? c2 FS r2 : r2
  c3 = c3 ? c3 FS r3 : r3
  NF == 3 && p = $1 
  }' OFS=\; infile

Very nice, thanks, this seems to work.... the output looks like.

Key1, FNAME1, LNAME1, Key2, FNAME2, LNAME2, FNAME2.1, LNAME2.1, FNAME2.2, LNAME2.2, Key3, FNAME3, LNAME3, FNAME3.1, LNAME3.1..........

How can I add line breaks towards the end of each key so I get this output?

Key1, FNAME1, LNAME1;
Key2, FNAME2, FNAME2.1, FNAME2.2, LNAME2, LNAME2.1, LNAME2.2;
Key3, FNAME3, FNAME3.1, LNAME3, LNAME3.1;

Thanks again!
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

manipulating csv to add extra row

hi how do i manipulate .csv file to add an extra row after each row using shell script? I need a blank line added for each 1000 records in my file? I will then need to copy and paste some data in the blank row created. thanks 4 ur support neil (3 Replies)
Discussion started by: neil546
3 Replies

2. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

3. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

4. Shell Programming and Scripting

Manipulating csv file

We need to convert a field in a csv file which is in cents to dollars.(divide by 100) in our shell script. Can some body help me? (3 Replies)
Discussion started by: Deepthz
3 Replies

5. SCO

Need advice: Copying large CSV report files off SCO system

I have a SCO Unix server from 1999 running SCO 5.0.5 and some ancient accounting software called Real World A report writer program on the system is used to generate CSV files from accounting that we write with DOSCOPY commands to 3.5" floppies In the next 60 days we will be decommissioning... (11 Replies)
Discussion started by: magnetman
11 Replies

6. Shell Programming and Scripting

Merging Very large CSV files in Unix

Hi, I have two very large CSV files, which I want to merge (equi-join) based on a key (column). One of the file (say F1) would have ~30 MM records and 700 columns. The other file (~f2) would have same # of records and lesser columns (say 50). I want to create an output file joining on a... (3 Replies)
Discussion started by: student_007
3 Replies

7. Shell Programming and Scripting

Comparing two large unsorted csv files

Hi All, My requirement is to write a shell script to compare two large csv files. I've created sample files for explaining my problem i.e., a.csv and b.csv contents of files: ----------------- a.csv ------ Type,Memory (Kb),Location HD,Size (Mb),Serial # XT,640,D402,0,MG0010... (2 Replies)
Discussion started by: vasavi
2 Replies

8. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

9. Shell Programming and Scripting

Output large volume of data to CSV file

I have a program that output the ownership and permission on each directory and file on the server to a csv file. I am getting error message when I run the program. The program is not outputting to the csv file. Error: the file access permissions do not allow the specified action cannot... (2 Replies)
Discussion started by: dellanicholson
2 Replies
OCR4GAMERA(1)															     OCR4GAMERA(1)

NAME
ocr4gamera - OCR system using the Gamera framework USAGE
ocr4gamera -x <traindata> [options] <imagefile> OPTIONS
-v <int>, --verbosity=<int> Set verbosity level to <int>. Possible values are 0 (default): silent operation; 1: information on progress; >2: segmentation info is written to PNG files with prefix debug_. -h, --help Display help and exit. -d, --deskew Do a skew correction (recommended). -f, --filter Filter out very large (images) and very small components (noise). -a, --automatic-group Autogroup glyphs with classifier. -x <file>, --xmlfile=<file> Read training data from <file>. -o <xml>, --output=<xml> Write recognized text to file <xml> (otherwise it is written to stdout). -c <csv>, --extra_chars_csvfile=<csv> Read additional class name conversions from file <csv>. <csv> must contain one conversion per line. -R <rules>, --heuristic_rules=<rules> Apply heuristic rules <rules> for disambiguation of some chars. <rules> can be roman (default) or none (for no rules). -D, --dictionary-correction Correct words using a dictionary (requires aspell or ispell). -L <lang>, --dictionary-language=<lang> Use <lang> as language for aspell (when option -D is set). -e <int>, --edit-distance=<int> Correct words only when edit distance not more than <int>. OCR4GAMERA(1)
All times are GMT -4. The time now is 07:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy