Sponsored Content
Top Forums Shell Programming and Scripting sed and awk not working on a large record file Post 302762959 by Scrutinizer on Tuesday 29th of January 2013 09:47:09 AM
Old 01-29-2013
That file is not a valid Unix text file. You could try if this awk works anyway:
Code:
awk 'NR>1{print p}{p=$0} END{printf "%s\n", p}' RS=\| ORS=\| file

If it works, it should add a linefeed at the end..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Sed working on lines of small length and not large length

Hi , I have a peculiar case, where my sed command is working on a file which contains lines of small length. sed "s/XYZ:1/XYZ:3/g" abc.txt > xyz.txt when abc.txt contains lines of small length(currently around 80 chars) , this sed command is working fine. when abc.txt contains lines of... (3 Replies)
Discussion started by: thanuman
3 Replies

2. UNIX for Advanced & Expert Users

fatal: record too large

while running a script to process a file , it ended in termination . I able to find from the logs that it ended due to the error : fatal: record too large 20480 . Then I tried lots to fix the issue and left in vain . Can any one help me in finding reason for it ? . If you provide me a solution , I... (1 Reply)
Discussion started by: sakthifire
1 Replies

3. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

4. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

5. Shell Programming and Scripting

how to delete \n in a large file with sed

Hello i have a big file with a specific format and delimiter is "§" : §field1§$field2§$field3§$field4§$field5§$field6§$field§ in this file we have a field which are very long (more than 20000 chars !!!!) so through vi i cant manipulate them. despite this i managed to suppress lines that... (11 Replies)
Discussion started by: ade05fr
11 Replies

6. Shell Programming and Scripting

How to delete 1 record in large file!

Hi All, I'm a newbie here, I'm just wondering on how to delete a single record in a large file in unix. ex. file1.txt is 1000 records nikki1 nikki2 nikki3 what i want to do is delete the nikki2 record in file1.txt. is it possible? Please advise, Thanks, (3 Replies)
Discussion started by: nikki1200
3 Replies

7. Shell Programming and Scripting

SED/AWK to edit/add field values in a record

Hi Experts, I am new to shell scripting. Need some help in doing one task given by the customer. The sample record in a file is as follows: 3538,,,,,,ID,ID1,,,,,,,,,,, It needs to be the following: 3538,,353800,353800,,,ID,ID1,,,,,COLX,,,,,COLY, And i want to modify this record in... (3 Replies)
Discussion started by: sugarcane
3 Replies

8. Shell Programming and Scripting

Split a large file in n records and skip a particular record

Hello All, I have a large file, more than 50,000 lines, and I want to split it in even 5000 records. Which I can do using sed '1d;$d;' <filename> | awk 'NR%5000==1{x="F"++i;}{print > x}'Now I need to add one more condition that is not to break the file at 5000th record if the 5000th record... (20 Replies)
Discussion started by: ibmtech
20 Replies

9. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

10. UNIX for Advanced & Expert Users

How to split large file with different record delimiter?

Hi, I have received a file which is 20 GB. We would like to split the file into 4 equal parts and process it to avoid memory issues. If the record delimiter is unix new line, I could use split command either with option l or b. The problem is that the line terminator is |##| How to use... (5 Replies)
Discussion started by: Ravi.K
5 Replies
Gedcom::Record(3pm)					User Contributed Perl Documentation				       Gedcom::Record(3pm)

NAME
Gedcom::Record - a module to manipulate Gedcom records Version 1.16 - 24th April 2009 SYNOPSIS
use Gedcom::Record; my $record = tag_record("CHIL", 2); my @records = tag_record("CHIL"); my @recs = $record->record("birth"); my @recs = $record->record("birth", "date"); my $rec = $record->record("birth date"); my $rec = $record->record(["birth", 2], "date"); my @recs = $record->get_record("birth"); my $val = $record->get_value; my @vals = $record->get_value("date"); my @vals = $record->get_value("birth", "date"); my $val = $record->get_value("birth date"); my $val = $record->get_value(["birth", 2], "date"); my $rec = $record->add("birth date", "1 Jan 2000"); my $rec = $record->set("birth date", "2 Jan 2000"); $self->parse($record, $grammar); $record->collect_xrefs($callback); my $xref = $record->resolve_xref($record->{value}); my @famc = $record->resolve $record->get_value("FAMC"); $record->resolve_xrefs($callback); $record->unresolve_xrefs($callback); return 0 unless $record->validate_semantics; $record->normalise_dates($format); $record->renumber($args); print $record->summary, " "; $record->delete_record($sub_record); DESCRIPTION
A selection of subroutines to handle records in a gedcom file. Derived from Gedcom::Item. HASH MEMBERS
Some of the more important hash members are: $record->{new_xref} Used by renumber(). $record->{recursed} Used by renumber(). METHODS
tag_record my $record = tag_record("CHIL", 2); my @records = tag_record("CHIL"); Get specific sub-records from the record. This function is identical to Gedcom::Item::get_item(). The arguments are the name of the tag, and optionally the count. In scalar context, returns the sub-record, or undef if it doesn't exist. In array context, returns all sub-records matching the specified tag. record my @recs = $record->record("birth"); my @recs = $record->record("birth", "date"); my $rec = $record->record("birth date"); my $rec = $record->record(["birth", 2], "date"); my @recs = $record->get_record("birth"); Retrieve a record. The get_record() function is identical to the record() function. In scalar context, record() returns the specified record, or undef if there is none. In list context, record() returns all the specified records. Records may be specified by a list of strings. Each string is either a Gedcom tag or a description. Starting from the first string in the list, specified records are retrieved. Then from those records, records specified by the next string in the list are retrieved. This continues until all strings from the list have been used. In list context, all specified records are retrieved. In scalar context, only the first record is retrieved. If a record other than the first is wanted, then instead of passing a string, a reference to an array containing the string and a count may be passed. Instead of specifying a list of strings, it is possible to specify a single space separated string. This can make the interface nicer. get_value my $val = $record->get_value; my @vals = $record->get_value("date"); my @vals = $record->get_value("birth", "date"); my $val = $record->get_value("birth date"); my $val = $record->get_value(["birth", 2], "date"); Retrieve a record's value. If arguments are specified, record() is first called with those arguments, and the values of those records are returned. add my $rec = $record->add("birth date", "1 Jan 2000"); Add a new record. Add a new record ($rec) as a sub-item of $record. Set its value to the last argument given. The first arguments may be specified as for record(). A new record will always be created for the last argument, and for any arguments for which the count is explicitly set to zero. If the new record does not take a value then do not supply one. This does mean that you cannot use the function with many arguments if the last one is a scalar, but not a value. In this case either specify the last argument as ["arg", 0], or add undef as the last argument. set my $rec = $record->set("birth date", "2 Jan 2000"); Set the value of a record. This is the same as add(), with the exception that a new record is not created for the last argument. parse $self->parse($record, $grammar); Parse a Gedcom record. Match a Gedcom::Record against a Gedcom::Grammar. Warn of any mismatches, and associate the Gedcom::Grammar with the Gedcom::Record as $record->{grammar}. Do this recursively. collect_xrefs $record->collect_xrefs($callback); Recursively collect all the xrefs. Called by Gedcom::collect_xrefs. $callback is not used yet. resolve_xref my $xref = $record->resolve_xref($value); See Gedcom::resolve_xrefs() resolve my @famc = $record->resolve $record->tag_value("FAMC"); For each argument, either return it or, if it an xref, return the referenced record. resolve_xrefs $record->resolve_xrefs($callback); See Gedcom::resolve_xrefs() unresolve_xrefs $record->unresolve_xrefs($callback); See Gedcom::unresolve_xrefs() validate_semantics return 0 unless $record->validate_semantics; Validate the semantics of the Gedcom::Record. This performs a number of consistency checks, but could do even more. Returns true iff the Record is valid. normalise_dates $record->normalise_dates($format); Change the format of all dates in the record. See the documentation for Gedcom::normalise_dates renumber $record->renumber($args); Renumber the record. See Gedcom::renumber(). child_value NOTE - This function is deprecated - use tag_value instead. my $child = $record->child_value("NAME"); child_values NOTE - This function is deprecated - use tag_value instead. my @children = $record->child_values("CHIL"); summary print $record->summary, " "; Return a line of text summarising the record. delete_record $record->delete_record($sub_record); Delete the specified sub-record from the record. Access functions All the Gedcom tag names can be used as function names. Depending on the context in which they are called, the functions return either an array of the specified sub-items, or the first specified sub-item. The descriptions of the tags, with spaces replaced by underscores, can also be used as function names. The function names can be of either, or mixed case. Unless you use the tag name, in either case, or the description in lower case, the function will not be pre- declared and you will need to qualify it or "use subs". perl v5.14.2 2012-04-12 Gedcom::Record(3pm)
All times are GMT -4. The time now is 09:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy