Sponsored Content
Top Forums Shell Programming and Scripting Split a large file in n records and skip a particular record Post 302877283 by Corona688 on Friday 29th of November 2013 12:21:06 PM
Old 11-29-2013
For starters you need to set the original value of 'f' so you won't be printing into a blank filename.

% can cause some precedence problems in C I know, I would bracket your expressions more carefully.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split A Large File

Hi, I have a large file(csv format) that I need to split into 2 files. The file looks something like Original_file.txt first name, family name, address a, b, c, d, e, f, and so on for over 100,00 lines I need to create two files from this one file. The condition is i need to ensure... (4 Replies)
Discussion started by: nbvcxzdz
4 Replies

2. Shell Programming and Scripting

Split Large File

HI, i've to split a large file which inputs seems like : Input file name_file.txt 00001|AAAA|MAIL|DATEOFBIRTHT|....... 00001|AAAA|MAIL|DATEOFBIRTHT|....... 00002|BBBB|MAIL|DATEOFBIRTHT|....... 00002|BBBB|MAIL|DATEOFBIRTHT|....... 00003|CCCC|MAIL|DATEOFBIRTHT|.......... (1 Reply)
Discussion started by: AMARA
1 Replies

3. Shell Programming and Scripting

Split a large file

I have a 3 GB text file that I would like to split. How can I do this? It's a giant comma-separated list of numbers. I would like to make it into about 20 files of ~100 MB each, with a custom header and footer. The file can only be split on commas, but they're plentiful. Something like... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

4. Shell Programming and Scripting

Split a single record to multiple records & add folder name to each line

Hi Gurus, I need to cut single record in the file(asdf) to multile records based on the number of bytes..(44 characters). So every record will have 44 characters. All the records should be in the same file..to each of these lines I need to add the folder(<date>) name. I have a dir. in which... (20 Replies)
Discussion started by: ram2581
20 Replies

5. Shell Programming and Scripting

How to delete 1 record in large file!

Hi All, I'm a newbie here, I'm just wondering on how to delete a single record in a large file in unix. ex. file1.txt is 1000 records nikki1 nikki2 nikki3 what i want to do is delete the nikki2 record in file1.txt. is it possible? Please advise, Thanks, (3 Replies)
Discussion started by: nikki1200
3 Replies

6. UNIX for Dummies Questions & Answers

Split single record to multiple records

Hi Friends, source .... col1,col2,col3 a,b,1;2;3 here colom delimeter is comma(,). here we dont know what is the max length of col3 means now we have 1;2;3 next time i will receive 1;2;3;4;5;etc... required output .............. col1,col2,col3 a,b,1 a,b,2 a,b,3 please give me... (5 Replies)
Discussion started by: bab.galary
5 Replies

7. UNIX for Dummies Questions & Answers

Using awk to skip record in file

I need to amend the code blow such that it reads a "black list" before the "print" statement; if "substr($1,1,6)" is found in the "blacklist" it will ignore that record and continue. the code is from an awk script that is being called from shell script which passes the input values. BEGIN { "date... (5 Replies)
Discussion started by: bazel
5 Replies

8. Shell Programming and Scripting

How to split one record to multiple records?

Hi, I have one tab delimited file which is having multiple store_ids in first column seprated by pipe.I want to split the file on the basis of store_id(separating 1st record in to 2 records ). I tried some more options like below with using split,awk etc ,But not able to get proper output. can... (1 Reply)
Discussion started by: jaggy
1 Replies

9. UNIX for Advanced & Expert Users

How to split large file with different record delimiter?

Hi, I have received a file which is 20 GB. We would like to split the file into 4 equal parts and process it to avoid memory issues. If the record delimiter is unix new line, I could use split command either with option l or b. The problem is that the line terminator is |##| How to use... (5 Replies)
Discussion started by: Ravi.K
5 Replies

10. UNIX for Beginners Questions & Answers

Trying To Split a Large File

Trying to split a 35gb file into 1000mb parts. My research shows I should you this. split -b 1000m file.txt and my return is "split: cannot open 'crunch1.txt' for reading: No such file or directory" so I tried split -b 1000m Documents/Wordlists/file.txt and I get nothing other than the curser just... (3 Replies)
Discussion started by: sub terra
3 Replies
GLOB(7) 						     Linux Programmer's Manual							   GLOB(7)

NAME
glob - Globbing pathnames DESCRIPTION
Long ago, in Unix V6, there was a program /etc/glob that would expand wildcard patterns. Soon afterwards this became a shell built-in. These days there is also a library routine glob(3) that will perform this function for a user program. The rules are as follows (POSIX 1003.2, 3.13). WILDCARD MATCHING
A string is a wildcard pattern if it contains one of the characters `?', `*' or `['. Globbing is the operation that expands a wildcard pat- tern into the list of pathnames matching the pattern. Matching is defined by: A `?' (not between brackets) matches any single character. A `*' (not between brackets) matches any string, including the empty string. Character classes An expression `[...]' where the first character after the leading `[' is not an `!' matches a single character, namely any of the charac- ters enclosed by the brackets. The string enclosed by the brackets cannot be empty; therefore `]' can be allowed between the brackets, provided that it is the first character. (Thus, `[][!]' matches the three characters `[', `]' and `!'.) Ranges There is one special convention: two characters separated by `-' denote a range. (Thus, `[A-Fa-f0-9]' is equivalent to `[ABCDE- Fabcdef0123456789]'.) One may include `-' in its literal meaning by making it the first or last character between the brackets. (Thus, `[]-]' matches just the two characters `]' and `-', and `[--/]' matches the three characters `-', `.', `/'.) Complementation An expression `[!...]' matches a single character, namely any character that is not matched by the expression obtained by removing the first `!' from it. (Thus, `[!]a-]' matches any single character except `]', `a' and `-'.) One can remove the special meaning of `?', `*' and `[' by preceding them by a backslash, or, in case this is part of a shell command line, enclosing them in quotes. Between brackets these characters stand for themselves. Thus, `[[?*]' matches the four characters `[', `?', `*' and `'. PATHNAMES
Globbing is applied on each of the components of a pathname separately. A `/' in a pathname cannot be matched by a `?' or `*' wildcard, or by a range like `[.-0]'. A range cannot contain an explicit `/' character; this would lead to a syntax error. If a filename starts with a `.', this character must be matched explicitly. (Thus, `rm *' will not remove .profile, and `tar c *' will not archive all your files; `tar c .' is better.) EMPTY LISTS
The nice and simple rule given above: `expand a wildcard pattern into the list of matching pathnames' was the original Unix definition. It allowed one to have patterns that expand into an empty list, as in xv -wait 0 *.gif *.jpg where perhaps no *.gif files are present (and this is not an error). However, POSIX requires that a wildcard pattern is left unchanged when it is syntactically incorrect, or the list of matching pathnames is empty. With bash one can force the classical behaviour by setting allow_null_glob_expansion=true. (Similar problems occur elsewhere. E.g., where old scripts have rm `find . -name "*~"` new scripts require rm -f nosuchfile `find . -name "*~"` to avoid error messages from rm called with an empty argument list.) NOTES
Regular expressions Note that wildcard patterns are not regular expressions, although they are a bit similar. First of all, they match filenames, rather than text, and secondly, the conventions are not the same: e.g., in a regular expression `*' means zero or more copies of the preceding thing. Now that regular expressions have bracket expressions where the negation is indicated by a `^', POSIX has declared the effect of a wildcard pattern `[^...]' to be undefined. Character classes and Internationalization Of course ranges were originally meant to be ASCII ranges, so that `[ -%]' stands for `[ !"#$%]' and `[a-z]' stands for "any lowercase let- ter". Some Unix implementations generalized this so that a range X-Y stands for the set of characters with code between the codes for X and for Y. However, this requires the user to know the character coding in use on the local system, and moreover, is not convenient if the collating sequence for the local alphabet differs from the ordering of the character codes. Therefore, POSIX extended the bracket notation greatly, both for wildcard patterns and for regular expressions. In the above we saw three types of item that can occur in a bracket expression: namely (i) the negation, (ii) explicit single characters, and (iii) ranges. POSIX specifies ranges in an internationally more useful way and adds three more types: (iii) Ranges X-Y comprise all characters that fall between X and Y (inclusive) in the currect collating sequence as defined by the LC_COL- LATE category in the current locale. (iv) Named character classes, like [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:] so that one can say `[[:lower:]]' instead of `[a-z]', and have things work in Denmark, too, where there are three letters past `z' in the alphabet. These character classes are defined by the LC_CTYPE category in the current locale. (v) Collating symbols, like `[.ch.]' or `[.a-acute.]', where the string between `[.' and `.]' is a collating element defined for the cur- rent locale. Note that this may be a multi-character element. (vi) Equivalence class expressions, like `[=a=]', where the string between `[=' and `=]' is any collating element from its equivalence class, as defined for the current locale. For example, `[[=a=]]' might be equivalent to `[aaaaa]' (warning: Latin-1 here), that is, to `[a[.a-acute.][.a-grave.][.a-umlaut.][.a-circumflex.]]'. SEE ALSO
sh(1), glob(3), fnmatch(3), locale(7), regex(7) Unix 1998-06-12 GLOB(7)
All times are GMT -4. The time now is 09:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy