Sponsored Content
Top Forums Shell Programming and Scripting Need to strip control-A characters from a column in a file Post 302944512 by wisecracker on Tuesday 19th of May 2015 05:39:54 PM
Old 05-19-2015
Quote:
Originally Posted by harsha1238
Here is a sample. I am using ',' as field delimiter instead of cntl-a.

1,A,USA,0
2,B,GERMANY,0
3,C,IND,IA,0
4,D,CH,INA,0

In the above example, the values "IND,IA" and "CH,INA" are coming from the table.

The files are in .gz format and the sizes are around 12 GB each.
Ignoring the 6th column in the OP you never mentioned that, (in this case), the third column(ish) might or might not have this delimiter.
Is this a random event or is it every pair as in your example?
Can we see your attempt please?
 

10 More Discussions You Might Find Interesting

1. Programming

Identifying and removing control characters in a file.

What is the best method to identify an remove control characters in a file. Would it be easier to do this in Unix or in C. (0 Replies)
Discussion started by: oracle8
0 Replies

2. Shell Programming and Scripting

Hidden control characters in a Unix Text File!

Can anyone seem to know how to find out whether a UNIX text file has 'hidden' control characters? Can I view them using 'vi' by some command line options? If there are control characters in a text file which are invisible/hidden.. then how do I get rid of them? Your intelletual answers are... (6 Replies)
Discussion started by: kewl_guy
6 Replies

3. Shell Programming and Scripting

display all possible control characters from .xml file in unix

Hi, I have a .xml file in unix. We are passing this file through a xml parser. But we are getting some control characters from input file and XML parser is failing for the control character in file.Now I am getting following error, Error at byte 243206625 of file filename_$.xml: Error... (1 Reply)
Discussion started by: fantushmayu
1 Replies

4. Shell Programming and Scripting

Extra control characters being added when I create a file using cat command

Hi, I am using Cygwin.I created a new file and type into it using cat > newfile. When I open this using vi editor, it contains loads of extra control characters. Whats happening? (1 Reply)
Discussion started by: erora
1 Replies

5. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

6. Shell Programming and Scripting

Strip First few Characters

I want to strip first few characters from each record until a proper datesamp is found. Request for getNextPage.................06/29/12 07:49:30 VVUKOVIC@67.208.166.131{7A805FEF76A62FCBB23EA78B5380EF95.tomcat1}TP-Processor14 LogExchUsage: ERROR:: isprof=false : exch=NSDQ output should be... (2 Replies)
Discussion started by: ratheeshjulk
2 Replies

7. Shell Programming and Scripting

Request for advise on how to remove control characters in a UNIX file extracted from top command

Hi, Please excuse for posting new thread on control characters, I am facing some difficulties in removing the control character from a file extracted from top command, i am able to see control characters using more command and in vi mode, through cat control characters are not visible ... (8 Replies)
Discussion started by: karthikram
8 Replies

8. Shell Programming and Scripting

How to view the control characters in a file?

Hello, How can I view control and special characters of a text file?. For example, space, tabs, new line chars etc. Can I use hexdump for it? Thanks (3 Replies)
Discussion started by: reddyr
3 Replies

9. Red Hat

Special control characters in file

Hi Guys, We receive some huge files on to Linux server. Source system use FTP mechanism to transfer these files on our server. Occasionally one record is getting corrupted while transfer, some control characters are injecting into the file. How to fix this issue ? please advice ? Sample... (2 Replies)
Discussion started by: srikanth38
2 Replies

10. Shell Programming and Scripting

How to strip some characters before putting in array?

Hi Gurus, my current code like below: nawk '{f1 = (NF>1)?$1:""}{print f1, $NF}'|sed -e 's/s(/,/g;s/)//g;s/ *,/,/'|nawk -F"," '{ab}END{for (i in b) if (!(i in a))print i}' I have file like below. (this is autosys job dependencies) the job with s() is dependencies, the job without s() is... (10 Replies)
Discussion started by: ken6503
10 Replies
cut(1)								   User Commands							    cut(1)

NAME
cut - cut out selected fields of each line of a file SYNOPSIS
cut -b list [-n] [file]... cut -c list [file]... cut -f list [-d delim] [-s] [file]... DESCRIPTION
Use the cut utility to cut out columns from a table or fields from each line of a file; in data base parlance, it implements the projection of a relation. The fields as specified by list can be fixed length, that is, character positions as on a punched card (-c option) or the length can vary from line to line and be marked with a field delimiter character like TAB (-f option). cut can be used as a filter. Either the -b, -c, or -f option must be specified. Use grep(1) to make horizontal ``cuts'' (by context) through a file, or paste(1) to put files together column-wise (that is, horizontally). To reorder columns in a table, use cut and paste. OPTIONS
The following options are supported: list A comma-separated or blank-character-separated list of integer field numbers (in increasing order), with optional - to indi- cate ranges (for instance, 1,4,7; 1-3,8; -5,10 (short for 1-5,10); or 3- (short for third through last field)). -b list The list following -b specifies byte positions (for instance, -b1-72 would pass the first 72 bytes of each line). When -b and -n are used together, list is adjusted so that no multi-byte character is split. -c list The list following -c specifies character positions (for instance, -c1-72 would pass the first 72 characters of each line). -d delim The character following -d is the field delimiter (-f option only). Default is tab. Space or other characters with special meaning to the shell must be quoted. delim can be a multi-byte character. -f list The list following -f is a list of fields assumed to be separated in the file by a delimiter character (see -d ); for instance, -f1,7 copies the first and seventh field only. Lines with no field delimiters will be passed through intact (useful for table subheadings), unless -s is specified. -n Do not split characters. When -b list and -n are used together, list is adjusted so that no multi-byte character is split. -s Suppresses lines with no delimiter characters in case of -f option. Unless specified, lines with no delimiters will be passed through untouched. OPERANDS
The following operands are supported: file A path name of an input file. If no file operands are specified, or if a file operand is -, the standard input will be used. USAGE
See largefile(5) for the description of the behavior of cut when encountering files greater than or equal to 2 Gbyte (2^31 bytes). EXAMPLES
Example 1 Mapping user IDs A mapping of user IDs to names follows: example% cut -d: -f1,5 /etc/passwd Example 2 Setting current login name To set name to current login name: example$ name=`who am i | cut -f1 -d' '` ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of cut: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 All input files were output successfully. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
grep(1), paste(1), attributes(5), environ(5), largefile(5), standards(5) DIAGNOSTICS
cut: -n may only be used with -b cut: -d may only be used with -f cut: -s may only be used with -f cut: cannot open <file> Either file cannot be read or does not exist. If multiple files are present, processing continues. cut: no delimiter specified Missing delim on -d option. cut: invalid delimiter cut: no list specified Missing list on -b, -c, or -f option. cut: invalid range specifier cut: too many ranges specified cut: range must be increasing cut: invalid character in range cut: internal error processing input cut: invalid multibyte character cut: unable to allocate enough memory SunOS 5.11 29 Apr 1999 cut(1)
All times are GMT -4. The time now is 07:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy