Sponsored Content
Top Forums Shell Programming and Scripting Delimited data contains line feeds where they shouldn't be Post 302508077 by ericdp63 on Friday 25th of March 2011 03:52:54 PM
Old 03-25-2011
getting closer :)

Thanks Kato, that works just fine. Let me see if I can dissemble this.

NF checks to see that there are less then four fields(columns).
If so, save that line and go read the next line
Else print the previous line and the current line together. and go on to the next line

what does the '1' at the end do? if I leave it out, just the two line are printed. is it a flag that just has to be there and doesn't matter what its value is?

so to make it more robust as a script I can do something like:

cat clean.awk
Code:
BEGIN {

  USAGE = "usage: awk -f clean.awk -v col_cnt=<#> <input file>"
  if ( length( col_cnt ) == 0 ) {

    print "col_cnt not defined" > "/dev/stderr"
    print USAGE > "/dev/stderr"
    exit 1

  }
  FS = "|"

} # BEGIN

NF < int( col_cnt ) {

  if ( short == "" ) {

    short = $0
    next

  } else { 

    print short$0
    short = ""
    next

  }
} 1

running it:

Code:
awk -f clean.awk -v col_cnt=4 < data.txt


I get the following mess. Changing it from a one-liner to a script somehow changed it.

Code:
2|short desc| long
1|short desc|long text|2001-01-01 01:012|short desc| long
  text |2002-02-02 02:02
  text |2002-02-02 02:023|short desc|  long  text  | 2003-03-03 03:03
4|short desc
  |  long  text    | 2004-04-04 04:04
4|short desc  |  long  text    | 2004-04-04 04:04

Getting closer. The one-liner works, but I would like to document this as a script for a later programmer to maintain.

Thanks much
Eric
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

carriage return/line feeds

Hello, I have a file that has got carriage returns in it and I want to take them out. Anyone know how I can do this in a ksh? thanks (4 Replies)
Discussion started by: pitstop
4 Replies

2. Shell Programming and Scripting

Remove line feeds

Hi, I have a fixed width flat file which has 1 as the first char and E as the last character. Some of the records have a carriage return /line feeds . how do I remove them? Let me know. Thanks VSK (8 Replies)
Discussion started by: vsk
8 Replies

3. Shell Programming and Scripting

line feeds in csv

:confused: hi all, i have csv file with three comma separated columns i/p file First_Name, Address, Last_Name XXX, "456 New albany \n newyork, Unitedstates \n 45322-33", YYY\n ZZZ, "654 rifle park \n toronto, canada \n 43L-w3b", RRR\n is there any way i can remove \n (newline) from... (10 Replies)
Discussion started by: gowrish
10 Replies

4. Shell Programming and Scripting

Spurious line feeds

Hi all, I know this is **awfully** general but..... I have a script which does, basically... for file in `find command`; do some stuff more stuff echo '.\c' done I want to output the '.' char just to give an idea of progress. However, it works fine for a while and then I... (2 Replies)
Discussion started by: ajcannon
2 Replies

5. Shell Programming and Scripting

supressing carrige returns/line feeds

Hi gurus I am stripping lots of email addresses from a file with this grep "^To" file.log |awk '{print "1,"$2}' > recipients.out file.log looks something like this: oasndfoasnosf To: person@email.co.uk lsdfjosd sdlfnmsopdfwer dtlghodrgn To: person2@emailsss.com sldfnsdf I... (5 Replies)
Discussion started by: terry2009
5 Replies

6. UNIX for Dummies Questions & Answers

.properties file and new line feeds

Hi, I have a .properties file that a read in some values in an .sh file but everytime I put it out on the server it fails. If I copy and paste the values of the .properties file on my local machine to the .properties file on the server it works just fine. Someone mentioned to see if it has dos... (3 Replies)
Discussion started by: vsekvsek
3 Replies

7. Shell Programming and Scripting

remove line feeds followed by character

Hi everyone, I'm very new to using sed, run through some tutorials and everything but I've hit a problem that I'm unable to solve by myself. I need to remove all linefeeds that are followed by a particular character (in this case a semicolon). So basically, all lines starting with a semicolon... (5 Replies)
Discussion started by: fluffdasheep
5 Replies

8. Shell Programming and Scripting

useless line feeds in ldapsearch output. Howto remove with shell script?

Hi $ cat ad.sh ldapsearorg -x -LLL -h sb1131z.testbadbigcorp.org -D "CN=ADMINZZ,OU=AdminRoles,DC=testbadbigcorp,DC=org" -w "UT3w4f57lll--4...4" -b "OU=Test,DC=testbadbigcorp,DC=org" "(&(&(&(&(objectCategory=person)(objectClass=user)(lockoutTime:1.2.840.113556.1.4.804:=4294967295)))))" dn$... (3 Replies)
Discussion started by: slashdotweenie
3 Replies

9. Shell Programming and Scripting

Removing carriage return/line feeds on multiple lines

I would like to remove carriage returns/line feeds in a text file, but in a specific cadence: Read first line (Header Line 1), remove cr/lf at the end (replace it with a space ideally); Read the next line (Line of Text 2), leave the cr/lf intact; Read the next line, remove the cr/lf; Read... (14 Replies)
Discussion started by: tomr2012
14 Replies

10. Shell Programming and Scripting

How to remove new line characters from data rows in a Pipe delimited file?

I have a file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 2345|FirstName2|MiddleName3|LastName4| Add1 || ADD2| 234|000000000 OUTPUT : ... (1 Reply)
Discussion started by: styris
1 Replies
GCONFTOOL-2(1)							   User Commands						    GCONFTOOL-2(1)

NAME
gconftool-2 - GNOME configuration tool SYNOPSIS
gconftool-2 [-?] [-?|--help] [--usage] [-s|--set] [-g|--get] [--set-schema] [-u|--unset] [--recursive-unset] [-a|--all-entries] [--all-dirs] [--dump] [--load=STRING] [-R|--recursive-list] [--dir-exists=STRING] [--shutdown] [-p|--ping] [--spawn] [-t|--type int|bool|float|string|list|pair] [-T|--get-type] [--get-list-size] [--get-list-element] [--list-type=int|bool|float|string] [--car-type=int|bool|float|string] [--cdr-type=int|bool|float|string] [--short-desc=DESCRIPTION] [--long-desc=DESCRIPTION] [--owner=OWNER] [--install-schema-file=FILENAME] [--config-source=SOURCE] [--direct] [--makefile-install-rule] [--makefile-uninstall-rule] [--break-key] [--break-directory] [--short-docs] [--long-docs] [--get-schema-name] [--apply-schema] [--unapply-schema] [--get-default-source] [-v|--ver- sion] DESCRIPTION
Each preference in the GConf repository is expressed as a key-value pair. A GConf preference key is an element in the GConf repository that corresponds to an application preference. Preference keys typically have simple values such as strings, integers, or lists of strings and integers. The default key-value pairs are installed on the system, and are known as schemas. gconftool-2 is the command-line interface tool that enables you to set the values of keys, display the values of keys, and install schemas from schema definition files when you install an application. The GConf preference keys are stored and managed by the GConf daemon (gconfd-2). These keys are cached in memory, and saved to disk in XML format when appropriate. OPTIONS
-s, --set Set a key to a value and sync. Use with --type. -g, --get Print the value of a key to standard output. -u, --unset Unset the keys on the command line. --recursive-unset Recursively unset all keys at or below the key/directory names on the command line. -a, --all-entries Print all key/value pairs in a directory. --all-dirs Print all subdirectories in a directory. -R, --recursive-list Print all subdirectories and entries under a dir, recursively. --dir-exists=STRING Return 0 if the directory exists, 2 if it does not. -t, --type=int|bool|float|string|list|pair Specify the type of the value being set, or the type of the value a schema describes. Unique abbreviations are allowed. --list-type=int|bool|float|string Specify the type of the list value being set, or the type of the value a schema describes. Unique abbreviations are allowed. --car-type=int|bool|float|string Specify the type of the car pair value being set, or the type of the value a schema describes. Unique abbreviations are allowed. --cdr-type=int|bool|float|string Specify the type of the cdr pair value being set, or the type of the value a schema describes. Unique abbreviations are allowed. --short-desc=DESCRIPTION Specify a short half-line description to go in a schema. --long-desc=DESCRIPTION Specify a several-line description to go in a schema. --owner=OWNER Specify the owner of a schema. --install-schema-file=FILENAME Specify a schema file to be installed --config-source=SOURCE Specify a configuration source to use rather than the default path --direct Access the config database directly, bypassing server. Requires that gconfd is not running. --set-schema Set a schema and sync. Use with --short-desc, --long-desc, --owner, and --type. --makefile-install-rule Properly installs schema files on the command line into the database. GCONF_CONFIG_SOURCE environment variable should be set to a non-default config source or set to the empty string to use the default. --break-key Torture-test an application by setting and unsetting a bunch of values of different types for keys on the command line. --break-directory Torture-test an application by setting and unsetting a bunch of keys inside the directories on the command line. --short-docs Get the short doc string for a key --long-docs Get the long doc string for a key --get-schema-name Get the name of the schema applied to this key --apply-schema Specify the schema name followed by the key to apply the schema name to --unapply-schema Remove any schema name applied to the given keys --get-default-source Get the name of the default source. --shutdown Shut down gconfd. Don't use this option without good reason. -p, --ping Return 0 if gconfd is running, 2 if not. --spawn Launch the config server (gconfd). (Normally happens automatically when needed.) -v, --version Print version Help options -?, --help Show the help message. EXAMPLES
Display the path to the current desktop wallpaper. gconftool-2 --get /desktop/gnome/background/picture_filename Change the widget font to "Serif 12" gconftool-2 --type string --set /desktop/gnome/interface/font_name "Serif 12" AUTHOR
GConf was written by Havoc Pennington <hp@redhat.com>. SEE ALSO
The GConf web site, http://www.gnome.org/projects/gconf/. gconftool-2 2.2.1 February 2006 GCONFTOOL-2(1)
All times are GMT -4. The time now is 01:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy