Help to identify blank space in a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help to identify blank space in a file
# 1  
Old 04-19-2018
Help to identify blank space in a file

Hello,
I have a dictionary of over 400,000 words with the following structure
Code:
source=target

The database contains single words as well as phrases. To train the data, I need only mappings with out a space i.e. where both source and target do not have any space in between.
I use Ultraedit as my editor and have been using the following regex in Unix to identify a blank space
Code:
^[^ ]+$

Since the database is too large, the editor runs out of memory and cannot store all instances to the clipboard.
Am giving below a small sample text
Code:
هيراآباد=हीरा आबाद
واسڪوڊيگاما=वास्कोडीगामा
کانسواءِ=खांसवाइ/खा सिवाइ
آوازنکي=आवाज़नखे
سانآهي=सान आहे
سڏبوآهي=सॾबो आहे
شڪارڪرڻ=शकार करण
ٺاهيندوآهي=ठाहींदो आहे
ٻولينجو=ॿोलीनजो
ٻولينجي=ॿोलीनजे
ڪنديآهي=कंदी आहे
گئسنجو=गैसन जो
ماموغلام=मामूग़ुलाम
زاهدچانڊيو=ज़ाहिद चांडियो
عطرڪمار=अतुरकुमार
غلاممحي=ग़ुलाममही
گلشيرڪوريجو=गुलशेर कोरीजो
زيرحراست=ज़ीर हिरासत

The script should identify only those entries without a space on either side and store them in a separate file, as in the sample output below
Code:
واسڪوڊيگاما=वास्कोडीगामा
آوازنکي=आवाज़नखे
ٻولينجو=ॿोलीनजो
ٻولينجي=ॿोलीनजे
ماموغلام=मामूग़ुलाम
عطرڪمار=अतुरकुमार
غلاممحي=ग़ुलाममही

A perl or awk script would help. I work in a windows environment.
Many thanks.
# 2  
Old 04-19-2018
Have a look at the -v option of grep:
Code:
grep -vF ' ' your_file

# 3  
Old 04-19-2018
Quote:
Originally Posted by rovf
Have a look at the -v option of grep:
Code:
grep -vF ' ' your_file

True. Still, as a measure of safety i would rule out trailing or leading spaces:

Code:
sed -n '/^[[:blank]]*//;s/[[:blank:]]*$//;/ /!p' > /result/file

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 4  
Old 04-19-2018
Quote:
Originally Posted by bakunin
True. Still, as a measure of safety i would rule out trailing or leading spaces:
For instance using grep:

Code:
grep -v '[^ ] [^ ]' your_file

This User Gave Thanks to rovf For This Post:
# 5  
Old 04-19-2018
Many thanks for all your kind help. My broadband connectivity was down all day and hence the delay. All the solutions worked. I had ensured that my data had no trailing spaces so the issue of trailing spaces does not arise but is is nice to have a solution which ensures that trailing spaces are handled.
Thanks once again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to identify exact text and then add a blank line above it using sed?

I need to identify the exact text of San Antonio Generator Running in the output my script which lands to a text file. Once SED finds the specific text, I need it to insert one line above the matched text. Here is what I have so far that isn't working all that well for me. Any help would be... (7 Replies)
Discussion started by: jbrass
7 Replies

2. Shell Programming and Scripting

How to check if the file is empty or has blank space.?

Hi, I am using KSH. I am trying to check if the output file is empty or not. I tried with ] but what i see is my file is empty but still manages to have a size of 1 instead of 0. But my file doesnot have anything its empty. I am not sure how to check this. can any one help? (10 Replies)
Discussion started by: Sharma331
10 Replies

3. Shell Programming and Scripting

Remove Space and blank line from file in UNIX shell script

I have below file. I want to remove space at begining of every line and then after also remove blank line from file. I use below code for each operation. sed -e 's/^*//' < check.txt > check1.txt sed '/^\s*$/d' < check1.txt > check2.txt above code not remove all the space... (12 Replies)
Discussion started by: Mohin Jain
12 Replies

4. Shell Programming and Scripting

Not delete space blank

Hi everyone, i need to "grep" a file with a string with space blanks, like this: grep "XXXX XX" file.txt The problem, i need put the "XXXX XX" in a string variable. When the script executes the grep, do: gresp XXXX XX file.txt How can i solve this problem? The... (5 Replies)
Discussion started by: Xedrox
5 Replies

5. Shell Programming and Scripting

Removing blank space in file

TT0000013101257 | JCJMMUJMMUB018 ... (0 Replies)
Discussion started by: sususa
0 Replies

6. UNIX for Dummies Questions & Answers

blank space

hi everyone, i have a problem in unix script , i need to remove line that has blank , not blank line . example: mahm,,jdggkhsd,ghskj,,fshjkl can anyone help? (4 Replies)
Discussion started by: Reham.Donia
4 Replies

7. Shell Programming and Scripting

Cut last blank space

Hello, I am using this to get only directories : ls -l | grep '^d'and here is the result : drwx------ 13 so_nic sonic 13 Nov 4 13:03 GLARY drwx------ 3 so_nic sonic 3 May 6 2010 PSY2R drwx------ 15 so_nic sonic 15 Oct 14 08:47 PSYR1 But I only need to keep this... (7 Replies)
Discussion started by: Aswex
7 Replies

8. Shell Programming and Scripting

Removing blank lines from comma seperated and space seperated file.

Hi, I want to remove empty/blank lines from comma seperated and space seperated files Thanks all for help (11 Replies)
Discussion started by: pinnacle
11 Replies

9. Shell Programming and Scripting

Blank Space is not appending in each row of CSV File - Shell Script

I am calling SQL script in my UNIX Shell script and trying to create the CSV file and my last column value of each row is 23 blank spaces. In my SQL script,the last column is like below. RPAD(' ',23,' ') -- Padding 23 blank Spaces The CSV file is generated but the sapce(23 spaces) is... (2 Replies)
Discussion started by: praka
2 Replies

10. Shell Programming and Scripting

append blank space

Hi, I would like to add blank space for fixed length(50) if length of string <30. Scenario: File Size AAA.CSV 123 BB.CSV 134 Expected: File Size AAA.CSV 123 BB.CSV 134 I want append blank space until 30 character. Thanks and Regards, HAA (1 Reply)
Discussion started by: HAA
1 Replies
Login or Register to Ask a Question