Regex Proofreader

Disclaimer

Please do not email me about this project without reading the disclaimer above.

If you have ever used Funduc Search and Replace or SlickEdit or any program using regular expressions, you know what a beast they can be to proofread. You are often quite amazed by what they do to your files when you turn them loose.

The main problem is quoting. Does < mean the literal character < or is it a command? In Funduc Search and Replace, it is a literal in 0the search argument, but a command in the replace argument. Arrgh! Every implementation of regexes uses a slightly different set of commands and command characters. In Java 1.4, the character \ has when used literally to represent data has to be written as \\\\ because regex and Java string quoting gang up on you.

If a character is reserved as a command, when you want to use it literally, you must precede it with a \. This turns expressions into unreadable nightmares like this:

You might display command characters in red and literal characters in blue, instead of like this:

A slightly cleverer proofreader would let you hide the \ characters (except those preceded by another \ ) and just display the raw colour-coded literals. You could think of this as unquoting. The unquoted regex expressions would then look much more like the actual strings they are intended to match. The expression may then look like this:

What then would you do with tab=\t, cr=\r and newline= \n ? You might cook up some special glyphs to represent them more literally, much the way MS word can be persuaded to make visible the spaces, line and paragraph ends with special symbols, like this:

Everything so far is pretty easy. You don’t need to know anything much about regex expressions to write such a simple proofreader. The only problem is  Java version 1.1 does not support rich text — multiple font colours in a TextArea, though Swing does. Unfortunately you can’t handle the colours by generating HTML (Hypertext Markup Language) <font color= commands and having something render it. You need to handle your colours at a lower level — like a Canvas.

For a more advanced proofreader, things get a little tougher. The user should be able to submit some sample strings and see what the regex would do to them. Do they match the pattern. What do they get converted to. You have to simulate Funduc Search and Replace, SlickEdit or whatever other Regex engine you are proofing for. If you dig about the web, you may find source code to do this for you that you could cannibalise. You have one advantage over the authors of the commercial regex programs. Java 1.4.1 has Perl-like Regex now part of the java.util.regex package. Your parsing and scanning code can be very slow and no one will mind. The user should be able to maintain little libraries of test strings that he can quickly test out. The before and after views should use colour or bold to highlight the changes.

Once you have that working, you can now try something even tougher, generate a sample set of strings to feed to the regex that exercise each part of the regex expression. Some should match, some should not. By examining the results on that test set of strings, the author of the regex expressions should have a pretty good idea of all the things that regex will do when it is turned loose in the real world on files. To do it very well, your generated sample set of strings should exercise every feature of the regex expression. You should generate strings that pass and fail each filtering point in the regex.

If your regex engine is sufficiently fast, you might consider turning your proofreader into a full blown search/replace engine, that can run scripts of commands or accept them one at a time via a GUI (Graphic User Interface). You might also consider writing your own text editor with this as the crown jewel. For blinding speed, generate byte codes to parse the regex expression. This may then be JITed on the fly, which should give Perl-like speeds. However, the easy way is just to use java.util.regex.

If you want to tackle a simplified version of this project, try creating proofreader for Java source code string literals. So that instead of seeing C:\\mydir\\myfile.txt you would see C:\mydir\myfile.txt. Or abc ¶def instead of abc\ndef.

I thought of an easy way to implement the proofreader. You scan the regex left to right classifying each char as ordinary, quoted, command or backslash and surrounding each character or run in a classifying <span class=??> </span> sandwich. Then you feed the decorated regex string to a JEditorPane to display the HTML. Depending on which styles you stick on the front you will see either:

• ordinary in blue and commands in red.
• the \ used to do quoting, hidden, revealed or partially hidden with small letters or pale colours.
• replace strings of command letters with an ellipsis: …
• Optionally show the extra \ you need for Java in yet another colour/font/size.

I think regexes would be much easier to comprehend with these visual character category clues. You could take it further with shades of colour to subliminally reflect even finer categories. Ideally this functionality should be built into the IDE (Integrated Development Environment), perhaps with plug-in.

Have a look at how RegexBuddy highlights regexes to help you proofread. They highlight space, (), commands and literal characters differently.

standard footer

	This page is posted on the web at:	http://mindprod.com/project/regexproofreader.html
	Optional Replicator mirror of mindprod.com on local hard disk J:	J:\mindprod\project\regexproofreader.html
	Please read the feedback from other visitors, or send your own feedback about the site. Contact Roedy. Please feel free to link to this page without explicit permission.
	Canadian Mind Products IP:[65.110.21.43] Your face IP:[18.119.19.251]
Feedback	You are visitor number