This document describes the design of a spelling server based on Wintertree Software's Sentry Spelling Checker Engine.
This document assumes familiarity with the concept and implementation of servers, client-server protocol, and concurrent request processing. It focuses on how the Sentry Spelling Checker Engine could be used to implement a server capable of processing spelling requests received from multiple clients. Issues common to all servers, such as communication protocols, client management, and security are not addressed.
A spelling server is typically used in a application such as Web-based e-mail or forms entry. The purpose of the spelling server is to check the spelling of text submitted by clients, and report any misspelled words detected and offer suggested alternative spellings for misspelled words. There is usually one server (conceptually, at least) and many clients. The figure below shows the configuration of a spelling server application:
The spelling server application may be part of a Web server, or it might be a separate process running on a server computer.
The client application is typically a Web page containing forms, plus script code (e.g., VBScript or JavaScript) or Java applets. The client software is used to compose the text to be spell-checked, and to communicate with the spelling server.
The client software communicates with the spelling server by sending a request. The spelling server processes the request and returns a reply.
Many client computers may issue requests to the same spelling server at the same time. The server processes requests in parallel by creating a new thread or process when a request is received.
An important goal in the design is that the protocol between client and server be connectionless. The client can issue any request to the server at any time without having to initiate or terminate a conversation. The benefit of this goal is that the server does not have to keep track of its clients. The server simply processes each request it receives and returns a reply. When a client no longer needs to use the server, it stops sending requests.
The design presented here makes the following assumptions:
The text being checked may have unlimited size
The user will interactively check the spelling of some text entered in a form (for example, an e-mail message). Each spelling error will be presented to the user, who will dispose of the problem by ignoring it or correcting it.
The bandwidth of the connection between client and server may be low (e.g., Internet connection through a modem).
The user enters some text in a form and requests the client software to check the spelling of the text. When a misspelled word is detected, the client software presents the problem to the user in some way. The problem display typically includes:
The misspelled word. If possible, the misspelled word is shown in context (i.e., with the text surrounding it).
A description of the problem (misspelled word, capitalization error, doubled word, conditional replacement)
A list of suggested replacements if the word is misspelled
The alternate word (for capitalization and conditional replacements)
The client software presents the user with a set of actions he or she can use to dispose of the problem:
Ignore (skip) this occurrence of the misspelled word, doubled word, or conditional replacement (Ignore)
Ignore (skip) this and all further occurrences of the misspelled word (Ignore All)
Change this occurrence of the misspelled word or conditional replacement with another word (Change) (also used to delete a doubled word by replacing it with an empty word)
Change this and all further occurrences of the misspelled word with another word (Change All)
(Optional) Add the misspelled word to the user's personal dictionary (Add)
In this design, the client software is responsible for assembling the text to be checked, for performing all edits (e.g., word replacements) to the text, and for interacting with the user. The server is responsible for checking the spelling of submitted text, offering suggested replacements for misspelled words, and identifying words which should be automatically or conditionally replaced.
The interaction described above is carried out via a series of requests (sent by the client to the server) and replies (sent by the server to the client). The conversation between client and server is always initiated by the client. The server never sends unsolicited information to the client. The server always sends exactly one reply for each request.
To check the spelling of some text, the client sends a Check Spelling Request to the server. The Check Spelling Request contains the following information:
The text to be checked
A cursor which contains the position of the first word to check
Spelling options, such as "ignore words in all caps" or "report doubled words." The options are specified as a bit-mask formed from SSCE_xxx_OPT values.
A list of words which should be ignored or replaced (the temporary dictionary)
(Optional) A list of words in the user's personal dictionary (the user dictionary)
In response to the request, the server checks the spelling of the provided text (described in greater detail later). When all the text has been checked, the server returns to the client a Check Spelling Reply containing the following information:
The number of problems detected
A list of problem descriptions containing:
The problem type (misspelled word, doubled word, capitalization error, automatic or conditional correction needed)
The position of the problem word in the text
A list of suggested replacement words (only for misspelled word problem types)
An alternate word used in automatic or conditional correction or for capitalization errors (e.g., if a word was spelled correctly but incorrectly capitalized, the alternate word would contain the correct capitalization form).
The server uses the Sentry Spelling Checker Engine (SSCE) to check spelling. All clients are assumed to submit text for checking in the same language. The dictionaries for that language are memory-based and are statically linked to the server's code. (If clients can submit text for checking in multiple languages, then the dictionaries for all available languages would be linked to the server, and the Check Spelling Request would contain a flag indicating which language to use.)
The server builds a memory-based temporary dictionary containing the words in the Check Spelling Request's temporary dictionary. If the Check Spelling Request contains a user dictionary, the server builds a memory-based temporary dictionary for that as well.
To check the text contained in the Check Spelling Request, the server calls the WS_CheckText function in SSCE's API. WS_CheckText checks the spelling of words in a text string, starting at a given cursor position, against a set of dictionaries (in this case, the set of dictionaries would include the temporary dictionary plus the main dictionary for the language of the text being checked). WS_CheckText returns when it finds a misspelled or doubled word, an automatic or conditional correction is needed, or the end of the text is reached.
If WS_CheckText reports that a misspelled word was detected, the server calls the WS_Suggest function to obtain a list of suggested replacements for the misspelled word. The server then adds the position of the misspelled word and the list of suggestions to the Check Spelling Reply.
If WS_CheckText reports that a doubled word was detected, the server adds the position of the word to the Check Spelling Reply. A list of suggestions is not needed.
If WS_CheckText reports that a capitalization error was detected or an automatic or conditional correction is needed, the server adds the position of the word plus the alternate word to the Check Spelling Reply.
When WS_CheckText reports that all words in the text have been checked, the server returns the Check Spelling Reply to the client. At this point, the server has completed processing the Check Spelling Request and can release any resources allocated during its processing. This would include destroying the temporary and user dictionaries.
The client software reponds to receipt of a Check Spelling Reply from the server by presenting any problems reported to the user. Depending on the user interface design, this could be done through a dialog box that presents each problem in sequence and requests the user to dispose of the problem, or the client software could mark each problem word in the document as misspelled (by changing the word's text color, for example) and allow the user to dispose of problems at his or her leisure.
The client software's response to each disposition is listed below:
Ignore: The client software skips to the next problem or un-marks the word in the document.
Ignore All: The client software adds the word to the temporary dictionary.
Change: The client software replaces the word in the document with the replacement word entered or selected by the user.
Change All: The client software adds the word and its replacement to the temporary dictionary.
Add: The client software adds the word to the user dictionary (see User dictionaries for more information).
For automatic corrections, the client software replaces the word in the document with the alternate word.
Typically, spelling checkers allow the user to add words to a user dictionary, which contains words personal to the user which are not contained in the spelling checker's main dictionary. The user dictionary might contain the user's family name, business name, technical terms specific to the user's business, etc. Normally, the user dictionary is stored persistently so the user need add his or her personal words just once.
The requirement for persistent storage makes user dictionaries problematical in a client-server system. A decision must be made about where the dictionary is stored: on the client computer or on the server.
If the client software is a specialized application (i.e., not a Web browser), then storing the user dictionary on the client computer is straightforward: The user dictionary is simply read and written as a disk file by the client software.
The more likely situation is that the client software is a Web browser. For security reasons, Web browsers usually prevent Web page scripts and applets from reading and writing disk files on the client computer. Some browsers relax these security restrictions if when running signed applets. Another approach is to store the contents of the user dictionary in a "cookie," which is a form of persistent storage implemented by browsers. The contents of the user dictionary would be read by the client software from the cookie, and placed in the Check Spelling Request. The cookie would need to be updated each time the user added a new word to the dictionary.
If the user dictionary is stored on the server, some means of mapping dictionaries to specific users is needed. If the server requires that the user log in, the user id could be used as a key to locate the user dictionary (the user dictionary contents could be stored in a database table, for example, and the key could be the user id, or the user id could be mapped to the name of a disk file containing the user dictionary on the server). The Check Spelling Request would contain the user id, and the server would use it to open the user dictionary when the Check Spelling Request was processed.
Responsibility for updating the user dictionary would be assigned to the server. A new request would be needed to allow the client software to ask the server to add a word to the user dictionary. This request would be sent by the client when the user disposed of a misspelled word by adding it to the user dictionary. The server's response would simply indicate the success or failure status of the request.
In this design, responsibility for editing the checked text (making replacements to correct misspellings) is assigned to the client, as is responsibility for maintaining the temporary dictionary (and optionally the user dictionary). The most compute- and data-intensive operations, checking spelling and obtaining suggestions for misspelled words, is assigned to the server. The advantage to this approach is that the complex software and large data files (the spelling checker engine and dictionaries) do not need to be downloaded to the client. The software required to edit text strings is relatively simple and small enough to be embedded in Web page applets, for example. In a high-bandwidth system, the task of editing the text and maintaining the temporary dictionary could be assigned to the server. Additional requests (replace word, add word to temporary dictionary, etc.) would be needed.
The design presented here is stateless and connectionless. Any client can send the server a Check Spelling Request at any time. The server processes each request as it arrives without regard for any requests sent previously by the same client. The server doesn't keep state information on behalf of its clients. This greatly simplifies the design of the server, since it doesn't have to deal with complex protocols or periodically cleaning up after clients that disappear. However, this approach does require that all information needed to process each request be contained within the request. For example, each Check Spelling Request sent to the server contains the text to check and the temporary dictionary, even if these haven't changed since the last Request was sent. If the text is large, bandwidth is wasted. One solution to this problem is to use a connection-oriented protocol, where the client initiates a spell-check session with the server by sending the text to be checked along with the temporary dictionary (and optionally the user dictionary). The client and server would then enter into a conversation where the server reports misspelled words and the client responds with the disposition (Ignore, Ignore All, Change, Change All, or Add) selected by the user. At the end of the spell-check session, the client would retrieve the possibly updated text and dictionaries from the server.
The design presented here allows multiple clients to check spelling of text concurrently on a centralized server. Issues specific to a common application of this architecture -- clients running a Web browser -- were addressed. Alternatives for specific exceptional cases were proposed.
[Home] [Products] [Catalog] [Ordering] [Search] [Contact Us]
Copyright © 2015 Wintertree Software Inc.