Target the right search solution - Internet/Web/Online Service Information

Communications News, Dec, 2000 by Kamran Khan

Hit the bull's eye with the option that is best for your company.

No matter what your business is, finding and sharing information is critical to your success. Whether you are providing a single point of communication via a corporate portal, entering the world of e-commerce, or simply implementing a traditional knowledge-management system, you must first carefully weigh the range of search technology options available to you.

Search engines vary considerably. Simple search will work well with small sites that are easy to navigate and whose data is easy to locate. Many sites require more advanced search solutions to connect people with information in an intelligent, timely manner.

Companies can choose between a wide array of search functionalities to discover the right solution for them. Boolean search, concept search, natural-language queries, relevance ranking and pattern recognition can be applied alone or together to achieve a customized search solution that is right for any organization's search needs.

Depending on your objectives, you may find that a low-end, Boolean-based search engine is sufficient. Boolean, or keyword, search represents the simplest link in the search food chain. Nearly all search engines that are free or are referred to as Internet search engines (e.g., AltaVista, Excite) are Boolean-based.

Boolean searching uses three words to determine results: and, or and not. For example, typing "Sweden or Denmark" will return results containing either of those two words anywhere in the text. This method matches word-for-word, letter-for-letter, leaving little room for interpretation and allowing no tolerance for misspelling or variations in spelling.

Some of today's more advanced search solutions allow you to enter a concept and intelligently recognize it, make assumptions about it and return accurate results. Say you are searching for "international commerce," for example. A search solution capable of concept searching would also automatically locate documents that discuss "foreign trade." The ability to paraphrase the query frees you from having to predict the words that an author might use when discussing a topic.

Concept searching also refines the meaning of the words you choose. For example, the meaning of the word "tank" can be partly determined by the words around it; if the words "military," "vehicle" and "army" occur nearby, then the word "tank" is more likely to refer to the fighting vehicle and less likely to be a container for holding fuel.

Concept searching enables users to find documents that may otherwise be impossible to find with a simple keyword search. More importantly, the search will yield a smaller number of more accurate results, rather than a long list of irrelevant ones.

Advanced search solutions that employ natural-language queries make searching easier by allowing your customers or employees to phrase queries the same way they speak: in plain everyday language. Such search systems understand, for example, that if "why do mosquito bites itch" is used in your query, you are looking for medical information on the side effects of bug bites, not information on pest or disease control.

Natural-language searching does not force the user to master structured search languages. Instead, it allows for the full expression of an information need, using the richness and variety of natural language.

If you frequently search for information that is likely to turn up in many documents or Web pages, you may want to consider a search system with highly accurate and tuned relevancy ranking. A search solution that can rank the relevancy of the documents it retrieves gives a weight to each one based on criteria set in the query.

By weighing a document based on the number of query words and related terms that appear, where they appear in relation to each other, and the ratio of these terms to the total number of words in a document, relevancy ranking brings the most appropriate material to light. Without such relevancy ranking, you would have to spend a prohibitive amount of time determining which of the documents are most pertinent to your needs.

In addition to relevancy ranking, some search solutions include hit highlighting--i.e., highlighting keywords that contain the searched topic to indicate to the user the most relevant portions of the document. This eliminates the confusing, time-consuming and often frustrating process of trying to determine where the relevant material is located in a lengthy document.

Today, there is technology available to combat these issues. Certain solutions are inherently fault tolerant. By evaluating the binary "pattern" used to represent words, such search systems can compensate for misspellings, spelling variations and even errors made by optical character recognition algorithms.

For example, the search might come to a quick end if a potential customer is searching for a "1974 Grimlin" and misspells the name of the American Motors Co. automobile. Pattern recognition technology compensates for the misspelling of "Gremlin." Combine pattern recognition with concept searching, and your search engine also distinguishes between the car and a number of toys with the same name.