Finding Information on the Web

Searching the Web

There are many freely available sources of information on the Internet, and many avenues to find them, including search engines, subject directories, listservs, and newsgroups. In addition to these sources, there are many subscription- or purchase-only online resources. Wesleyan Library's list of Indexes and Databases are examples of research sources available only by subscription or purchase. These subscription resources, though they are available over the Web, are better considered as library resources, since they have been selected and purchased by the library. This tutorial covers searching the "free" Web; see the Research Skills Tutorial for using print and online library resources.

When searching the free Web, it is important first to determine what specifically you are looking for, and then decide which of these avenues is best suited to your purposes:

State your research topic in the form of a question; treat your research project as an attempt to find a specific answer for a specific question.
Analyze your topic: List terms and ideas that describe your topic. List synonyms for those terms, along with broader and narrower topics, categories, and terms. List names (authors, organizations, etc), titles, abbreviations, and acronyms associated with your topic.
Determine what sort of an answer you need: From what subject or discipline perspective are you looking at this topic? Do you need scholarly or popular sources? Everything you can find on the topic, or just something brief? Do you want an overview of a broad topic, a narrow aspect of a topic, a specific fact, etc? How familiar are you with this topic?
Select the appropriate search avenues listed below and use them.
Evaluate your results: Are the results relevant, or do you need to change your search strategy? Do you have what you need to your question? Do you need to rephrase your question?
Repeat as necessary.

When selecting a search engine, subject directory, or other tool for searching the Web, keep in mind that no one tool or method comes close to covering the entire Web. Your favorite search engine covers only a fraction of what is available on the Web, so if you are doing a thorough search you will need to use several different tools.

In addition, there is no standard categorization or cataloging of resources on the Web. Unlike a library catalog or a subject index of journal articles, there is no standardized list of subject headings or defined structure to use to make sure you get all and only Web sites relevant to your topic. To do a thorough search for your topic, you will need to do many searches using different keywords, synonyms, names, etc. For a list of useful search strategies, see the Searching Indexes and Databases section of the Research Skills Tutorial.

Also keep in mind that anyone with a computer and a modem can put up a Web page. Just because something is on the Web does not mean that it is at all reliable. It is crucial to evaluate what you find on the Web before you rely on it for your research.

Avenues for Searching

Search engines allow you to perform keyword searches for Internet sources. Search engines send "spiders" through the Web, robot programs which follow links and make copies of what is available on the Web, and load those copies onto the search engine's database. You can then search these databases to find sites that match your keywords. Search engines are useful if you are doing a thorough search of as many Internet sources you can find on a topic, if you are researching a narrow or specific topic, or if you just want to quickly find a page or two relevant to your topic. Keep in mind that search engines can find a lot of information, but they cannot evaluate its reliability or accuracy.

Subject directories are collections of Internet resources organized into subject categories, usually arranged from general to more specific subjects. Many directories also provide evaluations of the sites they index, and/or allow you to perform a keyword search to find indexed sites. Subject directories are useful if you are looking for a broad or general topic, or if you are unfamiliar with a topic and you want sites recommended by subject specialists. Since they are compiled by people rather than by computers they cannot include more than a miniscule fraction of the Web, but a well done directory will do a good job of covering the most significant sites relevant to a topic. In other words, in a subject directory someone else has already done the work of searching and sorting for you.

Invisible Web refers to information on the Web that is freely available but not accessible by search engines. A search engine's spiders often cannot index graphics, CGI scripts, PDF files, and data in online databases. Databases are a particular problem for search engines, since they create "pages" dynamically on demand for an individual user, and thus are not even available in principle for a search engine's spider to scan. But since these databases are so versatile at providing customized content for users, more sites are putting content into online databases rather than on static pages. Thus, a growing portion of the Web cannot be indexed by standard search engines. There are, however, directories which help you find online databases and other sources for finding information that a search engine could not retrieve. Typically, the directories allow you to search or browse for a general topic to find a list of databases likely to contain specific information.

Listservs are e-mail mailing lists centered around a particular topic. When one person sends an e-mail message to the list, that message is forwarded to everyone subscribed to the list. All you need to participate in a listserv is an e-mail account.

Newsgroups are like bulletin boards where you can post and read messages on particular topics. Unlike listservs to which individuals subscribe, newsgroups are subscribed to by an institution or Internet service provider. So not all newsgroups are accessible at a given institution or Internet service provider. In addition, you need news reader software to access newsgroups (included in Netscape, Internet Explorer, and some other browsers).

Next: Search Engines

Contents - Internet - Searching - Search Engines - Subject Directories
Invisible Web - Listservs and Newsgroups - Evaluating - Citing