Today, the media/informational sector possesses extraordinary importance. [It] provides an increasingly integrated symbolic environment from which the nation derives its ideas, values, and expectations .... The character and quality of the message and image flow, therefore, is a crucial terrain of contention in the time ahead.

Herbert I. Schiller (1919-2000)

Internet users have seen many online directories become increasingly commercial in tone. Not surprisingly, alternatives have emerged and users of these new portals have been in no short supply. A number of portals and search engines, such as the Open Directory Project and Google have been quick to adopt rhetorically loaded terms such as "revolutionary" and "democratic" to separate themselves from their mercenary competitors. Still, especially in the case of Google, the claims may be little more than hot air. The Google "team" makes the following pronouncement:

PageRank capitalizes on the uniquely democratic characteristic of the web by using its vast link structure as an organizational tool. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. Google assesses a page's importance by the votes it receives. But Google looks at more than sheer volume of votes, or links; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important." Source

The early robotic search engines like AltaVista's were impartial; they applied the same keyword indexing procedure to each page that was registered. Even AltaVista has now changed its policies in several ways that make its agenda almost as suspicious as Google's. In a practice that resembles pay-for-play radio payola, AltaVista and Google now sell keywords to advertisers. And they aren't alone. On the strength of its formidable industry allies, including Microsoft, Realnames is turning a handsome profit selling words for $100. One member of the alliance, LookSmart, celebrates its marketing potential as follows:

LookSmart's quality navigation products, its advertising platform of desirable differentiated audiences, and the scale and flexibility of its distribution platform and deep content targeting capabilities combine to create a superior advertising infrastructure. Source

Despite the rather dubious "qualities" of such navigation products, content providers -- both large and small -- are clamoring to be present in the databases of these sorts of advertising cauldrons. The days when the general public could freely submit URLs to the most popular portals may be numbered. Services have existed for years that will register a website address so that the site will be indexed in many search engine databases. Formerly these services were optional. But some search engines, such as Mamma.com -- the "mother of all search engines" -- now only accept new website information when it is received through a paid registration service. WorldSubmit will register a website with 1550 search engines, including Alta Vista, AOL Netfind, Excite, HotBot, Infoseek, Lycos, Netscape, Northern Light, and WebCrawler. It is interesting to note, however, that of these search engines, many are using the same underlying databases. Netscape, AltaVista, and Lycos all make use of the Open Directory Project, a volunteer labor force. While Infoseek and Go are both using the same Disney search database; HotBot and AOL Netfind use Inktomi's index. Likewise, WebCrawler doesn't maintain its own database, instead relying on several other search engines. What this suggests is that control over the collection and quality of available information is not central to the branding of the portals. As LookSmart openly admits, their main product is the consumers using the site, whose attention is sold to corporate advertisers. Recent trends toward consolidation of power and exclusivity of access are transforming reference material into something like an infomercial.

Nevertheless, the emergence of a variety of portals probably also reflects a real need for navigation assistance. There is a reasonable argument to be made for simplifying online research. As the amount of information multiplies, the problem of separating signal from noise becomes more critical. Many millions of out-of-date documents remain available and indexed in the large search engine databases. Although a great deal of the 'content' of the World Wide Web pertains to yesterday's technology, frequently the old pages remain part of the search engine databases. This clouds the waters when one seeks information.

One notable initiative to filter and organize the enormous amounts of information generated on the Web is the Open Directory Project. In some respects the service provided by the project is similar to that of Yahoo!: it maintains a large, hierarchically organized database of links to websites. But unlike most comparable directories, the ODP's database is maintained by volunteers. Each volunteer manages one or more categories within the larger hierarchy of topics. When a site is registered, its address is passed along to the appropriate category editor. Although the results are often better than comparable commercial efforts, the ODP is also struggling to maintain the quality of its information. And there is the question of whether self-appointed experts who work for free are necessarily the right guides.

The ODP database is available for use by commercial portals, and many are availing themselves of the free labor force. Nothing prevents them from augmenting the content of the ODP database with their own paid product endorsements and "deep" targeted advertising. This parasitic relationship between corporate media and free information is likely to continue. Most people using the ODP database are probably unaware that they could use it directly, without advertising, at www.dmoz.org. To the extent that Internet users remain unaware of the origin of free information, they are likely to continue frequenting the most heavily advertised and co-marketed portals.