Invisible Web Gateways-Portals to the Deep Web

Discussion in 'Lounge' started by Krell, Sep 25, 2005.

Watchers:
This thread is being watched by 1 user.
  1. Krell

    Krell worthless dirtball Expert

    Search for the Cloaked Web



    The Invisible Web is easily accessible..that is, if you know where to look. Fortunately there are many sites that are set up to be "gateways", or portals, to the many databases and otherwise closed-off content that makes up the Invisible Web. Deep Web-Invisible Web-Cloaked Web Gateways
    • The University of Michigan has put together OAIster, (pronounced "oyster") and encourages you to "find the pearls" on the Invisible Web. They have millions of records from more than 405 institutions as diverse as African Journals Online and the Library Network of Western Switzerland.
    • LookSmart's Find Articles.com lets you search print publications for articles; anything from popular magazines to scholarly journals. Be sure to check out their Furl tool to organize your Invisible Web search snippets.
    • The Library Spot is a collection of databases, online libraries, references, and other good info from the Invisible Web. Be sure to check out their "You Asked For It" section, where popular readers' questions are featured.
    • The US Government's official web portal is FirstGov.gov, an extremely deep (as in lots of content) site. You could spend hours here. It's interesting to note how much stuff you can get done online here as well, such as renew your driver's license, shop government auctions, and contact elected officials.
    • Search the vast holding of the UCLA Library online, including their special collections only found on the Invisible Web.
    • Check out Infoplease.com and its searchable Invisible Web databases. Results come from encyclopedias, almanacs, dictionaries, and other online resources only found on the Deep Web.
    • The Central Intelligence Agency has the World Factbook, a searchable directory of flags of the world, reference maps, country profiles, and much, much more. Great for geography buffs or anyone who wants to learn more about their world.
    • University of Idaho has created this Repository of Primary Sources, which contains links to manuscripts, archives, rare books, and much more. Covers not only the United states, but countries all over the world.
    • Lund University Libraries maintains the Directory of Open Access Journals, a collection of searchable scientific and scholarly journals on the Invisible Web.
    • Looking for scientific information on the Invisible Web? Go to Scirus.com first. You can search either scholarly sources or Web sources or both.
    • Canada, ay? Then check out the Archival Records of Alberta. This is a web gateway to photographs, census records, and other archival records.
    • Want to find a plant that will survive overwatering, lack of sunlight, and general forgetfulness? You can probably find something in the USDA's Plants Database on the Invisible Web.
    • The Human Genome Database contains anything you would ever want to know..well, about the human genome on the Deep Web, at least.
    • If you've got a medical question, check out The Combined Health Information Database, or CHID online. Its searchable subject directory is very user-friendly, and you can find information on pretty much anything to do with human health here.
    • Nonprofit organizations need searching tools too. The National Database of Nonprofit Organizations is an extensive site on the Invisible Web that not only provides locations and contact information for nonprofits, but also gives detailed fiscal reports.
    • EEVL Xtra, a service put together by Heriot Watt University in Edinburgh, Scotland. This excellent service has the ability to cross-search 20 engineering, mathematics and computing databases, including content from 50 publishers. Find articles, websites, and more on the Invisible Web.

    What is the Invisible Web?

    What is the Invisible Web? Is it some kind of Area 52-ish, X-Files deal that only those with stamped numbers on their foreheads can access? Well, not exactly. The term "invisible web" mainly refers to the vast repository of information that search engines and directories don't have direct access to, like databases. Unlike pages on the visible Web (that is, the Web that you can access from search engines and directories), information in databases is generally inaccessible to the software spiders and crawlers that create search engine indexes. How Big is the Invisible Web?

    In a word, it's humungous. Bright Planet estimates the invisible, or deep, web as being 500 times bigger than the searchable, or surface of the web. Considering that Google alone covers around 8 billion pages, that's just mind boggling.

    Why Is It Called "The Invisible Web"?

    Spiders meander throughout the Web, indexing the addresses of pages they discover. When these software programs run into a page from the Invisible Web, they don't know quite what to do with it. These spiders can record the address, but can't tell you squat about the information the page contains. Why? There's a lot of factors, but mainly they boil down to technical barriers and/or deliberate decisions on the part of the site owner(s) to exclude their pages from search engine spiders. For instance, university library sites that require passwords to access their information will not be included in search engine results, as well as script-based pages that are not easily read by search engine spiders.
    Why Is The Invisible Web Important?



    Perhaps you think it would be easier to just stick with what you can find with Google or Yahoo. Maybe. However, it's not always easy to find what you're looking for with a search engine, especially if you're looking for something a bit complicated or obscure. Think about the Web as a vast library. You wouldn't expect to just walk in the front door and immediately find information on the history of paper clips lying on the front desk, right? You might have to dig for it. This is where search engines will not necessarily help you, and the Invisible Web will.

    Plus, the fact that search engines only search a very small portion of the web make the Invisible Web a very tempting resource. There's a lot more information out there than we could ever imagine. How Do I Use The Invisible Web?



    Fortunately for you and I, there are many other people that have asked themselves the exact same question, and have put together great sites that serve as a launching point into the Invisible Web. Here are some general gateways:
    • One of the best ones out there is the Direct Search site put together by Gary Price, a librarian and information research consultant. His page is nicely organized into searchable categories and is updated frequently.
    • Another good resource is the Invisible Web Directory, put together by the aforementioned Gary Price and search guru Chris Sherman. This site is a directory of searchable databases, organized by subject.
    • The Resource Discovery Network has resources mostly from the United Kingdom, and is extremely well-organized and very searchable.
    • The University of California at Irvine maintains InfoMine, an incredible resource that at last count included over 100,000 links and access to hundreds, if not thousands, of databases.
    • The Virtual Library is simple and easy to use, with annotated subject links. I especially appreciate the annotations because it helps rule out extraneous search time.


    What About Other Invisible Web Resources?

    There are many, many sites that are set up to dig into the Invisible Web. The University of Kansas's ProFusion metasearch engine provides topical deep Web searches. CompletePlanet.com is a directory of "over 70,000+ searchable databases and specialty search engines."

    Most of the information on the Invisible Web is maintained by academic institutions, and has a higher quality than search engine results. There are "academic gateways" that can help you find this information. The SJSU Academic Gateway is a fabulous resource that enables you to get into not only San Jose public libraries, but the San Jose State University library as well. In addition, there are governmental (US) databases such as Ask Eric, which provide access to over 3000 educational resources (organized by category), and the US Securities and Exchange Commission, which has given a whole new meaning to the phrase "a little light reading." The Bottom Line About The Invisible Web

    This is just the tip of the iceberg, folks. The links I've highlighted in this article barely begin to touch the vast resources available on the Invisible Web. As time goes on, the Deep Web will only get bigger, and that's why it's a good idea to learn how to use it now.





    .
     
  2. .:sp00ky:.

    .:sp00ky:. Poptart Tramp Established Member

    Good read. I had ago on profusion it searches a hole bunch of search sites il try some of the others too.
     
  3. zpman

    zpman Member

    Nice find. I may never sleep again, heh. Too much information for my brain to process. Meltdown...in 15 seconds...
     
  4. truelyme

    truelyme Member

    Thank you for a very informative post, Krell. l have bookmarked this article for futher research as it seems to be something I will certainly use later.
     
  5. napho

    napho Antisocial Bastard Established Member

    This gives people hope that there really ARE nude pictures of Jennifer Love Hewitt out there somewhere.
     
  6. sister_sue48

    sister_sue48 Member

    Very informative post thank you. I will be spending sometime going through a few of the links.
     

Share This Page