There is no guarantee of any kind, expressed or implied, regarding the Work or its contents. The Work is sold AS IS and WITHOUT WARRANTY. You may have other legal rights, which vary from state to state.
In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other incidental or consequential damages arising out from the Work or its contents. Because some states do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you.
You should always use reasonable care, including backup and other appropriate precautions, when working with computers, networks, data, and iles.
Syngress Media®, Syngress®, "Career Advancement Through Skill Enhancement®," "Ask the Author UPDATE®" and "Hack Proofing®" are registered trademarks of Syngress Publishing, Inc. "Syngress: The Definition of a Serious Security Library"™, "Mission Critical™" and "The Only Way to Stop a Hacker is to Think Like One™" are trademarks of Syngress Publishing, Inc. Brands and product names mentioned in this book are trademarks or service marks of their respective companies.
KEY
SERIAL NUMBER
001
HJIRTCV764
002
PO9873D5FG
03
829KM8NJH2
004
FGDD458876
005
CVPLQ6WQ23
006
VBP965T5T5
007
HJJJ863WD3E
008
2987GVTWMK
009
629MP5SDJT
010
IMWQ295T6T
PUBLISHED BY Syngress Publishing, Inc. 800 Hingham Street Rockland, MA 02370
Google Hacking for Penetration Testers
Copyright © 2005 by Syngress Publishing, Inc. All rights reserved. Printed in the United States of America. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher, with the exception that the program listings may be entered, stored, and executed in a computer system, but they may not be reproduced for publication.
Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 ISBN: 1-931836-36-1
Publisher: Andrew Williams Acquisitions Editor: Jaime Quigley Technical Editor: Alrik "Murf" van Eijkelenborg Cover Designer: Michael Kavish
Distributed by O'Reilly Media, Inc. in the United States and Canada.
For information on rights and translations, contact Matt Pedersen, Director of Sales and Rights, at Syngress Publishing; email matt@syngress.com or fax to 781-681-3585.
Syngress would like to acknowledge the following people for their kindness and support in making this book possible.
Syngress books are now distributed in the United States and Canada by O'Reilly Media, Inc.The enthusiasm and work ethic at O'Reilly is incredible and we would like to thank everyone there for their time and efforts to bring Syngress books to market:Tim O'Reilly, Laura Baldwin, Mark Brokering, Mike Leonard, Donna Selenko, Bonnie Sheehan, Cindy Davis, Grant Kikkert, Opol Matsutaro, Steve Hazelwood, Mark Wilson, Rick Brown, Leslie Becker, Jill Lothrop,Tim Hinton, Kyle Hart, Sara Winge, C.J. Rayhill, Peter Pardo, Leslie Crandell, Valerie Dow, Regina Aggio, Pascal Honscher, Preston Paull, Susan Thompson, Bruce Stewart, Laura Schmier, Sue Willing, Mark Jacobsen, Betsy Waliszewski, Dawn Mann, Kathryn Barrett, John Chodacki, and Rob Bullington. And a hearty welcome to Aileen Berg—glad to be working with you.
The incredibly hard working team at Elsevier Science, including Jonathan Bunkell, Ian Seager, Duncan Enright, David Burton, Rosanna Ramacciotti, Robert Fairbrother, Miguel Sanchez, Klaus Beran, Emma Wyatt, Rosie Moss, Chris Hossack, Mark Hunt, and Krista Leppiko, for making certain that our vision remains worldwide in scope.
David Buckland, Marie Chieng, Lucy Chong, Leslie Lim, Audrey Gan, Pang Ai Hua, and Joseph Chan of STP Distributors for the enthusiasm with which they receive our books.
Kwon Sung June at Acorn Publishing for his support.
David Scott, Tricia Wilden, Marilla Burgess, Annette Scott, Andrew Swaffer, Stephen O'Donoghue, Bec Lowe, and Mark Langley of Woodslane for distributing our books throughout Australia, New Zealand, Papua New Guinea, Fiji Tonga, Solomon Islands, and the Cook Islands.
Winston Lim of Global Publishing for his help and support with distribution of Syngress books in the Philippines.
A special thanks to Tim MacLellan and Darci Miller for their eternal patience and expertise.
Author
Johnny Long has spoken on network security and Google hacking at several computer security conferences around the world including SANS, Defcon, and the Black Hat Briefings. During his recent career with Computer Sciences Corporation (CSC), a leading global IT services company, he has performed active network and physical security assessments for hundreds of government and commercial clients. His website, currently the Internet's largest repository of Google hacking techniques, can be found at http://johnny.ihack-stuff.com.
Alrik "Murf" van Eijkelenborg is a systems engineer for MBH Automatisering. MBH provides web applications, hardware, hosting, network, firewall, and VPN solutions. His specialties include technical support and consulting on Linux, Novell and Windows networks. His background includes positions as a network administrator for Multihouse, NTNT, K+V Van Alphen, Oranjewoud and Intersafe Holding. Alrik holds a bachelor's degree from the Business School of Economics (HES) in Rotterdam,The Netherlands. He is one of the main moderators for the Google Hacking Forums and a key contributor to the Google Hacking Database (GHDB).
Contributing Authors
Steven "The Psyko" Whitacre [MCSE] is a senior network engineer with OPT, Inc, a leading provider of networking solutions in the San Francisco Bay Area, providing senior level network administration and security consulting to companies throughout the greater Bay Area. His specialties include: network design, implementation, administration, data recovery, network reconstruction, system foren-sics, and penetration testing. Stevens consulting background includes work for large universities, financial institutions, local law enforcement, and US and foreign government agencies. Steven is a former member of COTSE/Packetderm, and currently volunteers his time as a moderator for one of the largest security related forums on the Internet. Steven resides in San Francisco, CA with his wife and two daughters, and credits his success to their unwavering support.
James C. Foster, Fellow, is the Deputy Director of Global Security Solution Development for Computer Sciences Corporation where he is responsible for the vision and development of physical, personnel, and data security solutions. Prior to CSC, Foster was the Director of Research and Development for Foundstone Inc. (acquired by McAfee) and was responsible for all aspects of product, consulting, and corporate R&D initiatives. Prior to joining Foundstone, Foster was an Executive Advisor and Research Scientist with Guardent Inc. (acquired by Verisign) and an adjunct author at Information Security Magazine (acquired by TechTarget), subsequent to working as Security Research Specialist for the Department of Defense. With his core competencies residing in high-tech remote management, international expansion, application security, protocol analysis, and search algorithm technology, Foster has conducted numerous code reviews for commercial OS components, Win32 application assessments, and reviews on commercial-grade cryptography implementations.
Foster is a seasoned speaker and has presented throughout North America at conferences, technology forums, security summits, and research symposiums with highlights at the Microsoft Security Summit, Black Hat USA, Black Hat Windows, MIT Wireless Research Forum, SANS, MilCon,TechGov, InfoSec World 2001, and the Thomson Security Conference. He also is commonly asked to comment on pertinent security issues and has been sited in USAToday, Information Security Magazine, Baseline, Computer World, Secure Computing, and the MIT Technologist. Foster holds an A.S., B.S., MBA and numerous technology and management certifications and has attended or conducted research at the Yale School of Business, Harvard University, the University of Maryland, and is currently a Fellow at University of Pennsylvania's Wharton School of Business. Foster is also a well published author with multiple commercial and educational papers; and has authored, contributed, or edited for major publications including Snort 2.1 Intrusion Detection (Syngress Publishing, ISBN: 1-931836-04-3); Hacking Exposed, Fourth Edition, Anti-Hacker Toolkit, Second Edition;Advanced Intrusion Detection; Hacking the Code:ASP.NET Web Application Security (Syngress, ISBN: 1-932266-65-8); Anti-Spam Toolkit; and Google Hacking for Penetration Testers (Syngress, ISBN: 1-931836-36-1).
Matt Fisher is a Senior Security Engineer for SPI Dynamics, which specializes in automated web application security assessments products for the entire software development lifecycle. As an engineer at SPI Dynamics, he has performed hundreds of web application assessments and consulted to the Fortune 500, Federal Government, and Department of Defense. He has educated thousands on web application security through presentations at numerous conferences and workshops both domestically and abroad. Prior to working for SPI Dynamics, he managed large-scale complex Fortune 500 websites at Digex. He has held technical certifications from Novell, Checkpoint, Microsoft, ISC2, and SPI Dynamics.
Matt lives in Columbia, MD, and was only able to write his contribution for this book only through the grace and enduring patience of his family Lisa, Jacob, and Olivia. He'd like to take this last line to give a shout to his coworkers and friends at SPI Dynamics and SPI Labs whom that make it the best place in the world to work, Nummish for the constant help with his futile coding efforts, and of course his Mum who is eternally proud of him."Hi Mom!"
Pete Herzog (OPST, OPSA, HHST), is co-creator of ISECOM and is directly involved in all ISECOM projects as Managing Director. He has arrived from a long career in the security line of business. His main objective is for ISECOM is to improve international security and ethics (www.isecom.org/projects/rules.shtml) from the night watchman to the high-tech system designers to the high school student (http://www.hackerhighschool.org).This has led beyond methodologies to the successful Hacker Highschool program, a free security awareness program for high schools. In addition to managing ISECOM, Pete teaches the masters for security at La Salle University in Barcelona which accredits the OPST and OPSA training courses as well as Business Information Security in the ESADE MBA program, which is the foundation of the OPSA. Additionally Pete provides both paid and pro-bono consultancy on the business of security and security testing to companies of all sizes in an effort to raise the bar on security practice as well as to stay current in the security industry.
I'm Johnny. I hack stuff.
Have you ever had a hobby that changed your life? I have a tendency to get hyper-focused on my hobbies, but this "Google Hacking thing", although it's labeled me "That Google Guy" has been a real blessing for me. I've been published in the papers, written about, and linked more times than I can count. I'm now invited to speak at the conferences I once attended in awe. I've been to Japan and back, and now, much to my disbelief, written a large portion of the book you hold now. I've met many, many amazing people and I've made some close friends despite the fact that I've never actually "met" most of them. I've been given amazing opportunities, and there's no apparent end in sight. I owe many people a huge debt of thanks, but it's "printing day" for this book, and I'm left with a few short minutes to express my gratitude. It's simply not enough, and to all those I've forgotten, I'm sorry.You know you helped, so thanks. = /
First and foremost, thanks to God for the many blessings in my life. Christ for the Living example, and the Spirit of God that encourages me to live each day with real purpose.Thanks to my wife and three wonderful children. Words can't express how much you mean to me. Thanks for putting up with the "real" jOhnny.
Thanks to Mom and Dad for letting me stay up all hours as I fed my digital addiction.
Thanks to the book team, Alrik "Murf" van Eijkelenborg, James Foster, Steve, Matt, Pete and Roelof. Mr. Cooper, Mrs. Elliott, Athy C, Vince Ritts,Jim Chapple,Topher H, Mike Schiffman, Dominique Brezinski and rain.forest.puppy all stopped what they were doing to help shape my future. I couldn't make it without the help of close friends to help me through life: Nathan B, Sujay S, Stephen S.Thanks to Mark Norman for keeping it real.
The Google Masters from the Google Hacking forums made many contributions to the forums and the GHDB, and I'm honored to list them here in descending post total order: murfie, jimmyneutron, klouw, l0om,ThePsyko,
MILKMAN, cybercide, stonersavant, Deadlink, crash_monkey, zoro25, Renegade334, wasabi, urban, mlynch, digital.revolution, Peefy, brasileiro, john, Z!nCh, ComSec, yeseins, sfd, sylex, wolveso, xlockex, injection33, Murk. A special thanks to Murf for keeping the site afloat while I wrote this book, and also to mod team:ThePsyko, l0om, wasabi, and jimmyneutron.
The StrikeForce was always hard to describe, but it encompassed a large part of my life, and I'm very thankful that I was able to play even a small part: Jason A, Brian A, Jim C, Roger C, Carter, Carey, Czup, Ross D, Fritz, Jeff G, Kevin H, Micha H,Troy H, Patrick J, Kristy,Dave Klug, Logan L,Laura,Don M, Chris Mclelland, Murray, Deb N, Paige, Roberta, Ron S, Matty T, Chuck T, Katie W, Tim W, Mike W
Thanks to CSC and the many awesome bosses I've had.You rule:"FunkSoul", Chris S, Matt B, Jason E, and Al E.Thanks to the 'TIP crew for making life fun and interesting five days out of seven.You're too many to list, but some I remember I've worked with more than others: Anthony, Brian, Chris, Christy, Don, Heidi, Joe, Kevan,The 'Mikes', "O", Preston, Richard, Rob, Ron H, Ron D, Steve, Torpedo, Thane.
It took a lot of music to drown out the noise so I could churn out this book. Thanks to P.O.D. (thanks Sonny for the words), Pillar, Project 86,Avalon O2 remix, D.J. Lex,Yoshinori Sunahara, Hashim and SubSeven (great name!).
Shouts to securitytribe, Joe Grand, Russ Rogers, Roelof Temmingh, Seth Fogie, Chris Hurley, Bruce Potter, Jeff, Ping, Eli, Grifter at Blackhat, and the whole Syngress family of authors. I'm honored to be a part of the group, although you all keep me humble! Thanks to Andrew and Jaime.You guys rule!
Thanks to Apple Computer, Inc for making an awesome laptop (and OS). Despite being bounced down my driveway due to a heartbreaking bag failure a month after I bought it, my 12" G4 PowerBook wasn't affected in the slightest. That same laptop was used to layout, author and proof more than 10 chapters of this book, maintain and create my website, and present to the masses at all the conferences. No ordinary laptop could have done all that. I only wish it wasn't so ugly and dented. (http://johnny.ihackstuff.com/images/dent.jpg)
Hr Foreword
Have you ever seen the movie, The Matrix? If you haven't, I strongly recommend that you rent this timeless sci-fi classic. Those who have seen The Matrix will recall that Keanu Reeves's character, a hacker named Neo, awakes to find himself in a vicious battle between humans and computer programs with only a rag-tag crew of misfits to help him win the fight.
Neo learns the skills he needs for battle from Morpheus, a Zen-like master played by Laurence Fishburne. As the movie unfolds, Neo is wracked with questions about his identity and destiny. In a crucial scene, Morpheus takes Neo to someone who can answer all of his questions: the Oracle, a kindly but mysterious grandmother who leads Neo down the right path by telling him just what he needs to know. And to top off her advice, the Oracle even gives Neo a cookie to help him feel better.
So what does The Matrix have to do with this book? Well, my friends, in our matrix (that is, the universe that you and I inhabit), the Oracle is none other than Google itself.Think about it.Whenever you have a question, whether big or small, you go to the Oracle (Google) and ask away. "What's a good recipe for delicious pesto?" "Are my dog's dentures a legitimate tax write-off?""Where can I read a summary of the post-modern philosophical work Simulacra and Simulation?"The Oracle answers them all. And if you configure some search preferences, the Oracle—i.e., Google—will even give your Web browser a cookie.
But, of course, you'll get far more information from the Oracle if you ask the proper questions. And here's the best part: in this book, Johnny Long plays Morpheus, and you get to be Neo. Just as Fishburne's character tutored and inspired Neo, so too will Johnny show you how to maximize the value of your interactions with Google. With the skills Johnny covers in this book, your Google kung fu will improve dramatically, making you a far better penetration tester and security practitioner.
In fact, even outside the realm of information security, I personally believe that solid Google skills are some of the most important professional capabilities you can have over the next five to 10 years. Are you a professional penetration tester? Puzzled parent? Political partisan? Pious proselyte? Whatever your walk is in life, if you go to Google and ask the right questions using the techniques from this book, you will be more thoroughly armed with the information that you need to live successfully.
What's more, Johnny has written this book so that you can learn to ask Google for the really juicy stuff—secrets about the security vulnerabilities of Web sites. Using the time-tested advice on these pages, you'll be able to find and fix potentially massive problems before the bad guys show up and give you a very bad day. I've been doing penetration testing for a decade, and have consistently been astounded by the usefulness of Web site searches in our craft. When Johnny originally started his Web site, inventorying several ultra-powerful search strategies a few years back, I became hooked on his stuff. In this book, he's now gathered his best tricks, added a plethora of new ideas, and wrapped this information in a comprehensive methodology for penetration testing and ethical hacking.
If you think,"Oh, that Google search stuff isn't very useful in a real-world penetration test... that's just playing around," then you have no idea what you are talking about. Whenever we conduct a detailed penetration test, we try to schedule at least one or two days for a very thorough investigation to get a feel for our target before firing a single packet from a scanner. If we can get even more time from the client, we perform a much deeper investigation, starting with a thorough interrogation of our favorite recon tool, Google.With a good investigation, using the techniques Johnny so masterfully shares in this book, our penetration-testing regimen really gets off on the right foot.
I especially like Johnny's clear-cut, no-bones-about-it style in explaining exactly what each search means and how you can maximize the value of your results.The summary and FAQs at the end of each chapter help novices and experts examine a treasure trove of information.With such intrinsic value, I'll be keeping this book on the shelf near my desk during my next penetration test, right next to my well-used Matrix DVD.
Google Searching Basics
Solutions in this Chapter:
■ Exploring Google's Web-Based Interface
■ Building Google Queries
■ Working With Google URLs
Summary
Solutions Fast Track
Frequently Asked Questions
Introduction
Google's Web interface is unmistakable. Its "look and feel" is copyright-protected, and for good reason. It is clean and simple. What most people fail to realize is that the interface is also extremely powerful. Throughout this book, we will see how you can use Google to uncover truly amazing things. However, as in most things in life, before you can run, you must learn to walk.
This chapter takes a look at the basics of Google searching. We begin by exploring the powerful Web-based interface that has made Google a household word. Even the most advanced Google users still rely on the Web-based interface for the majority of their day-to-day queries. Once we understand how to navigate and interpret the results from the various interfaces, we will explore basic search techniques.
Understanding basic search techniques will help us build a firm foundation on which to base more advanced queries.You will learn how to properly use the Boolean operators (AND, NOT, and OR) as well as exploring the power and flexibility of grouping searches. We will also learn Google's unique implementation of several different wildcard characters.
Finally, you will learn the syntax of Google's URL structure. Learning the ins and outs of the Google URL will give you access to greater speed and flexibility when submitting a series of related Google searches. We will see that the Google URL structure provides an excellent "shorthand" for exchanging interesting searches with friends and colleagues.
Exploring Google's Web-Based Interface
Soon we will begin using advanced queries aimed at pages containing very specific content. Locating these pages requires skill in search reduction. The following sections cover this in detail.
Google's Web Search Page
The main Google Web page, shown in Figure 1.1, can be found at www.google.com.The interface is known for its clean lines, pleasingly uncluttered feel, and friendly interface. Although the interface might seem relatively featureless at first glance, we will see that many different search functions can be performed right from this first page.
Figure 1.1 The Main Google Web Page
06P
As shown in Figure 1.1, there is only one place on the page in which the user can type.This is the search field. In order to ask Google a question or query, you simply type what you're looking for and either press Enter (if your browser supports it) or click the Google Search button to be taken to the results page for your query.
The links above the search field (Web, Images, Groups, and so on) open the other search areas shown in Table l.l.The basic search functionality of each section is the same. Each search area of the Google Web interface has different capabilities and accepts different search operators, as we will see in the next chapter. For example, the inauthor operator was designed to be used in the groups search area.Table 1.1 outlines the functionality of each distinct area of the main Google Web page.
Table 1.1 The Links and Functions of Google's Main Page
Interface Section
Description
Web, Images, Groups, Directory; News; Froogle; and more >> tabs
Search term input field
Submit Search button
I'm Feeling Lucky button
Advanced Search
Preferences
Language tools
These tabs allow you to search Web pages, photographs, message group postings, Google directory listings, news stories, and retail print advertisements, respectively. If you are a first-time Google user, understand that these tabs are not always a replacement for the Submit Search button.
Located directly below the alternate search tabs, this text field allows you to enter a Google search term. We will discuss the syntax of Google searching throughout this book.
This button submits your search term. In many browsers, simply pressing the Enter/Return key after typing a search term will activate this button.
Instead of presenting a list of search results, this button will forward you to the highest-ranked page for the entered search term. Often this page is the most relevant page for the entered search term.
This link takes you to the Advanced Search page as shown. Much of the advanced search functionality is accessible from this page. Some advanced features are not listed on this page. We will look at these advanced options in the next chapter.
This link allows you to select several options (which are stored in cookies on your machine for later retrieval). Available options include language selection, parental filters, number of results per page, and window options.
This link allows you to set many different language options and translate text to and from various languages.
Google Web Results Page
After processing a search query, Google displays a results page.The results page, shown in Figure 1.2, lists the results of your search and provides links to the Web pages that contain your search text.
The top part of the search result page mimics the main Web search page. Notice the Images, Groups, News, and Froogle links at the top of the page. By clicking these links, you automatically resubmit your search as an Image, Group, News, or Froogle search, without having to retype your query.
The results line shows which results are displayed (1—10, in this case), the approximate total number of matches (here, about 634,000), the search query itself (including links to dictionary lookups of individual words), and the amount of time the query took to execute.The speed of the query is often overlooked, but it is quite impressive. Even large queries resulting in millions of hits are returned within a fraction of a second!
For each entry on the results page, Google lists the name of the site, a summary of the site (usually the first few lines of content), the URL of the page that matched, the size and date the page was last crawled, a cached link that shows the page as it appeared when Google last crawled it, and a link to pages with similar content. If the result page is written in a language other than your native language and Google supports the translation from that language into yours (set in
the preferences screen), a link titled Translate this page will appear, allowing you to read an approximation of that page in your own language (see Figure 1.3).
It's possible to use Google as a transparent proxy server via the translation service. When you click a Translate this page link, you are taken to a translated copy of that page hosted on Google's servers. This serves as a sort of proxy server, fetching the page on your behalf. If the page you want to view requires no translation, you can still use the translation service as a proxy server by modifying the hl variable in the URL to match the native language of the page. Bear in mind that images are not proxied in this manner. We will cover Translation Proxies further in Chapter 3.
Google Groups
Due to the surge in popularity of Web-based discussion forums, blogs, mailing lists, and instant-messaging technologies, USENET newsgroups, the oldest of public discussion forums, have become an overlooked form of online public discussion. Thousands of users still post to USENET on a daily basis. A thorough discussion about what USENET encompasses can be found at www.faqs.org/ faqs/usenet/what-is/part1/. DejaNews (deja.com) was once considered the
authoritative collection point for all past and present newsgroup messages until Google acquired deja.com in February 2001 (see www.google.com/press/ pressrel/pressrelease48.html).This acquisition gave users the ability to search the entire archive of USENET messages posted since 1995 via the simple, straightforward Google search interface. Google refers to USENET groups as Google Groups.Today, Internet users around the globe turn to Google Groups for general discussion and problem solving. It is very common for IT practitioners to turn to Google's Groups section for answers to all sorts of technology-related issues.The old USENET community still thrives and flourishes behind the sleek interface of the Google Groups search engine.
The Google Groups search can be accessed by clicking the Groups tab of the main Google Web page or by surfing to http://groups.google.com.The search interface (shown in Figure 1.4) looks a bit different from other Google search pages, yet the search capabilities operate in much the same way. The major difference between the Web search page and the Groups search page lies in the newsgroup browsing links.
Entering a search term into the entry field and clicking the Search button whisks you away to the Groups search results page (summarized in Table 1.2), which varies quite a bit from the other Google results pages.
Table 1.2 Google Groups Search Links
Interface Section
Description
Advanced Groups Search
Groups Help
alt., biz., comp., etc. links
This link takes you to the Advanced Groups Search page, which allows for more precise searches. Not all advanced features are listed on this page. We will look at these advanced options in the next chapter.
This link takes you to the Google Groups Frequently Asked Question page.
These links reflect the topical hierarchy of USENET itself. By clicking on the links, you can browse through Google groups to read messages in a 'threaded' format.
Google Image Search
The Google Image search feature allows you to search (at the time of this writing) over 880 million graphic files that match your search criteria. Google will attempt to locate your search terms in the image filename, in the image caption, in the text surrounding the image, and in other undisclosed locations, to return a "de-duplicated" list of images that match your search criteria.The Google Image search operates identically to the Web search, with the exception of a few of the advanced search terms, which we will discuss in the next chapter. The search results page is also slightly different, as you can see in Figure 1.5.
The page header is nearly identical to the Web search results page, as is the results line. The Show: line is unique to image results. This line allows you to select images of various sizes to show in the results.The default is to display images of all sizes. Each matching image is shown in a thumbnail view with the original resolution and size followed by the URL of the image.
Google Preferences
You can access the Preferences page by clicking the Preferences link from any Google search page or by browsing to www.google.com/preferences.These options primarily pertain to language and locality settings, as shown in Figure 1.6.
The Interface Language option describes the language that Google will use when printing tips and informational messages. In addition, this setting controls the language of text printed on Google's navigation items, such as buttons and links. Google assumes that the language you select here is your native language and will "speak" to you in this language whenever possible. Setting this option is not the same as using the translation features of Google (discussed in the following section). Web pages written in French will still appear in French, regardless of what you select here.
To get an idea of how Google's Web pages would be altered by a change in the interface language, take a look at Figure 1.7 to see Google's main page rendered in
"hacker speak." In addition to changing this setting on the preferences screen, you can access all the language-specific Google interfaces directly from the Language Tools screen at www.google.com/language_tools.
Even though the main Google Web page is now rendered in "hacker speak," Google is still searching for Web pages written in any language. If you are interested in locating Web pages that are written in a particular language, modify the Search Language setting on the Google preferences page. By default, Google will always try to locate Web pages written in any language.
Proxy Server Language Hijinks
Proxy servers can be used to help hide your location and identity while you're surfing the Web. Depending on the geographical location of a proxy server, the language settings of the main Google page may change to match the language of the country where the proxy server is located. If your language settings change inexplicably, be sure to check your proxy server settings. It's easy to lose track of when you are running under a proxy and when you're not. As we will see later, language settings can be reverted directly via the URL.
The preferences screen also allows you to modify other search parameters, as shown in Figure 1.8.
SafeSearch Filtering blocks explicit sexual content from appearing in Web searches. Although this is a welcome option for day-to-day Web searching, this option should be disabled when you're performing searches as part of a vulnerability assessment. If sexually explicit content exists on a Web site whose primary
content is not sexual in nature, the existence of this material may be of interest to the site owner.
The Number of Results setting describes how many results are displayed on each search result page.This option is highly subjective, based on your tastes and Internet connection speed. However, you may quickly discover that the default setting of 10 hits per page is simply not enough. If you're on a relatively fast connection, you should consider setting this to 100, the maximum number of results per page.
When checked, the Results Window setting opens search results in a new browser window. This setting is subjective based on your personal tastes. Checking or unchecking this option should have no ill effects unless your browser (or other software) detects the new window as a pop-up advertisement and blocks it. If you notice that your Google results pages are not displaying after you click the Search button, you might want to uncheck this setting in your Google preferences.
Language Tools
The Language Tools screen, accessed from the main Google page, offers several different utilities for locating and translating Web pages written in different lan-guages.The first portion of the Language Tools screen (shown in Figure 1.9) allows you to perform a quick search for documents written in other languages as well as documents located in other countries.
The Language Tools screen also includes a utility that performs basic translation services.The translation form (shown in Figure 1.10) allows you to paste a block of text from the clipboard or supply a Web address to a page that Google can translate into a variety of languages.
In addition to the translation options available from this screen, Google integrates translation options into the search results page. The translation options available from the search results page are based on the language options that are set from the Preferences screen shown in Figure 1.11. In other words, if your interface language is set to English and a Web page listed in a search result is French, Google will give you the option to translate that page into your native language, English. The list of available language translations is shown in Figure 1.11.
Figure 1.11 Google's Translation Languages
English to German English to Spanish English to French English to Italian English to Portuguese f German to English Cerman to French Spanish to English French to English French to German Italian to English Portuguese to English
Google Toolbars
Don't get distracted by the allure of Google "helper" programs such as browser toolbars. You'll find that you have full access to all the important features right from the main Google search screen. Each toolbar offers minor conveniences such as one-click directory traversals or select-and-search capability, but there are so many different toolbars available, you'll have to decide for yourself which one is right for you and your operating environment. Check the FAQ at the end of this section for a list of some popular alternatives.
Building Google Queries
Google query building is a process. There's really no such thing as an incorrect search. It's entirely possible to create an ineffective search, but with the explosive growth of the Internet and the size of Google's cache, a query that's inefficient today may just provide good results tomorrow—or next month or next year.The idea behind effective Google searching is to get a firm grasp on the basic syntax and then to get a good grasp of effective narrowing techniques. Learning the Google query syntax is the easy part. Learning to effectively narrow searches can take quite a bit of time and requires a bit of practice. Eventually, you'll get a feel for it, and it will become second nature to find the needle in the haystack.
The Golden Rules of Google Searching
Before we discuss Google searching, we should understand some of the basic ground rules:
■ Google queries are not case sensitive. Google doesn't care if you type your query in lowercase letters (hackers), uppercase (HACKERS), camel case (hAcKeR), or psycho-case (haCKeR)—the word is always regarded the same way. This is especially important when you're searching things like source code listings, when the case of the term carries a great deal of meaning for the programmer. The one notable
exception is the word or. When used as the Boolean operator, or must be written in uppercase, as OR.
■ Google wildcards. Google's concept of wildcards is not the same as a programmer's concept of wildcards. Most consider wildcards to be either a symbolic representation of any single letter (UNIX fans may think of the question mark) or any series of letters represented by an asterisk. This type of technique is called stemming. Google's wildcard, the asterisk (*), represents nothing more than a single word in a search phrase. Using an asterisk at the beginning or end of a word will not provide you any more hits than using the word by itself.
■ Google stems automatically. Google will stem, or expand, words automatically when it's appropriate. For example, consider a search for pet lemur dietary needs, as shown in Figure 1.12. Google will return a hit that includes the word lemur along with pet and, surprisingly, the word diet, which is short for dietary. Keep in mind that this automatic stemming feature can provide you with unpredictable results.
Figure 1.12 Automatic Stemming
Lemur
... Because the exotic pet industry is considered a business ... anything but a commercially prepared diet (as USDA ... The lemurs were transferred to Wildlife Rescue and ...
www.bi9catrescue.or9/lemur.htm - 13k - Cached - Similar pages
■ Google reserves the right to ignore you. Google ignores certain common words, characters, and single digits in a search. These are sometimes called stop words. When Google ignores any of your search terms, you will be notified on the results page, just below the query box, as shown in Figure 1.13. Some common stop words include who, where, what, the, a, or an. Curiously enough, the logic for word exclusion can vary from search to search.
Consider the search what the cat dragged in. In this example, Google will ignore the terms what, the, and in. However, if any of these terms are searched for individually, Google will accept them as valid terms. Examples include searching just for the term what; this term produces over 300,000,000 hits. Another way to force Google into using common words is to include them in quotes. Doing so submits the search as a phrase, and results will include all the words in the term, regardless of how common they may be. A third way to include ignored words in a search is to precede the term with a + sign, as in the query +and. Submitted without the quotes, taking care not to put a space between the + and the word and, this search returns nearly 4 billion results!
Super-Size That Search!
One very interesting search is the search for +the **. This search produces somewhere in the neighborhood of 5.8 billion search results, making it one of the most prolific searches known! Can you top this
search?
Ten-word limit. Google limits searches to 10 terms.This includes search terms as well as advanced operators, which we'll discuss in a moment.There is a fairly effective way to get more than 10 search terms crammed into a query: Replace Google's ignored terms with the wildcard character (*). Google does not count the wildcard character as a
search term, allowing you to extend your searches quite a bit! Consider a query for the wording of the beginning of the U.S. Constitution:
we the people of the united states in order to form a more perfect union establish justice
This search term is 17 words long. Google ignores many of the terms in the query, specifically the, of, the, in, to, and a. Despite these ignored words, Google further complains that the search is too long and that the word justice was ignored because the search limit is 10 words. If we replace some of the words with the asterisk (the wildcard character) and submit it as:
"we * people * * united states * order * form * more perfect * establish *"
When we include the asterisks, Google no longer complains about the number of words in our search, because we've only submitted nine words (and eight uncounted wildcard characters). We could extend our search even farther, by two more real words and just about any number of wildcards.
Basic Searching
Google searching is a process, the goal of which is to find information about a topic. The process begins with a basic search, which is modified in a variety of ways until only the pages of relevant information are returned. Google's ranking technology helps this process along by placing the highest-ranking pages on the first results page. The details of this ranking system are complex and somewhat speculative, but suffice it to say that for our purposes Google rarely gives us exactly what we need following a single search.
The simplest Google query consists of a single word or a combination of individual words typed into the search interface. Some basic word searches could include:
■ hacker
■ FBI hacker Mitnick
■ mad hacker dpak
Slightly more complex than a word search is a phrase search. A phrase is a group of words enclosed in double-quote marks. When Google encounters a phrase, it searches for all words in the phrase, in the exact order you provide them. Google does not exclude common words found in a phrase. Phrase searches can include
■ "Google hacker"
■ "adult humor"
■ "Carolina gets pwnt"
Phrase and word searches can be combined and used with advanced operators, as we will see in the next chapter.
Using Boolean
Operators and Special Characters
More advanced than basic word searches, phrase searches are still a basic form of a Google query.To perform advanced queries, it is necessary to understand the Boolean operators AND, OR, and NOT.To properly segment the various parts of an advanced Google query, we must also explore visual grouping techniques that use the parenthesis characters. Finally, we will combine these techniques with certain special characters that may serve as shorthand for certain operators, wildcard characters, or placeholders.
If you have used any other Web search engines, you have probably been exposed to Boolean operators. Boolean operators help specify the results that are returned from a query. If you are already familiar with Boolean operators, take a moment to skim this section to help you understand Google's particular implementation of these operators, since many search engines handle them in different ways. Improper use of these operators could drastically alter the results that are returned.
The most commonly used Boolean operator is AND. This operator is used to include multiple terms in a query. For example, a simple query like hacker could be expanded with a Boolean operator by querying for hacker AND cracker.The latter query would include not only pages that talk about hackers but also sites that talk about hackers and the snacks they might eat. Some search engines require the use of this operator, but Google does not.The term AND is redundant to Google. By default, Google automatically searches for all the terms you
Note
When first learning the ways of Google-fu, keep an eye on the area
below the query box on the Web interface. You'll pick up great pointers
to help you improve your query syntax.
The plus symbol (+) forces the inclusion of the word that follows it.There should be no space following the plus symbol. For example, if you were to search for and, justice,for, and all as separate, distinct words, Google would warn that several of the words are too common and are excluded from the search.To force Google to search for those common words, preface them with the plus sign. It's okay to go overboard with the plus sign. It has no ill effects if it is used excessively. To perform this search with the inclusion of all words, consider a query such as +and justice for +all. In addition, the words could be enclosed in double quotes. This generally will force Google to include all the common words in the phrase.This query presented as a phrase would be "and justice for all!'
Another common Boolean operator is NOT. Functionally the opposite of the AND operator, the NOT operator excludes a word from a search. One way to use this operator is to preface a search word with the minus sign (—). Be sure to leave no space between the minus sign and the search term. Consider a simple query such as hacker. This query is very generic and will return hits for all sorts of occupations, like golfers, woodchoppers, serial killers, and those with chronic bronchitis. With this type of query, you are most likely not interested in each and every form of the word hacker but rather a more specific rendition of the term. To narrow the search, you could include more terms, which Google would automatically AND together, or you could start narrowing the search by using NOT
to remove certain terms from your search. To remove some of the more unsavory characters from your search, consider using queries such as hacker —golf or hacker —phlegm.This would allow you to get closer to the hackers you're really looking for: wood choppers!
A less common and sometimes more confusing Boolean operator is OR.The OR operator, represented by the pipe symbol ( | )or simply the word OR in uppercase letters, instructs Google to locate either one term or another in a query. Although this seems fairly straightforward when considering a simple query such as hacker or "evil cybercriminal," things can get terribly confusing when you string together a bunch of ANDs and ORs and NOTs.To help alleviate this confusion, don't think of the query as anything more than a sentence read from left to right. Forget all that order of operations stuff you learned in high school algebra. For our purposes, an AND is weighed equally with an OR, which is weighed as equally as an advanced operator. These factors may affect the rank or order in which the search results appear on the page, but the have no bearing on how Google handles the search query.
Let's take a look at a very complex example, the exact mechanics of which we will discuss in the next chapter:
intext:password | passcode intext:username | userid | user filetype:csv
This example uses advanced operators combined with the OR Boolean to create a query that reads like a sentence written as a polite request. The request asked of Google would read,"Locate all pages that have either password or pass-code in the text of the document. From those pages, show me only the pages that contain either the words username, userid, or user in the text of the document. From those pages, only show me documents that are CSV files." Google doesn't get confused by the fact that technically those OR symbols break up the query into all sorts of possible interpretations. Google isn't bothered by the fact that from an algebraic standpoint, your query is syntactically wrong. For the purposes of learning how to create queries, all we need to remember is that Google read our query from left to right.
Google's cut and dry approach to combining Boolean operators is still very confusing to the reader. Fortunately, Google is not offended (or affected by) parenthesis. The previous query can also be submitted as
intext:(password | passcode) intext:(username | userid | user) filetype:csv
This query is infinitely more readable for us humans, and it produces exactly the same results as the more confusing query that lacked parentheses.
Search Reduction
To achieve the most relevant results, you'll often need to narrow your search by modifying the search query. Although Google tends to provide very relevant results for most basic searches, soon we will begin using advanced queries aimed at pages containing very specific content. Locating these pages requires skill in search reduction.The vast majority of this book focuses on search reduction techniques and suggestions, but it's important that you at least understand the basics of search reduction. As a simple example, we'll take a look at GNU Zebra, free software that manages TCP/IP-based routing protocols. GNU Zebra uses a file called zebra.conf to store configuration settings, including interface information and passwords. After downloading the latest version of Zebra from the Web, we learn that the included zebra.conf.sample file looks like this:
! -*- zebra -*-!
! zebra sample configuration file !
! $Id: zebra.conf.sample,v 1.14 1999/02/19 17:26:38 developer Exp $ !
hostname Router password zebra enable password zebra !
! Interface's description. !
!interface lo
! description test of desc. !
!interface sit0 ! multicast
!
! Static default route sample. !
!ip route 0.0.0.0/0 203.181.89.241 !
!log file zebra.log
To attempt to locate these files with Google, we might try a simple search such as:
"! Interface's description. "
This is considered the base search. Base searches should be as unique as possible in order to get as close to our desired results as possible. Starting with a poor base search completely negates all the hard work you'll put into reduction. Our base search is unique not only because we have focused on the words Interface's and description, but we have also included the exclamation mark, the spaces, and the period following the phrase as part of our search.This is the exact syntax that the configuration file itself uses, so this seems like a very good place to start. However, Google takes some liberties with this search query, making the results less than adequate, as shown in Figure 1.15.
Figure 1.15 Dealing with a Base Search
r\r\rs
Google Search; "I Interface's description.
Results 1 - 59 of about 117 for M! Interface's description, "' . {Dr56 seconds)
Did you mean:"! interface description,"
Rem: Changing the Description Field in Server Manager
... Thus, ADSl's IADs FileServ ice interface exposes this setting, and you use
Ibis interface's Description property to modify I be setting-. „.
w^.winnetmag.cDm/WindowsScrlptlno/ Articla/ArticlelDV2304fi/23046.hlml - Similar pages
CodeGunj: SkDCOMbridoe : COM/DCOM Handler Object
... intcrtactObject: t"o. fin], Interface's duscripUon ... Remarks. Retrieves interface's
110 (even if main method was called with name for interface description] ...
wwwxodegunj.com/Cpp/COM-Tech/ aclivexterttde.phpVG2S55/ - 40k - Cached - Similar pages
Use of OVI with NNM in windows 2000. -HP Dev Resource Cenlral
ovitidala>
do G/cfa maker
,r. by interface Description -iffef=nsme by Interface Type -ifdesc=nr interface
description uses Interface Number (default) -ifdesc^ip
www.tuvalu.tv/mrtg/cfgmakar.litml - 19k ■ Cachcc - Similar oagos
ITwoi1d.com - LINUX DESKTOP APPLICATIONS - Freshen Your Apps with . .., LINUX DESKTOP APPLICATIONS Archive Giatfe 3tores the uaer interface's description you create in an XML file. Al runtime* you call .r.
www.itworid.conVn^nx_flesktap/u^2Q01/ - 37k - Cached - Similar papas
rrwQrld.com - LINUX DESKTOP APPLICATIONS - Freshen Your Apps with . ... Additionally. Glade works with many programming largL-aoes and supports XML. Glade stores the user interface's description you create in an XML file. ...
www itworld.com'nl/ln)(_desktOp/06302001/pf_lncleK html - 19k - Cached - S mi la' pages
Pi spiny .i i-
First, notice that none of the result summaries look anything like our zebra.conf file. Google effectively ignored our punctuation marks and spacing, despite the fact that we enclosed them in double quotes. Google has instead keyed on the words Interface's and description. In addition, Google's auto stemming feature located the word interface in our fourth returned result. Sometimes auto stemming just plain gets in the way.
Bad Form on Purpose
In some cases, there's nothing wrong with using poor Google syntax in a search. If Google safely ignores part of a human-friendly query, leave it alone. The human readers will thank you!
I recommend leaving the syntax as is for clarity, but adding another reduction element to our search, zebra.conf, making our next query:
"! Interface's description.
zebra.conf
This narrows our search and returns results that look much more like the conf file we're looking for, as shown in Figure 1.16.
It's tempting in this situation to simply add:
-"zebra.conf.sample"
to our query to get rid of any search that shows sample zebra.conf files. However, it helps to step into the shoes of the software's users for just a moment. Software installations like this one often ship with a sample configuration file to help guide the process of setting up a custom configuration. Most users will simply edit this file, changing only the settings that need to be changed for their environments, saving the file not as a .sample file but as a .conf file. In this situation, the user could have a live configuration file with the term zebra.conf.sample still in place. Reduction based on this term may remove live configuration files created in this manner.
There's another reduction angle. Notice that our zebra.conf.sample file contained the term hostname Router. This is most likely one of the settings that a user will change, although we're making an assumption that his machine is not named Router.This is less a gamble than reducing based on zebra.conf.sample, however. Adding the reduction term —"hostname Router" to our query brings our results number down and reduces our hits on potential sample files, all without sacrificing potential live hits.
Although it's certainly possible to keep reducing, often it's enough to make just a few minor reductions that can be validated by eye than to spend too much time coming up with the perfect search reduction. Our final (that's four qualifiers for just one word!) query becomes:
"! Interface's description. " zebra.conf -"hostname Router"
This is not the best query for locating these files, however. Advanced operators, discussed in the next chapter, will get us even closer to that perfect query!
Working With Google URLs
Advanced Google users begin testing advanced queries right from the Web interface's search field, refining queries until they are just right. Every Google query can be represented with a URL that points to the results page. Google's results pages are not static pages.They are dynamic and are created "on the fly" when you click the Search button or activate a URL that links to a results page. Submitting a search through the Web interface takes you to a results page that can be represented by a single URL. For example, consider the query ihackstuff.
Once you enter this query, you are whisked away to the following URL, or something similar:
www.google.com/search?q=ihackstuff
If you bookmark this URL and return to it later or simply enter the URL into your browser's address bar, Google will reprocess your search for ihackstuff and display the results.This URL then becomes not only an active connection to a list of results, it also serves as a nice, compact sort of shorthand for a Google query. Any experienced Google searcher can take a look at this URL and realize the search subject.This URL can also be modified fairly easily. By changing the word ihackstuff to iwritestuff, the Google query is changed to find the term iwritestuff.This simple example illustrates the usefulness of the Google URL for advanced searching. A quick modification of the URL can make changes happen fast!
Uncomplicating URL Construction
The only URL parameter that is required in most cases is a query (the q parameter), making the simplest Google URL www.google.com/ search?q=google.
URL Syntax
To fully understand the power of the URL, we need to understand the syntax. The first part of the URL, www.google.com/search, is the location of Google's search script. I refer to this URL, as well as the question mark that follows it, as the base, or starting URL. Browsing to this URL presents you with a nice, blank search page. The question mark after the word search indicates that parameters are about to be passed into the search script. Parameters are options that instruct the search script to actually do something. Parameters are separated by the ampersand (&) and consist of a variable followed by the equal sign (=) followed by the value that the variable should be set to. The basic syntax will look something like this:
www.google.com/search?variablel=value&variable2=value
Let's break apart a simple Google URL to look at the various components:
www.google.com/search?hl=en&q=ihackstuff
The base URL is followed by several parameters, each separated by the ampersand (&) character. Each parameter is made of several variables and values, as shown in Table 1.3.
Special Characters
A URL represents special characters and spaces with hex-encoded equivalents of the characters. Some browsers will adjust a typed URL, replacing special characters and spaces with hex-encoded equivalents. If your browser supports this behavior, your job of URL construction is that much easier.Try this simple test. Type the following URL in your browser's address bar, making sure to use spaces between i, hack, and stuff:
www.google.com/search?q="i hack stuff"
If your browser supports this auto-correcting feature, after you press Enter in the address bar, the URL should be corrected to www.google.com/ search?q="i%20hack%20stuff" or something similar. Notice that the spaces were changed to %20.The percent sign indicates that the next two digits are the hexadecimal value of the space character, 20. Some browsers will take the conversion one step further, changing the double-quotes to %22 as well.
Quick Hex Conversions
To quickly determine hex codes for a character, you can run man ASCII from a UNIX or Linux machine, or Google for the term "ascii table".
Putting the Pieces Together
Google search URL construction is like putting together Legos.You start with a URL and you modify it as needed to achieve varying search results. Many times your starting URL will come from a search you submitted via the Google Web interface. If you need some added parameters, you can add them directly to your URL in any order. If you need to modify parameters in your search, you can change the value of the parameter and resubmit your search. If you need to remove a parameter, you can delete that entire parameter from the URL and resubmit your search. This process is especially easy if you are modifying the URL directly in your browser's address bar.You simply make changes to the URL and press Enter. The browser will automatically fetch the address and take you to an updated search page.You could achieve similar results by poking around Google's advanced search page (www.google.com/advanced_search, shown in Figure 1.17) and by setting various preferences, as discussed earlier, but ultimately you'll find it faster and easier to make quick search adjustments directly through URL modification.
A Google search URL can contain many different parameters. Depending on the options you selected and the search terms you provided, you will see some or all of the variables listed in Table 1.4.These parameters can be added or modified as needed to change your search criteria.
Table 1.4 Google's Search Parameters
Variable
Value
Description
q
start
The search query
0 to the max number of hits
filter
restrict hl
num maxResults 1 to 100
0 or 1
restrict code language code
language code UTF-8 UTF-8
a search phrase
i = include file
type
e = exclude file type
a file extension
m3 = 3 months m6 = 6 months y = past year
The search query.
Used to display pages of results. Result 0 is the first result on the first page of results.
The number of results per page (max 100).
If filter is set to 0, show potentially duplicate results.
Restrict results to a specific country.
This parameter describes the language Google uses when displaying results. This should be set to your native tongue. Located Web pages are not translated.
Language restrict. Only display pages written in this language.
The input encoding of Web searches. Google suggests UTF-8.
The output encoding of Web searches. Google suggests UTF-8.
The value is submitted as an exact phrase. This negates the need to surround the phrase with quotes.
Include or exclude the file type indicated by as_filetype.
Include or exclude this file type as indicated by the value of as_ft.
Locate pages updated within the specified timeframe.
Continued
Table 1.4 Google's Search Parameters
Variable
Value
Description
as_nlo
as_nhi
as_oq as_occt
as_dt
as_sitesearch safe as_rq as_lq
a list of words
any = anywhere title = title of page body = text of page url = in the page URL links = in links to the page
i = only include site or domain
e = exclude site or domain domain or site
active = enable SafeSearch off = disable SafeSearch
URL URL
Find search term in a specific location.
Include or exclude searches from the domain specified by as_sitesearch.
Include or exclude this domain or site as specified by as_dt.
Enable or disable SafeSearch.
Locate pages similar to this URL.
Locate pages that link to this URL.
Some parameters accept a language restrict (Ir) code as a value. The Ir value instructs Google to only return pages written in a specific language. For example, lr=lang_ar only returns pages written in Arabic.Table 1.5 lists all the values available for the Ir field:
Table 1.5 Language Restrict Codes
Ir Language Code
Language
lang_ar
Arabic
lang_bg
Bulgarian
lang_ca
Catalan
Iang_zh-CN
Chinese (Simplified)
Iang_zh-TW
Chinese (Traditional)
lang_hr
Croatian
Table 1.5 Language Restrict Codes
lr Language Code
Language
lang_cs
Czech
lang_da
Danish
lang_nl
Dutch
lang_en
English
lang_et
Estonian
langjl
Finnish
langjr
French
lang_de
German
lang_el
Greek
lang_iw
Hebrew
lang_hu
Hungarian
lang_i s
Icelandic
lang_id
Indonesian
lang_ t
Italian
langja
Japanese
lang_ko
Korean
lang_lv
Latvian
lang_lt
Lithuanian
lang_no
Norwegian
lang_pl
Polish
lang_pt
Portuguese
langjo
Romanian
langju
Russian
lang_sr
Serbian
lang_sk
Slovak
lang_sl
Slovenian
lang_es
Spanish
lang_sv
Swedish
langjr
Turkish
The hl variable changes the language of Google's messages and links. This is not the same as the lr variable, which restricts our results to pages written in a
specific language, nor is it like the translation service, which translates a page from one language to another. Figure 1.18 shows the results of a search for the word food with an hl variable set to DA (Danish). Notice that Google's messages and links are in Danish, whereas the search results are written in English. We have not asked Google to restrict or modify our search in any way.
To understand the contrast between hl and lr, consider the food search resubmitted as an lr search, as shown in Figure 1.19. Notice that our URL is different: There are now far fewer results, the search results are written in Danish, Google added a Search Danish pages button, and Google's messages and links are written in English. Unlike the hl option (Table 1.6 lists the values for the hl field), the lr option changes our search results. We have asked Google to return only pages written in Danish.
The hl value is sticky! This means that if you change this value in your URL, it sticks for future searches. The best way to change it back is through Google preferences or by changing the hl code directly inside the URL.
The restrict variable is easily confused with the Ir variable, since it restricts your search to a particular language. However, restrict has nothing to do with lan-guage.This variable gives you the ability to restrict your search results to one or more countries, determined by the top-level domain name (.us, for example) and/or by geographic location of the server's IP address. If you think this smells somewhat inexact, you're right. Although inexact, this variable works amazingly well. Continuing with our fascination for food, consider a search for food, this time restricting results to DK (Denmark), as shown in Figure 1.20.
Our URL has changed to include the restrict value (select countries shown in Table 1.7), but more important, notice that the returned Web pages are not all from DK.The first hit, for example, from www.euro.who.int, is thought by Google to be physically located in Denmark.
wh0 0p:~# whois 194.234.173.80
% This is the RIPE Whois server.
% The objects are in RPSL format. %
% Rights restricted by copyright.
% See http://www.ripe.net/ripencc/pub-services/db/copyright.html
inetnum: 194.234.173.0 - 194.234.173.255
netname: DK-SUPERTEL
descr: SUPERTEL DANMARK ApS
descr: Telephone Operator
Table 1.7 restrict Code Values (see full table in Appendix C) restrict country code Country
countryKR
Korea, Republic of
countryKW
Kuwait
countryKY
Cayman Islands
countryLK
Sri Lanka
countryMX
Mexico
countryNL
Netherlands
countryNO
Norway
countryNZ
New Zealand
countryPA
Panama
countryPE
Peru
countryPH
Philippines
countryPK
Pakistan
countryPL
Poland
countryPR
Puerto Rico
countryPT
Portugal
countryRO
Romania
countryRU
Russian Federation
countrySA
Saudi Arabia
countrySE
Sweden
countryUA
Ukraine
countryUG
Uganda
countryUM
United States Minor Outlying Islands
countryUS
United States
countryUY
Uruguay
countryUZ
Uzbekistan
countryVA
Holy See (Vatican City State)
countryVG
Virgin Islands (British)
countryVI
Virgin Islands (U.S.)
countryVN
Vietnam
countryZA
South Africa
countryZR
Zaire
Summary
Google is deceptively simple in appearance but offers many powerful options that provide the groundwork for powerful searches. Many different types of content can be searched, including Web pages, message groups such as USENET, images, and more. Beginners to Google searching are encouraged to use the Google-provided forms for searching, paying close attention to the messages and warnings Google provides about syntax. Boolean operators such as OR and NOT are available through the use of the minus sign and the word OR (or the | symbol), respectively, whereas the AND operator is ignored, since Google automatically includes all terms in a search. Advanced search options are available through the Advanced Search page, which allows users to narrow search results quickly. Advanced Google users narrow their searches through customized queries and a healthy dose of experience and good old common sense.
Solutions Fast Track
Exploring Google's Web-Based Interface
0 There are several distinct Google search areas (including Web, group, and image searches), each with distinct searching characteristics and results pages.
0 The Web search page, the heart and soul of Google, is simple,
streamlined, and powerful, enabling even the most advanced searches.
0 A Google Groups search allows you to search all past and present newsgroup posts.
0 The Image search feature allows you to search for nearly a billion graphics by keyword.
0 Google's preferences and language tools enable search customization, translation services, language-specific searches, and much more.
Building Google Queries
0 Google query building is a process that includes determining a solid base search and expanding or reducing that search to achieve the desired results.
0 Always remember the "golden rules" of Google searching. These basic premises serve as the foundation for a successful search.
0 Used properly, Boolean operators and special characters help expand or reduce searches. They can also help clarify a search for fellow humans who might read your queries later on.
Working With Google URLs
0 Once a Google query has been submitted, you are whisked away to the Google results page, the URL of which can be used to modify a search or recall it later.
0 Although there are many different variables that can be set in a Google search URL, the only one the is really required is the q, or query, variable.
0 Some advanced search options, such as as_qdr (date-restricted search by month), cannot be easily set anywhere besides the URL.
Frequently Asked Questions
The following Frequently Asked Questions, answered by the authors of this book, are designed to both measure your understanding of the concepts presented in this chapter and to assist you with real-life implementation of these concepts. To have your questions about this chapter answered by the author, browse to www.syngress.com/solutions and click on the "Ask the Author" form. You will also gain access to thousands of other FAQs at ITFAQnet.com.
Q: Some people like using nifty toolbars. Where can I find information about Google toolbars?
A: Ask Google. Seriously, if you aren't already in the habit of simply asking Google when you have a Google-related question, you should get in that habit. Google can almost always provide an answer if you can figure out the query.
Here's a list of some popular Google search tools:
Q: Are there any techniques I can use to learn how to build Google URL's?
A: Yes.There are a fw ways. First, submit basic queries through the web interface and look at the URL that's generated when you submit the search. From the search results page, modify the query slightly and look at how the URL changes when you submit it.This boils down to "do, then do again."The second way involves using "query builder" programs that present a graphical interface which allows you to select the search options you want, building a Google URL as you navigate through the interface. Keep an eye on the search engine hacking forums at http://johnny.ihackstuff.com, specifically the
"coders corner" where users discuss programs that perform this type of functionality.
Q: What's better? Using Google's interface, using toolbars, or writing URL's?
A: It's not fair to claim that any one technique is better than the others. It boils down to personal preference, and many advanced Google users use each of these techniques in different ways. Many lengthy Google sessions begin as a simple query typed into the www.google.com web interface. Depending on the narrowing process, it may be easier to add or subtract from the query right in the search field. Other times, like in the case of the daterange operator (covered in the next chapter), it may be easier to add a quick 'as_qdr' parameter to the end of the URL.Toolbars excel at providing you quick access to a Google search while you're browsing another page. Most toolbars allow you to select text on a page, right-click on the page and select 'Google search' to submit the selected text as a query to Google. Which technique you decide to use ultimately depends on your tastes and the context in which you perform searches.
No comments:
Post a Comment