На главную РАБОТА В ИНТЕРНЕТЕ Инструменты Панель оптимизатора

Новости Реклама у нас Обратная связь

Анализ сайта

Войти или зарегистрироваться

Забыли пароль?

Главная Инструменты Анализ сайта

Данный инструмент поможет Вам Выполнить анализ сайта по наиболее значимым показателям сайта.

URL	digimarconmidwest.com (WHOIS IP, WHOIS DOMAIN)
Заголовок	DigiMarCon Midwest 2024 · Chicago, IL · May 22 - 23, 2024 · Digital Marketing, Media and Advertising Conference
Ключевые слова	(нет)
Описание	The Premier Digital Marketing, Media and Advertising Conference in the Midwestern United States held annually in Chicago, Illinois! Join your peers for 2 days jam packed with digital marketing best practices, latest trends and strategy, also check out the next generation of technology & innovation; Internet, Mobile, AdTech, MarTech & SaaS Technology. See why people keep coming back year after year! Register and secure your seat today.
Тэг h1	<b>Hurry, Limited Passes Remaining!</b>
Размер страницы	571.33 Kb
Кодировка	UTF-8
IP сайта	172.67.165.175 Все сайты на IP
Скорость загрузки	296.83 Kb/s
Время загрузки	1.92 sec
Перенаправление	https://digimarconmidwest.com/
Внутренние / внешние ссылки	Анализ
Анализ контента страницы	Выполнить
Посетителей за сутки по LI	?
Посетителей за месяц по LI	?
Сервер сайта	172.67.165.175
Расположение датацентра	?
Сервер	cloudflare
Заработок	Вы можете зарабатывать на сайте Sape $150 Вы можете зарабатывать на сайте Popunder $150 Заработать на сайте более TeaserNet $150

Постоянная ссылка на страницу

http://oesseo.ru/tools/analysis/digimarconmidwest.com

Основные показатели сайта[ Скрыть ]

Яндекс ТиЦ: 0

Склей домена: Нет

С WWW

Без WWW

Яндекс Ранг: 0 из 6

Яндекс.Каталог: (не описан в каталоге)

Поднять ТиЦ регистрируя сайт в каталогах

Google PageRank: 0 из 10

Alexa Traffic Rank: ?

Alexa Traffic Rank (график):

Счетчик Live Internet:

История показателей[ Скрыть ]

(?.. wait)

Backlink History (Majestic SEO)[ Скрыть ]

Наличие сайта в каталогах[ Скрыть ]

Яндекс Каталог: (Нет)	Посмотреть	DMOZ:	Посмотреть
Google каталог:	Посмотреть	Mail.ru:	Посмотреть
Aport:	Посмотреть	Rambler Top 100:	Посмотреть

Проиндексировано страниц[ Скрыть ]

Яндекс:	Посмотреть	Google:	Посмотреть
Bing:	Посмотреть	Yahoo:	Посмотреть
Rambler:	Посмотреть

Ссылаются на сайт из[ Скрыть ]

Яндекс:	Посмотреть	Сайты из Я-Каталога:	Посмотреть
Яндекс блоги:	Посмотреть	Google:	Посмотреть
Google блоги:	Посмотреть	Yahoo:	Посмотреть
Bing:	Посмотреть	Rambler:	Посмотреть
Яндекс картинки:	Посмотреть	Google картинки:	Посмотреть

Дополнительная информация[ Скрыть ]

В кэше Google:	Посмотреть	В кэше Yandex:	Посмотреть
Похожие сайты в Google:	Посмотреть	История сайта:	Посмотреть
Упоминания в Ответы@Mail.ru:	Посмотреть	Поиск плагиата:	Посмотреть

Дополнительные инструменты[ Скрыть ]

Проверка валидности HTML:	Посмотреть	Проверка валидности CSS:	Посмотреть
Ping\Traceroute\DNS:	Посмотреть	Скорость загрузки страницы:	Посмотреть

Данные страницы[ Скрыть ]

Ответ сервера

Файл robots.txt, размер: 46.74 Kb , проверить на Яндекс.Вебмастер

User-agent: ia_archiver
Disallow: /

User-agent: *
Crawl-delay: 500
Sitemap: /sitemap.xml
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/

User-Agent: Googlebot
Crawl-delay: 500
Disallow: /wp-admin/
Allow: /wp-includes/
Allow: /wp-content/plugins/
Allow: /wp-content/themes/
Allow: /.js$*
Allow: /.inc$*
Allow: /.css$*
Allow: /.php$*

User-Agent: Googlebot-Mobile
Crawl-delay: 500
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/

User-Agent: Googlebot-Image
Crawl-delay: 500
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/

User-Agent: Slurp
Crawl-delay: 500
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/

User-Agent: bingbot
Crawl-delay: 300
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/

User-agent: Mediapartners-Google
Allow: /

User-Agent: ABAC	
Disallow: /

User-Agent: Abont
Disallow: /

User-Agent: abot
Disallow: /

User-Agent: Accept
Disallow: /

User-Agent: Access
Disallow: /

User-Agent: Accoo
Disallow: /

User-Agent: AceFTP
Disallow: /

User-Agent: Acme
Disallow: /

User-Agent: ActiveTouristBot
Disallow: /

User-Agent: Address
Disallow: /

User-Agent: Adopt
Disallow: /

User-Agent: adress
Disallow: /

User-Agent: adressendeutschland
Disallow: /

User-Agent: ADSARobot
Disallow: /

User-Agent: AESOP\_com\_SpiderMan
Disallow: /

User-Agent: agent
Disallow: /

User-Agent: ah\-ha
Disallow: /

User-Agent: Ahead
Disallow: /

User-Agent: ahrefsbot
Disallow: /

User-Agent: ahrefsbot/1.0
Disallow: /

User-Agent: aipbot
Disallow: /

User-Agent: Akregat
Disallow: /

User-Agent: aktuelles
Disallow: /

User-Agent: Alarm
Disallow: /

User-Agent: Albert
Disallow: /

User-Agent: Alek
Disallow: /

User-Agent: Alexibot
Disallow: /

User-Agent: Alligator
Disallow: /

User-Agent: AllSubmitter
Disallow: /

User-Agent: alma
Disallow: /

User-Agent: almaden
Disallow: /

User-Agent: ALot
Disallow: /

User-Agent: Alpha
Disallow: /

User-Agent: Amfi
Disallow: /

User-Agent: amzn\_assoc
Disallow: /

User-Agent: Anal
Disallow: /

User-Agent: Anarchie
Disallow: /

User-Agent: andit
Disallow: /

User-Agent: Anon
Disallow: /

User-Agent: AnotherBot
Disallow: /

User-Agent: Ansearch
Disallow: /

User-Agent: AnswerBus
Disallow: /

User-Agent: antivirx
Disallow: /

User-Agent: Apexoo
Disallow: /

User-Agent: appie
Disallow: /

User-Agent: Aqua_Products
Disallow: /

User-Agent: Arachmo
Disallow: /

User-Agent: archive
Disallow: /

User-Agent: arian
Disallow: /

User-Agent: ASPSe
Disallow: /

User-Agent: ASSORT
Disallow: /

User-Agent: aster
Disallow: /

User-Agent: asterias
Disallow: /

User-Agent: Atari
Disallow: /

User-Agent: ATHENS
Disallow: /

User-Agent: AtHome
Disallow: /

User-Agent: Atlocal
Disallow: /

User-Agent: Atomic_Email_Hunter
Disallow: /

User-Agent: Atomz
Disallow: /

User-Agent: Atrop
Disallow: /

User-Agent: attrib
Disallow: /

User-Agent: autoemailspider
Disallow: /

User-Agent: autohttp
Disallow: /

User-Agent: axod
Disallow: /

User-Agent: b2w
Disallow: /

User-Agent: Back
Disallow: /

User-Agent: BackDoorBot
Disallow: /

User-Agent: BackDoorBot/1.0
Disallow: /

User-Agent: BackStreet
Disallow: /

User-Agent: BackWeb
Disallow: /

User-Agent: Badass
Disallow: /

User-Agent: Baid
Disallow: /

User-Agent: Baidu
Disallow: /

User-Agent: Baiduspider
Disallow: /

User-Agent: Baiduspider/2.0
Disallow: /

User-Agent: Bali
Disallow: /

User-Agent: Bandit
Disallow: /

User-Agent: Barry
Disallow: /

User-Agent: BasicHTTP
Disallow: /

User-Agent: batch
Disallow: /

User-Agent: BatchFTP
Disallow: /

User-Agent: bdfetch
Disallow: /

User-Agent: beat
Disallow: /

User-Agent: Become
Disallow: /

User-Agent: Beij
Disallow: /

User-Agent: BenchMark
Disallow: /

User-Agent: berts
Disallow: /

User-Agent: bew
Disallow: /

User-Agent: big.brother
Disallow: /

User-Agent: Bigfoot
Disallow: /

User-Agent: Bilgi
Disallow: /

User-Agent: Bison
Disallow: /

User-Agent: Bitacle
Disallow: /

User-Agent: Biz360
Disallow: /

User-Agent: Black
Disallow: /

User-Agent: Black Hole
Disallow: /

User-Agent: Black.Hole
Disallow: /

User-Agent: BlackWidow 
Disallow: /

User-Agent: BlackWidow
Disallow: /

User-Agent: bladder.fusion
Disallow: /

User-Agent: Blaiz
Disallow: /

User-Agent: Blog.Checker
Disallow: /

User-Agent: Blogl
Disallow: /

User-Agent: BlogPeople
Disallow: /

User-Agent: Blogshares.Spiders
Disallow: /

User-Agent: Bloodhound
Disallow: /

User-Agent: Blow
Disallow: /

User-Agent: BlowFish/1.0
Disallow: /

User-Agent: bmclient
Disallow: /

User-Agent: Board
Disallow: /

User-Agent: boardreader
Disallow: /

User-Agent: BOI
Disallow: /

User-Agent: boitho
Disallow: /

User-Agent: Bolt\ 0
Disallow: /

User-Agent: Bond
Disallow: /

User-Agent: Bookmark.search.tool
Disallow: /

User-Agent: boris
Disallow: /

User-Agent: Bost
Disallow: /

User-Agent: Boston.Project
Disallow: /

User-Agent: Bot.mailto:craftbot@yahoo.com
Disallow: /

User-Agent: BotALot
Disallow: /

User-Agent: botpaidtoclick
Disallow: /

User-Agent: BotRightHere
Disallow: /

User-Agent: botw
Disallow: /

User-Agent: brandwatch
Disallow: /

User-Agent: BravoBrian
Disallow: /

User-Agent: Brok
Disallow: /

User-Agent: Bropwers
Disallow: /

User-Agent: Broth
Disallow: /

User-Agent: browseabit
Disallow: /

User-Agent: BrowseX
Disallow: /

User-Agent: Browsezilla
Disallow: /

User-Agent: Bruin
Disallow: /

User-Agent: bsalsa
Disallow: /

User-Agent: Buddy
Disallow: /

User-Agent: Build
Disallow: /

User-Agent: Built
Disallow: /

User-Agent: BuiltBotTough
Disallow: /

User-Agent: Bulls
Disallow: /

User-Agent: Bullseye/1.0
Disallow: /

User-Agent: bumblebee
Disallow: /

User-Agent: Bunny
Disallow: /

User-Agent: BunnySlippers
Disallow: /

User-Agent: Busca
Disallow: /

User-Agent: Busi
Disallow: /

User-Agent: butterfly
Disallow: /

User-Agent: Buy
Disallow: /

User-Agent: bwh3
Disallow: /

User-Agent: c\-spider
Disallow: /

User-Agent: CafeK
Disallow: /

User-Agent: Cafi
Disallow: /

User-Agent: camel
Disallow: /

User-Agent: Cand
Disallow: /

User-Agent: captu
Disallow: /

User-Agent: Catch
Disallow: /

User-Agent: CazoodleBot
Disallow: /

User-Agent: CC-rget/5.818
Disallow: /

User-Agent: cd34
Disallow: /

User-Agent: Ceg
Disallow: /

User-Agent: Cegbfeieh
Disallow: /

User-Agent: CFNetwork
Disallow: /

User-Agent: cgichk
Disallow: /

User-Agent: Cha0s
Disallow: /

User-Agent: Chang
Disallow: /

User-Agent: chaos
Disallow: /

User-Agent: Char
Disallow: /

User-Agent: char$32\,35$
Disallow: /

User-Agent: charlotte
Disallow: /

User-Agent: CheeseBot
Disallow: /

User-Agent: Chek
Disallow: /

User-Agent: CherryPicker
Disallow: /

User-Agent: CherryPickerElite/1.0
Disallow: /

User-Agent: CherryPickerSE/1.0
Disallow: /

User-Agent: chill
Disallow: /

User-Agent: ChinaClaw
Disallow: /

User-Agent: CICC
Disallow: /

User-Agent: Cisco
Disallow: /

User-Agent: Cita
Disallow: /

User-Agent: Clam
Disallow: /

User-Agent: Claw
Disallow: /

User-Agent: Click.Bot
Disallow: /

User-Agent: clipping
Disallow: /

User-Agent: clshttp
Disallow: /

User-Agent: Clush
Disallow: /

User-Agent: COAST
Disallow: /

User-Agent: ColdFusion
Disallow: /

User-Agent: Coll
Disallow: /

User-Agent: Comb
Disallow: /

User-Agent: commentreader
Disallow: /

User-Agent: Compan
Disallow: /

User-Agent: Conc
Disallow: /

User-Agent: contact
Disallow: /

User-Agent: Control
Disallow: /

User-Agent: contype
Disallow: /

User-Agent: Conv
Disallow: /

User-Agent: Copernic
Disallow: /

User-Agent: Copi
Disallow: /

User-Agent: Copy
Disallow: /

User-Agent: CopyRightCheck
Disallow: /

User-Agent: Coral
Disallow: /

User-Agent: core-project
Disallow: /

User-Agent: Corn
Disallow: /

User-Agent: cosmos
Disallow: /

User-Agent: costa
Disallow: /

User-Agent: cr4nk
Disallow: /

User-Agent: craft
Disallow: /

User-Agent: crank
Disallow: /

User-Agent: Crap
Disallow: /

User-Agent: Crawler0
Disallow: /

User-Agent: Crazy
Disallow: /

User-Agent: Cres
Disallow: /

User-Agent: Crescent
Disallow: /

User-Agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /

User-Agent: cs\-CZ
Disallow: /

User-Agent: CSHttp
Disallow: /

User-Agent: cuill
Disallow: /

User-Agent: Curl
Disallow: /

User-Agent: Custo
Disallow: /

User-Agent: Cute
Disallow: /

User-Agent: Cyber
Disallow: /

User-Agent: cyberalert
Disallow: /

User-Agent: daoBot
Disallow: /

User-Agent: DARK
Disallow: /

User-Agent: Data
Disallow: /

User-Agent: Daten
Disallow: /

User-Agent: Daum
Disallow: /

User-Agent: dcbot
Disallow: /

User-Agent: dcs
Disallow: /

User-Agent: Deep
Disallow: /

User-Agent: Default\ Browser\ 0
Disallow: /

User-Agent: DepS
Disallow: /

User-Agent: Detect
Disallow: /

User-Agent: Deweb
Disallow: /

User-Agent: Diam
Disallow: /

User-Agent: Digger
Disallow: /

User-Agent: Digimarc
Disallow: /

User-Agent: digout4uagent
Disallow: /

User-Agent: DIIbot
Disallow: /

User-Agent: Dillo
Disallow: /

User-Agent: Ding
Disallow: /

User-Agent: DISC
Disallow: /

User-Agent: DISCo
Disallow: /

User-Agent: discobot
Disallow: /

User-Agent: discobot/1.0
Disallow: /

User-Agent: Disp
Disallow: /

User-Agent: Ditto
Disallow: /

User-Agent: DittoSpyder
Disallow: /

User-Agent: DLC
Disallow: /

User-Agent: DnloadMage
Disallow: /

User-Agent: DNS-Digger/1.0
Disallow: /

User-Agent: DNS-Digger-Explorer/1.0
Disallow: /

User-Agent: DotBot
Disallow: /

User-Agent: Doubanbot
Disallow: /

User-Agent: Download
Disallow: /

User-Agent: Download.Demon
Disallow: /

User-Agent: Download.Devil
Disallow: /

User-Agent: Download.Wonder
Disallow: /

User-Agent: Download\ Demon
Disallow: /

User-Agent: Downloader
Disallow: /

User-Agent: drag
Disallow: /

User-Agent: DreamPassport
Disallow: /

User-Agent: Drec
Disallow: /

User-Agent: Drip
Disallow: /

User-Agent: dsdl
Disallow: /

User-Agent: dsok
Disallow: /

User-Agent: DSurf
Disallow: /

User-Agent: DTAAgent
Disallow: /

User-Agent: DTS
Disallow: /

User-Agent: Dual
Disallow: /

User-Agent: dumb
Disallow: /

User-Agent: DynaWeb
Disallow: /

User-Agent: e\-collector
Disallow: /

User-Agent: eag
Disallow: /

User-Agent: earn
Disallow: /

User-Agent: EARTHCOM
Disallow: /

User-Agent: EasyDL
Disallow: /

User-Agent: ebin
Disallow: /

User-Agent: EBM-APPLE
Disallow: /

User-Agent: EBrowse
Disallow: /

User-Agent: eCatch
Disallow: /

User-Agent: echo
Disallow: /

User-Agent: ecollector
Disallow: /

User-Agent: ecxi
Disallow: /

User-Agent: Edco
Disallow: /

User-Agent: edgeio
Disallow: /

User-Agent: efp\@gmx\.net
Disallow: /

User-Agent: EirGrabber
Disallow: /

User-Agent: email
Disallow: /

User-Agent: Email.Extractor
Disallow: /

User-Agent: EmailCollector
Disallow: /

User-Agent: EmailSearch
Disallow: /

User-Agent: EmailSiphon
Disallow: /

User-Agent: EmailWolf
Disallow: /

User-Agent: Emer
Disallow: /

User-Agent: empas
Disallow: /

User-Agent: Enfi
Disallow: /

User-Agent: Enhan
Disallow: /

User-Agent: Enterprise\_Search
Disallow: /

User-Agent: envolk
Disallow: /

User-Agent: erck
Disallow: /

User-Agent: EroCr
Disallow: /

User-Agent: EroCrawler
Disallow: /

User-Agent: ESurf
Disallow: /

User-Agent: Eval
Disallow: /

User-Agent: Evere
Disallow: /

User-Agent: Evil
Disallow: /

User-Agent: EWH
Disallow: /

User-Agent: Exabot
Disallow: /

User-Agent: Exabot/2.0
Disallow: /

User-Agent: Exact
Disallow: /

User-Agent: EXPLOITER
Disallow: /

User-Agent: Expre
Disallow: /

User-Agent: Express\ WebPictures
Disallow: /

User-Agent: Extra
Disallow: /

User-Agent: ExtractorPro
Disallow: /

User-Agent: EyeN
Disallow: /

User-Agent: EyeNetIE
Disallow: /

User-Agent: ezooms
Disallow: /

User-Agent: FairAd
Disallow: /

User-Agent: Fake
Disallow: /

User-Agent: FANG
Disallow: /

User-Agent: FAST
Disallow: /

User-Agent: fastlwspider
Disallow: /

User-Agent: FavOrg
Disallow: /

User-Agent: Favorites.Sweeper
Disallow: /

User-Agent: Faxo
Disallow: /

User-Agent: FDM\_1
Disallow: /

User-Agent: FDSE
Disallow: /

User-Agent: fetch
Disallow: /

User-Agent: FEZhead
Disallow: /

User-Agent: Filan
Disallow: /

User-Agent: FileHound
Disallow: /

User-Agent: find
Disallow: /

User-Agent: Firebat
Disallow: /

User-Agent: Firefox.2\.0
Disallow: /

User-Agent: Firs
Disallow: /

User-Agent: Flam
Disallow: /

User-Agent: Flash
Disallow: /

User-Agent: FlashGet
Disallow: /

User-Agent: FlickBot
Disallow: /

User-Agent: Flip
Disallow: /

User-Agent: fluffy
Disallow: /

User-Agent: flunky
Disallow: /

User-Agent: focus
Disallow: /

User-Agent: Foob
Disallow: /

User-Agent: Foobot
Disallow: /

User-Agent: Fooky
Disallow: /

User-Agent: Forex
Disallow: /

User-Agent: Forum
Disallow: /

User-Agent: ForV
Disallow: /

User-Agent: Fost
Disallow: /

User-Agent: Foto
Disallow: /

User-Agent: Foun
Disallow: /

User-Agent: Franklin.Locator
Disallow: /

User-Agent: freefind
Disallow: /

User-Agent: FreshDownload
Disallow: /

User-Agent: FrontPage
Disallow: /

User-Agent: FSurf
Disallow: /

User-Agent: Fuck
Disallow: /

User-Agent: Fuer
Disallow: /

User-Agent: futile
Disallow: /

User-Agent: Fyber
Disallow: /

User-Agent: Gais
Disallow: /

User-Agent: GalaxyBot
Disallow: /

User-Agent: Galbot
Disallow: /

User-Agent: Gamespy\_Arcade
Disallow: /

User-Agent: GbPl
Disallow: /

User-Agent: Gener
Disallow: /

User-Agent: geni
Disallow: /

User-Agent: Geona
Disallow: /

User-Agent: Get
Disallow: /

User-Agent: GetRight
Disallow: /

User-Agent: GetWeb!
Disallow: /

User-Agent: gigabaz
Disallow: /

User-Agent: Ginxbot
Disallow: /

User-Agent: Gira
Disallow: /

User-Agent: gluc
Disallow: /

User-Agent: glx.?v
Disallow: /

User-Agent: gnome
Disallow: /

User-Agent: Go!Zilla
Disallow: /

User-Agent: Go.Zilla
Disallow: /

User-Agent: Go-Ahead-Got-It
Disallow: /

User-Agent: GOFORIT
Disallow: /

User-Agent: Goldfire
Disallow: /

User-Agent: gonzo
Disallow: /

User-Agent: GornKer
Disallow: /

User-Agent: GoSearch
Disallow: /

User-Agent: Got\-It
Disallow: /

User-Agent: gozilla
Disallow: /

User-Agent: grab
Disallow: /

User-Agent: Grabber
Disallow: /

User-Agent: GrabNet
Disallow: /

User-Agent: Graf
Disallow: /

User-Agent: Grafula
Disallow: /

User-Agent: Green.Research
Disallow: /

User-Agent: Grub
Disallow: /

User-Agent: grub\-client
Disallow: /

User-Agent: Grup
Disallow: /

User-Agent: gsa\-cra
Disallow: /

User-Agent: GSearch
Disallow: /

User-Agent: GT::WWW
Disallow: /

User-Agent: GT\:\:WWW
Disallow: /

User-Agent: GuideBot
Disallow: /

User-Agent: guruji
Disallow: /

User-Agent: gvfs
Disallow: /

User-Agent: Gyps
Disallow: /

User-Agent: hack
Disallow: /

User-Agent: haha
Disallow: /

User-Agent: hailo
Disallow: /

User-Agent: Harv
Disallow: /

User-Agent: Harvest/1.5
Disallow: /

User-Agent: Hatena
Disallow: /

User-Agent: Hax
Disallow: /

User-Agent: Head
Disallow: /

User-Agent: Helm
Disallow: /

User-Agent: herit
Disallow: /

User-Agent: heritrix
Disallow: /

User-Agent: hgre
Disallow: /

User-Agent: hhjhj\@yahoo
Disallow: /

User-Agent: Hippo
Disallow: /

User-Agent: hloader
Disallow: /

User-Agent: HMSE
Disallow: /

User-Agent: HMView
Disallow: /

User-Agent: holm
Disallow: /

User-Agent: holy
Disallow: /

User-Agent: HomePageSearch
Disallow: /

User-Agent: HooWWWer
Disallow: /

User-Agent: HouxouCrawler
Disallow: /

User-Agent: HPPrint
Disallow: /

User-Agent: htdig
Disallow: /

User-Agent: http.generic
Disallow: /

User-Agent: HTTP::Lite
Disallow: /

User-Agent: HTTPConnect
Disallow: /

User-Agent: httpdown
Disallow: /

User-Agent: HTTPGet
Disallow: /

User-Agent: httplib
Disallow: /

User-Agent: HTTPRetriever
Disallow: /

User-Agent: HTTrack
Disallow: /

User-Agent: HuaweiSymantecSpider
Disallow: /

User-Agent: human
Disallow: /

User-Agent: humanlinks
Disallow: /

User-Agent: Huron
Disallow: /

User-Agent: hverify
Disallow: /

User-Agent: Hybrid
Disallow: /

User-Agent: Hyper
Disallow: /

User-Agent: ia\_archiver
Disallow: /

User-Agent: ia_archiver
Disallow: /

User-Agent: iaskspi
Disallow: /

User-Agent: IBM\_Planetwide
Disallow: /

User-Agent: iCCra
Disallow: /

User-Agent: ichiro
Disallow: /

User-Agent: ID\-Search
Disallow: /

User-Agent: IDA
Disallow: /

User-Agent: IDBot
Disallow: /

User-Agent: id-search
Disallow: /

User-Agent: id-search\.org
Disallow: /

User-Agent: IEAuto
Disallow: /

User-Agent: IEMPT
Disallow: /

User-Agent: iexplore\.exe
Disallow: /

User-Agent: iGetter
Disallow: /

User-Agent: iimds\_monitor
Disallow: /

User-Agent: Ilse
Disallow: /

User-Agent: Iltrov
Disallow: /

User-Agent: Image
Disallow: /

User-Agent: Image.Stripper
Disallow: /

User-Agent: Image.Sucker
Disallow: /

User-Agent: Image\ Stripper
Disallow: /

User-Agent: Image\ Sucker
Disallow: /

User-Agent: imagefetch
Disallow: /

User-Agent: Incutio
Disallow: /

User-Agent: IncyWincy
Disallow: /

User-Agent: Indexer
Disallow: /

User-Agent: Industry.Program
Disallow: /

User-Agent: Indy
Disallow: /

User-Agent: Indy\ Library
Disallow: /

User-Agent: InetURL
Disallow: /

User-Agent: InfoNav
Disallow: /

User-Agent: InfoNaviRobot
Disallow: /

User-Agent: informant
Disallow: /

User-Agent: InfoTekies
Disallow: /

User-Agent: Ingelin
Disallow: /

User-Agent: Innerpr
Disallow: /

User-Agent: Inspect
Disallow: /

User-Agent: InstallShield.DigitalWizard
Disallow: /

User-Agent: Insuran\.
Disallow: /

User-Agent: Intellig
Disallow: /

User-Agent: Intelliseek
Disallow: /

User-Agent: InterGET
Disallow: /

User-Agent: Internet.Ninja
Disallow: /

User-Agent: Internet.x
Disallow: /

User-Agent: Internet\ Ninja
Disallow: /

User-Agent: Internet\_Explorer
Disallow: /

User-Agent: InternetLinkagent
Disallow: /

User-Agent: InternetSeer.com
Disallow: /

User-Agent: InternetSeer\.com
Disallow: /

User-Agent: Intraf
Disallow: /

User-Agent: IP2
Disallow: /

User-Agent: Ipsel
Disallow: /

User-Agent: Iria
Disallow: /

User-Agent: IRLbot
Disallow: /

User-Agent: Iron33
Disallow: /

User-Agent: Irvine
Disallow: /

User-Agent: ISC\ Systems\ iRc\ Search\ 2\.1
Disallow: /

User-Agent: ISC\_Sys
Disallow: /

User-Agent: iSilo
Disallow: /

User-Agent: ISRCCrawler
Disallow: /

User-Agent: ISSpi
Disallow: /

User-Agent: IUPUI.Research.Bot
Disallow: /

User-Agent: Jady
Disallow: /

User-Agent: Jaka
Disallow: /

User-Agent: jakarta
Disallow: /

User-Agent: Jakarta Commons-HttpClient/3.0
Disallow: /

User-Agent: Jam
Disallow: /

User-Agent: Java
Disallow: /

User-Agent: Java$tm$
Disallow: /

User-Agent: java\/
Disallow: /

User-Agent: JBH.agent
Disallow: /

User-Agent: Jenny
Disallow: /

User-Agent: JennyBot
Disallow: /

User-Agent: JetB
Disallow: /

User-Agent: JetC
Disallow: /

User-Agent: JetCar
Disallow: /

User-Agent: jeteye
Disallow: /

User-Agent: jikespider
Disallow: /

User-Agent: jiro
Disallow: /

User-Agent: JoBo
Disallow: /

User-Agent: JOC
Disallow: /

User-Agent: JOC\ Web\ Spider
Disallow: /

User-Agent: jupit
Disallow: /

User-Agent: Just
Disallow: /

User-Agent: Jyx
Disallow: /

User-Agent: Kapere
Disallow: /

User-Agent: kash
Disallow: /

User-Agent: Kazo
Disallow: /

User-Agent: KBee
Disallow: /

User-Agent: Kenjin
Disallow: /

User-Agent: Kenjin Spider
Disallow: /

User-Agent: Kernel
Disallow: /

User-Agent: Keywo
Disallow: /

User-Agent: Keyword Density/0.9
Disallow: /

User-Agent: KFSW
Disallow: /

User-Agent: KKma
Disallow: /

User-Agent: Know
Disallow: /

User-Agent: kosmix
Disallow: /

User-Agent: KRAE
Disallow: /

User-Agent: KRetrieve
Disallow: /

User-Agent: Krug
Disallow: /

User-Agent: ksibot
Disallow: /

User-Agent: ksoap
Disallow: /

User-Agent: Kum
Disallow: /

User-Agent: KWebGet
Disallow: /

User-Agent: Lachesis
Disallow: /

User-Agent: lanshan
Disallow: /

User-Agent: Lapo
Disallow: /

User-Agent: larbin
Disallow: /

User-Agent: leacher
Disallow: /

User-Agent: leech
Disallow: /

User-Agent: LeechFTP
Disallow: /

User-Agent: LeechGet
Disallow: /

User-Agent: leipzig\.de
Disallow: /

User-Agent: Lets
Disallow: /

User-Agent: Lexi
Disallow: /

User-Agent: LexiBot
Disallow: /

User-Agent: lftp
Disallow: /

User-Agent: Libby
Disallow: /

User-Agent: libcrawl
Disallow: /

User-Agent: libcurl
Disallow: /

User-Agent: libfetch
Disallow: /

User-Agent: libghttp
Disallow: /

User-Agent: libWeb
Disallow: /

User-Agent: libWeb/clsHTTP
Disallow: /

User-Agent: libwhisker
Disallow: /

User-Agent: libwww
Disallow: /

User-Agent: libwww\-FM
Disallow: /

User-Agent: libwww\-perl
Disallow: /

User-Agent: libwww-perl
Disallow: /

User-Agent: libwww-perl/5.813
Disallow: /

User-Agent: LightningDownload
Disallow: /

User-Agent: likse
Disallow: /

User-Agent: Linc
Disallow: /

User-Agent: Link
Disallow: /

User-Agent: Link.Sleuth
Disallow: /

User-Agent: LinkextractorPro
Disallow: /

User-Agent: Linkie
Disallow: /

User-Agent: LINKS.ARoMATIZED
Disallow: /

User-Agent: LinkScan
Disallow: /

User-Agent: LinkScan/8.1a Unix
Disallow: /

User-Agent: LinksManager.com_bot
Disallow: /

User-Agent: linktiger
Disallow: /

User-Agent: linkwalker
Disallow: /

User-Agent: Lint
Disallow: /

User-Agent: List
Disallow: /

User-Agent: lmcrawler
Disallow: /

User-Agent: LMQ
Disallow: /

User-Agent: LNSpiderguy
Disallow: /

User-Agent: loader
Disallow: /

User-Agent: LocalcomBot
Disallow: /

User-Agent: Locu
Disallow: /

User-Agent: London
Disallow: /

User-Agent: lone
Disallow: /

User-Agent: looksmart
Disallow: /

User-Agent: loop
Disallow: /

User-Agent: Lork
Disallow: /

User-Agent: LTH\_
Disallow: /

User-Agent: LWP
Disallow: /

User-Agent: lwp\-request
Disallow: /

User-Agent: lwp-request
Disallow: /

User-Agent: lwp-trivial
Disallow: /

User-Agent: lwp-trivial/1.34
Disallow: /

User-Agent: Mac.Finder
Disallow: /

User-Agent: Mac\_F
Disallow: /

User-Agent: Macintosh\;.I\;.PPC
Disallow: /

User-Agent: Mag\-Net
Disallow: /

User-Agent: magi
Disallow: /

User-Agent: Magnet
Disallow: /

User-Agent: Magp
Disallow: /

User-Agent: magpie-crawler 1.1
Disallow: /

User-Agent: Mail.Sweeper
Disallow: /

User-Agent: main
Disallow: /

User-Agent: majest
Disallow: /

User-Agent: Mam
Disallow: /

User-Agent: Mana
Disallow: /

User-Agent: MarcoPolo
Disallow: /

User-Agent: mark.blonin
Disallow: /

User-Agent: MarkWatch
Disallow: /

User-Agent: MaSagool
Disallow: /

User-Agent: Mass
Disallow: /

User-Agent: Mass.Downloader
Disallow: /

User-Agent: Mass\ Downloader
Disallow: /

User-Agent: Mata
Disallow: /

User-Agent: Mata Hari
Disallow: /

User-Agent: mavi
Disallow: /

User-Agent: Maxthon$
Disallow: /

User-Agent: McBot
Disallow: /

User-Agent: MCspider
Disallow: /

User-Agent: Mecha
Disallow: /

User-Agent: mediapartners
Disallow: /

User-Agent: MEGAUPLOAD
Disallow: /

User-Agent: MetaProducts.Download.Express
Disallow: /

User-Agent: Metaspin
Disallow: /

User-Agent: Mete
Disallow: /

User-Agent: MFC_Tear_Sample
Disallow: /

User-Agent: Microsoft URL Control - 5.01.4511
Disallow: /

User-Agent: Microsoft URL Control - 6.00.8169
Disallow: /

User-Agent: Microsoft.Data.Access
Disallow: /

User-Agent: Microsoft.URL
Disallow: /

User-Agent: Microsoft\ URL\ Control
Disallow: /

User-Agent: microsoft\.url
Disallow: /

User-Agent: Microsoft\_Internet\_Explorer
Disallow: /

User-Agent: MIDo
Disallow: /

User-Agent: MIDown\ tool
Disallow: /

User-Agent: MIIx
Disallow: /

User-Agent: MIIxpc
Disallow: /

User-Agent: MIIxpc/4.2
Disallow: /

User-Agent: miner
Disallow: /

User-Agent: Mira
Disallow: /

User-Agent: MIRE
Disallow: /

User-Agent: Mirror
Disallow: /

User-Agent: Miss
Disallow: /

User-Agent: Missauga
Disallow: /

User-Agent: Missigua.Locator
Disallow: /

User-Agent: Missigua\ Locator
Disallow: /

User-Agent: Missouri.College.Browse
Disallow: /

User-Agent: Mist
Disallow: /

User-Agent: Mister PiX
Disallow: /

User-Agent: Mister\ PiX
Disallow: /

User-Agent: Mizz
Disallow: /

User-Agent: MJ12
Disallow: /

User-Agent: MJ12bot
Disallow: /

User-Agent: MJ12bot/v1.2.0
Disallow: /

User-Agent: MJ12bot/v1.2.5
Disallow: /

User-Agent: mkdb
Disallow: /

User-Agent: MLBot
Disallow: /

User-Agent: MLM
Disallow: /

User-Agent: MMMoCrawl
Disallow: /

User-Agent: MnoG
Disallow: /

User-Agent: moge
Disallow: /

User-Agent: moget
Disallow: /

User-Agent: moget/2.1
Disallow: /

User-Agent: Moje
Disallow: /

User-Agent: Monster
Disallow: /

User-Agent: Monza.Browser
Disallow: /

User-Agent: Mooz
Disallow: /

User-Agent: Moreoverbot
Disallow: /

User-Agent: moreoverbot/5.0
Disallow: /

User-Agent: MOT\-MPx220
Disallow: /

User-Agent: mothra\/netscan
Disallow: /

User-Agent: mouse
Disallow: /

User-Agent: MovableType
Disallow: /

User-Agent: Mozdex
Disallow: /

User-Agent: Mozi\!
Disallow: /

User-Agent: mozilla/4
Disallow: /

User-Agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT)
Disallow: /

User-Agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP)
Disallow: /

User-Agent: mozilla/5
Disallow: /

User-Agent: Mozilla\.*Indy
Disallow: /

User-Agent: Mozilla\.*NEWT
Disallow: /

User-Agent: Mozilla\/1\.22
Disallow: /

User-Agent: Mozilla\/22
Disallow: /

User-Agent: Mozilla\/3\.Mozilla\/2\.01
Disallow: /

User-Agent: Mozilla\/4\.0\(compatible
Disallow: /

User-Agent: Mozilla\/4\.08
Disallow: /

User-Agent: Mozilla\/4\.61.\(Macintosh
Disallow: /

User-Agent: Mozilla\/5\.0
Disallow: /

User-Agent: Mozilla\/7\.0
Disallow: /

User-Agent: Mozilla\/8
Disallow: /

User-Agent: Mozilla\/9
Disallow: /

User-Agent: Mozilla\/Firefox
Disallow: /

User-Agent: Mozilla\:
Disallow: /

User-Agent: Mp3Bot
Disallow: /

User-Agent: MPF
Disallow: /

User-Agent: MRA
Disallow: /

User-Agent: MS.?Search
Disallow: /

User-Agent: MS.FrontPage
Disallow: /

User-Agent: MSFrontPage
Disallow: /

User-Agent: MSIE\_6\.0
Disallow: /

User-Agent: MSIE6
Disallow: /

User-Agent: MSIECrawler
Disallow: /

User-Agent: msnbot\-media
Disallow: /

User-Agent: msnbot\-Products
Disallow: /

User-Agent: MSNPTC
Disallow: /

User-Agent: MSProxy
Disallow: /

User-Agent: MSRBOT
Disallow: /

User-Agent: multithreaddb
Disallow: /

User-Agent: musc
Disallow: /

User-Agent: MVAC
Disallow: /

User-Agent: MWM
Disallow: /

User-Agent: My\_age
Disallow: /

User-Agent: MyApp
Disallow: /

User-Agent: MyDog
Disallow: /

User-Agent: MyEng
Disallow: /

User-Agent: MyFamilyBot
Disallow: /

User-Agent: MyGetRight
Disallow: /

User-Agent: MyIE2
Disallow: /

User-Agent: mysearch
Disallow: /

User-Agent: myurl
Disallow: /

User-Agent: NAG
Disallow: /

User-Agent: NAMEPROTECT
Disallow: /

User-Agent: NASA.Search
Disallow: /

User-Agent: nationaldirectory
Disallow: /

User-Agent: Naver
Disallow: /

User-Agent: Navr
Disallow: /

User-Agent: Navroad
Disallow: /

User-Agent: Near
Disallow: /

User-Agent: NearSite
Disallow: /

User-Agent: Net.Vampire
Disallow: /

User-Agent: Net\ Vampire
Disallow: /

User-Agent: NetAnts
Disallow: /

User-Agent: netattache
Disallow: /

User-Agent: Netcach
Disallow: /

User-Agent: NetCarta
Disallow: /

User-Agent: Netcraft
Disallow: /

User-Agent: NetCrawl
Disallow: /

User-Agent: NetMech
Disallow: /

User-Agent: netprospector
Disallow: /

User-Agent: NetResearchServer
Disallow: /

User-Agent: NetSeer crawler/2.0
Disallow: /

User-Agent: netseer/0.1
Disallow: /

User-Agent: NetSeer/Nutch-0.9
Disallow: /

User-Agent: NetSp
Disallow: /

User-Agent: NetSpider
Disallow: /

User-Agent: netX
Disallow: /

User-Agent: NetZ
Disallow: /

User-Agent: NetZIP
Disallow: /

User-Agent: Neut
Disallow: /

User-Agent: newLISP
Disallow: /

User-Agent: NewsGatorInbox
Disallow: /

User-Agent: NEWT
Disallow: /

User-Agent: NEWT.ActiveX
Disallow: /

User-Agent: Next
Disallow: /

User-Agent: NICE
Disallow: /

User-Agent: NICErsPRO
Disallow: /

User-Agent: nikto
Disallow: /

User-Agent: Nimb
Disallow: /

User-Agent: Ninja
Disallow: /

User-Agent: Ninte
Disallow: /

User-Agent: NIPGCrawler
Disallow: /

User-Agent: Noga
Disallow: /

User-Agent: nogo
Disallow: /

User-Agent: Noko
Disallow: /

User-Agent: Nomad
Disallow: /

User-Agent: Norb
Disallow: /

User-Agent: noxtrumbot
Disallow: /

User-Agent: NPbot
Disallow: /

User-Agent: NuSe
Disallow: /

User-Agent: Nutch
Disallow: /

User-Agent: Nutex
Disallow: /

User-Agent: NWSp
Disallow: /

User-Agent: Obje
Disallow: /

User-Agent: Ocel
Disallow: /

User-Agent: Octo
Disallow: /

User-Agent: Octopus
Disallow: /

User-Agent: ODI3
Disallow: /

User-Agent: oegp
Disallow: /

User-Agent: Offline
Disallow: /

User-Agent: Offline Explorer
Disallow: /

User-Agent: Offline.Explorer
Disallow: /

User-Agent: Offline.Navigator
Disallow: /

User-Agent: Offline\ Explorer
Disallow: /

User-Agent: Offline\ Navigator
Disallow: /

User-Agent: OK.Mozilla
Disallow: /

User-Agent: omg
Disallow: /

User-Agent: Omni
Disallow: /

User-Agent: omniexplorer_bot
Disallow: /

User-Agent: OmniExplorer_Bot/6.11.1
Disallow: /

User-Agent: Onfo
Disallow: /

User-Agent: onyx
Disallow: /

User-Agent: OpaL
Disallow: /

User-Agent: OpenBot
Disallow: /

User-Agent: Openf
Disallow: /

User-Agent: Openfind
Disallow: /

User-Agent: Openfind data gathere
Disallow: /

User-Agent: OpenTextSiteCrawler
Disallow: /

User-Agent: OpenU
Disallow: /

User-Agent: Orac
Disallow: /

User-Agent: OrangeBot
Disallow: /

User-Agent: Orbit
Disallow: /

User-Agent: Oreg
Disallow: /

User-Agent: osis
Disallow: /

User-Agent: Outf
Disallow: /

User-Agent: Owl
Disallow: /

User-Agent: P3P
Disallow: /

User-Agent: PackRat
Disallow: /

User-Agent: PageGrabber
Disallow: /

User-Agent: PagmIEDownload
Disallow: /

User-Agent: pansci
Disallow: /

User-Agent: panscient.com
Disallow: /

User-Agent: Papa
Disallow: /

User-Agent: Papa\ Foto
Disallow: /

User-Agent: PaperLiBot
Disallow: /

User-Agent: Pars
Disallow: /

User-Agent: Patw
Disallow: /

User-Agent: pavu
Disallow: /

User-Agent: pavuk
Disallow: /

User-Agent: Pb2Pb
Disallow: /

User-Agent: pcBrow
Disallow: /

User-Agent: pcBrowser
Disallow: /

User-Agent: PEAR
Disallow: /

User-Agent: PECL
Disallow: /

User-Agent: PECL::HTTP
Disallow: /

User-Agent: PEER
Disallow: /

User-Agent: PeoplePal
Disallow: /

User-Agent: pepe
Disallow: /

User-Agent: Perl
Disallow: /

User-Agent: PerMan
Disallow: /

User-Agent: PersonaPilot
Disallow: /

User-Agent: Persuader
Disallow: /

User-Agent: petit
Disallow: /

User-Agent: PHP
Disallow: /

User-Agent: PHP.vers
Disallow: /

User-Agent: PHPCrawl
Disallow: /

User-Agent: PHPot
Disallow: /

User-Agent: Phras
Disallow: /

User-Agent: PicaLo
Disallow: /

User-Agent: Piff
Disallow: /

User-Agent: Pige
Disallow: /

User-Agent: pigs
Disallow: /

User-Agent: Ping
Disallow: /

User-Agent: PingALink
Disallow: /

User-Agent: Pingd
Disallow: /

User-Agent: Pipe
Disallow: /

User-Agent: Plag
Disallow: /

User-Agent: Plant
Disallow: /

User-Agent: playstarmusic
Disallow: /

User-Agent: PleaseCrawl
Disallow: /

User-Agent: Pluck
Disallow: /

User-Agent: Pockey
Disallow: /

User-Agent: POE\-Com
Disallow: /

User-Agent: Poirot
Disallow: /

User-Agent: Pomp
Disallow: /

User-Agent: Port.Huron
Disallow: /

User-Agent: Post
Disallow: /

User-Agent: powerset
Disallow: /

User-Agent: Preload
Disallow: /

User-Agent: press
Disallow: /

User-Agent: Privoxy
Disallow: /

User-Agent: Probe
Disallow: /

User-Agent: Program.Shareware
Disallow: /

User-Agent: Progressive.Download
Disallow: /

User-Agent: ProPowerBot
Disallow: /

User-Agent: ProPowerBot/2.14
Disallow: /

User-Agent: prospector
Disallow: /

User-Agent: Provider.Protocol.Discover
Disallow: /

User-Agent: ProWebWalker
Disallow: /

User-Agent: Prowl
Disallow: /

User-Agent: Proxy
Disallow: /

User-Agent: Prozilla
Disallow: /

User-Agent: psbot
Disallow: /

User-Agent: psbot/0.1
Disallow: /

User-Agent: PSurf
Disallow: /

User-Agent: psycheclone
Disallow: /

User-Agent: Pulse
Disallow: /

User-Agent: Pump
Disallow: /

User-Agent: PushSite
Disallow: /

User-Agent: PussyCat
Disallow: /

User-Agent: PuxaRapido
Disallow: /

User-Agent: PycURL
Disallow: /

User-Agent: PyQ
Disallow: /

User-Agent: Pyth
Disallow: /

User-Agent: QRVA
Disallow: /

User-Agent: QuepasaCreep
Disallow: /

User-Agent: Query
Disallow: /

User-Agent: QueryN Metasearch
Disallow: /

User-Agent: Quest
Disallow: /

User-Agent: Qweer
Disallow: /

User-Agent: R6_CommentReader
Disallow: /

User-Agent: R6_FeedFetcher
Disallow: /

User-Agent: radian
Disallow: /

User-Agent: radian6
Disallow: /

User-Agent: Radiation
Disallow: /

User-Agent: Rambler
Disallow: /

User-Agent: RAMP
Disallow: /

User-Agent: RealDownload
Disallow: /

User-Agent: Reap
Disallow: /

User-Agent: Recorder
Disallow: /

User-Agent: RedCarpet
Disallow: /

User-Agent: RedKernel
Disallow: /

User-Agent: ReGet
Disallow: /

User-Agent: relevantnoise
Disallow: /

User-Agent: replacer
Disallow: /

User-Agent: Repo
Disallow: /

User-Agent: RepoMonkey
Disallow: /

User-Agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /

User-Agent: requ
Disallow: /

User-Agent: Rese
Disallow: /

User-Agent: Retrieve
Disallow: /

User-Agent: Rip
Disallow: /

User-Agent: Rippers\ 0
Disallow: /

User-Agent: Rix
Disallow: /

User-Agent: RMA
Disallow: /

User-Agent: Roboz
Disallow: /

User-Agent: Rogue
Disallow: /

User-Agent: Rover
Disallow: /

User-Agent: RPT\-HTTP
Disallow: /

User-Agent: Rsync
Disallow: /

User-Agent: RTG30
Disallow: /

User-Agent: ruby
Disallow: /

User-Agent: Rufus
Disallow: /

User-Agent: Salt
Disallow: /

User-Agent: Sample
Disallow: /

User-Agent: SAPO
Disallow: /

User-Agent: Sauger
Disallow: /

User-Agent: savvy
Disallow: /

User-Agent: SBIder
Disallow: /

User-Agent: SBP
Disallow: /

User-Agent: SCAgent
Disallow: /

User-Agent: scan
Disallow: /

User-Agent: SCEJ\_
Disallow: /

User-Agent: Sched
Disallow: /

User-Agent: Schizo
Disallow: /

User-Agent: Schlong
Disallow: /

User-Agent: Schmo
Disallow: /

User-Agent: Scooter
Disallow: /

User-Agent: Scorp
Disallow: /

User-Agent: Scout
Disallow: /

User-Agent: ScoutJet
Disallow: /

User-Agent: ScoutOut
Disallow: /

User-Agent: SCrawl
Disallow: /

User-Agent: screen
Disallow: /

User-Agent: script
Disallow: /

User-Agent: SeaMonkey$
Disallow: /

User-Agent: SearchExpress
Disallow: /

User-Agent: searchhippo
Disallow: /

User-Agent: Searchme
Disallow: /

User-Agent: searchpreview
Disallow: /

User-Agent: searchterms
Disallow: /

User-Agent: Second.Street.Research
Disallow: /

User-Agent: Security.Kol
Disallow: /

User-Agent: Seekbot
Disallow: /

User-Agent: Seeker
Disallow: /

User-Agent: Sega
Disallow: /

User-Agent: SemrushBot/0.9
Disallow: /

User-Agent: Sensis
Disallow: /

User-Agent: Sept
Disallow: /

User-Agent: Serious
Disallow: /

User-Agent: Sezn
Disallow: /

User-Agent: SeznamBot/3.0
Disallow: /

User-Agent: Shai
Disallow: /

User-Agent: Share
Disallow: /

User-Agent: Sharp
Disallow: /

User-Agent: Shaz
Disallow: /

User-Agent: shell
Disallow: /

User-Agent: shelo
Disallow: /

User-Agent: Sherl
Disallow: /

User-Agent: Shim
Disallow: /

User-Agent: Shiretoko
Disallow: /

User-Agent: ShopWiki
Disallow: /

User-Agent: SickleBot
Disallow: /

User-Agent: Simple
Disallow: /

User-Agent: Siph
Disallow: /

User-Agent: Site.Sniper
Disallow: /

User-Agent: SiteBot/0.1
Disallow: /

User-Agent: sitecheck
Disallow: /

User-Agent: sitecheck\.internetseer\.com
Disallow: /

User-Agent: SiteCrawler
Disallow: /

User-Agent: SiteSnagger
Disallow: /

User-Agent: SiteSucker
Disallow: /

User-Agent: sitevigil
Disallow: /

User-Agent: SiteX
Disallow: /

User-Agent: Sleip
Disallow: /

User-Agent: Slide
Disallow: /

User-Agent: Slurpy.Verifier
Disallow: /

User-Agent: Sly
Disallow: /

User-Agent: Smag
Disallow: /

User-Agent: SmartDownload
Disallow: /

User-Agent: Smurf
Disallow: /

User-Agent: sna\-
Disallow: /

User-Agent: snag
Disallow: /

User-Agent: Snake
Disallow: /

User-Agent: Snapbot
Disallow: /

User-Agent: Snip
Disallow: /

User-Agent: Snoop
Disallow: /

User-Agent: Snoopy
Disallow: /

User-Agent: So\-net
Disallow: /

User-Agent: SocialSpider-Finder/0.2
Disallow: /

User-Agent: SocSci
Disallow: /

User-Agent: sogou
Disallow: /

User-Agent: Sogouwebspider/3.0
Disallow: /

User-Agent: Sogouwebspider/4.0
Disallow: /

User-Agent: Sohu
Disallow: /

User-Agent: solr
Disallow: /

User-Agent: sootle
Disallow: /

User-Agent: Soso
Disallow: /

User-Agent: sosospider+
Disallow: /

User-Agent: SpaceBison
Disallow: /

User-Agent: Spad
Disallow: /

User-Agent: Span
Disallow: /

User-Agent: SpankBot
Disallow: /

User-Agent: spanner
Disallow: /

User-Agent: Speed
Disallow: /

User-Agent: Spegla
Disallow: /

User-Agent: Sphere
Disallow: /

User-Agent: Sphider
Disallow: /

User-Agent: spider
Disallow: /

User-Agent: SpiderBot
Disallow: /

User-Agent: SpiderEngine
Disallow: /

User-Agent: SpiderView
Disallow: /

User-Agent: Spin
Disallow: /

User-Agent: Spinn3r
Disallow: /

User-Agent: sproose
Disallow: /

User-Agent: Spurl
Disallow: /

User-Agent: Spyder
Disallow: /

User-Agent: SQ.Webscanner
Disallow: /

User-Agent: Squi
Disallow: /

User-Agent: sqwid
Disallow: /

User-Agent: Sqworm
Disallow: /

User-Agent: SSM\_Ag
Disallow: /

User-Agent: Stack
Disallow: /

User-Agent: Stamina
Disallow: /

User-Agent: stamp
Disallow: /

User-Agent: Stanford
Disallow: /

User-Agent: Statbot
Disallow: /

User-Agent: State
Disallow: /

User-Agent: Steel
Disallow: /

User-Agent: Steeler
Disallow: /

User-Agent: Strateg
Disallow: /

User-Agent: Stress
Disallow: /

User-Agent: Strip
Disallow: /

User-Agent: studybot
Disallow: /

User-Agent: Style
Disallow: /

User-Agent: subot
Disallow: /

User-Agent: Suck
Disallow: /

User-Agent: Sume
Disallow: /

User-Agent: sun4m
Disallow: /

User-Agent: Sunrise
Disallow: /

User-Agent: SuperBot
Disallow: /

User-Agent: SuperBro
Disallow: /

User-Agent: SuperHTTP
Disallow: /

User-Agent: Supervi
Disallow: /

User-Agent: Surf4Me
Disallow: /

User-Agent: Surfbot
Disallow: /

User-Agent: SurfWalker
Disallow: /

User-Agent: Susi
Disallow: /

User-Agent: suza
Disallow: /

User-Agent: suzu
Disallow: /

User-Agent: suzuran
Disallow: /

User-Agent: Sweep
Disallow: /

User-Agent: sygol
Disallow: /

User-Agent: syncrisis
Disallow: /

User-Agent: Systems
Disallow: /

User-Agent: Szukacz
Disallow: /

User-Agent: Szukacz/1.4
Disallow: /

User-Agent: Tagger
Disallow: /

User-Agent: Tagyu
Disallow: /

User-Agent: tAke
Disallow: /

User-Agent: tAkeOut
Disallow: /

User-Agent: Talkro
Disallow: /

User-Agent: TALWinHttpClient
Disallow: /

User-Agent: tamu
Disallow: /

User-Agent: Tandem
Disallow: /

User-Agent: Tarantula
Disallow: /

User-Agent: tarspider
Disallow: /

User-Agent: tBot
Disallow: /

User-Agent: TCF
Disallow: /

User-Agent: Tcs\/1
Disallow: /

User-Agent: TeamSoft
Disallow: /

User-Agent: Tecomi
Disallow: /

User-Agent: Teleport
Disallow: /

User-Agent: Teleport\ Pro
Disallow: /

User-Agent: TeleportPro
Disallow: /

User-Agent: Telesoft
Disallow: /

User-Agent: Templeton
Disallow: /

User-Agent: Tencent
Disallow: /

User-Agent: Terrawiz
Disallow: /

User-Agent: Test
Disallow: /

User-Agent: TexNut
Disallow: /

User-Agent: The Intraformant
Disallow: /

User-Agent: The.Intraformant
Disallow: /

User-Agent: TheNomad
Disallow: /

User-Agent: Thomas
Disallow: /

User-Agent: TightTwatBot
Disallow: /

User-Agent: Timely
Disallow: /

User-Agent: Titan
Disallow: /

User-Agent: TMCrawler
Disallow: /

User-Agent: TMhtload
Disallow: /

User-Agent: Toata\ dragostea\ mea\ pentru\ diavola
Disallow: /

User-Agent: toCrawl
Disallow: /

User-Agent: toCrawl/UrlDispatcher
Disallow: /

User-Agent: Todobr
Disallow: /

User-Agent: Tongco
Disallow: /

User-Agent: topic
Disallow: /

User-Agent: Torrent
Disallow: /

User-Agent: Track
Disallow: /

User-Agent: translate
Disallow: /

User-Agent: Traveler
Disallow: /

User-Agent: TREEVIEW
Disallow: /

User-Agent: trivial
Disallow: /

User-Agent: TRUE
Disallow: /

User-Agent: True_Robot
Disallow: /

User-Agent: True_Robot/1.0
Disallow: /

User-Agent: Tunnel
Disallow: /

User-Agent: turing
Disallow: /

User-Agent: turingos
Disallow: /

User-Agent: Turnitin
Disallow: /

User-Agent: TutorGig
Disallow: /

User-Agent: TV33\_Mercator
Disallow: /

User-Agent: Twat
Disallow: /

User-Agent: Tweak
Disallow: /

User-Agent: Twice
Disallow: /

User-Agent: Twisted.PageGetter
Disallow: /

User-Agent: Tygo
Disallow: /

User-Agent: ubee
Disallow: /

User-Agent: UCmore
Disallow: /

User-Agent: UdmSearch
Disallow: /

User-Agent: UIowaCrawler
Disallow: /

User-Agent: Ultraseek
Disallow: /

User-Agent: UMBC
Disallow: /

User-Agent: unf
Disallow: /

User-Agent: UniversalFeedParser
Disallow: /

User-Agent: unknown
Disallow: /

User-Agent: UPG1
Disallow: /

User-Agent: URI::Fetch
Disallow: /

User-Agent: URL.Control
Disallow: /

User-Agent: URL\_Spider\_Pro
Disallow: /

User-Agent: URLBase
Disallow: /

User-Agent: urldispatcher
Disallow: /

User-Agent: URLGetFile
Disallow: /

User-Agent: urllib
Disallow: /

User-Agent: URLSpiderPro
Disallow: /

User-Agent: URLy
Disallow: /

User-Agent: URLy Warning
Disallow: /

User-Agent: User\-Agent
Disallow: /

User-Agent: UserAgent
Disallow: /

User-Agent: User-Agent
Disallow: /

User-Agent: USyd
Disallow: /

User-Agent: UtilMind
Disallow: /

User-Agent: Vacuum
Disallow: /

User-Agent: vagabo
Disallow: /

User-Agent: Valet
Disallow: /

User-Agent: Valid
Disallow: /

User-Agent: Vamp
Disallow: /

User-Agent: vayala
Disallow: /

User-Agent: VB\_
Disallow: /

User-Agent: VCI
Disallow: /

User-Agent: VCI WebViewer VCI WebViewer Win32
Disallow: /

User-Agent: VERI\~LI
Disallow: /

User-Agent: verif
Disallow: /

User-Agent: versus
Disallow: /

User-Agent: via
Disallow: /

User-Agent: Viewer
Disallow: /

User-Agent: virtual
Disallow: /

User-Agent: visibilitygap
Disallow: /

User-Agent: Visual
Disallow: /

User-Agent: vobsub
Disallow: /

User-Agent: Void
Disallow: /

User-Agent: VoidEYE
Disallow: /

User-Agent: VoilaBot
Disallow: /

User-Agent: voyager
Disallow: /

User-Agent: vspider
Disallow: /

User-Agent: VSyn
Disallow: /

User-Agent: w\:PACBHO60
Disallow: /

User-Agent: w0000t
Disallow: /

User-Agent: W3C
Disallow: /

User-Agent: w3m
Disallow: /

User-Agent: w3search
Disallow: /

User-Agent: walhello
Disallow: /

User-Agent: Walker
Disallow: /

User-Agent: Wand
Disallow: /

User-Agent: WAOL
Disallow: /

User-Agent: WAPT
Disallow: /

User-Agent: Watch
Disallow: /

User-Agent: Wavefire
Disallow: /

User-Agent: wbdbot
Disallow: /

User-Agent: Weather
Disallow: /

User-Agent: Web Image Collector
Disallow: /

User-Agent: web.by.mail
Disallow: /

User-Agent: Web.Data.Extractor
Disallow: /

User-Agent: Web.Downloader
Disallow: /

User-Agent: Web.Ima
Disallow: /

User-Agent: Web.Mole
Disallow: /

User-Agent: Web.Sucker
Disallow: /

User-Agent: Web\ Image\ Collector
Disallow: /

User-Agent: Web\ Sucker
Disallow: /

User-Agent: Web2Mal
Disallow: /

User-Agent: Web2WAP
Disallow: /

User-Agent: webalta
Disallow: /

User-Agent: WebaltBot
Disallow: /

User-Agent: WebAuto
Disallow: /

User-Agent: WebBandit
Disallow: /

User-Agent: WebBandit/3.50
Disallow: /

User-Agent: Webbot
Disallow: /

User-Agent: WebCapture
Disallow: /

User-Agent: WebCat
Disallow: /

User-Agent: Webclip
Disallow: /

User-Agent: WebCollage
Disallow: /

User-Agent: WebCollector
Disallow: /

User-Agent: WebCopier
Disallow: /

User-Agent: WebCopy
Disallow: /

User-Agent: WebCor
Disallow: /

User-Agent: webcraft\@bea
Disallow: /

User-Agent: webcrawl
Disallow: /

User-Agent: WebDat
Disallow: /

User-Agent: WebDav
Disallow: /

User-Agent: webdevil
Disallow: /

User-Agent: webdownloader
Disallow: /

User-Agent: Webdup
Disallow: /

User-Agent: WebEMail
Disallow: /

User-Agent: WebEMailExtrac
Disallow: /

User-Agent: WebEnhancer
Disallow: /

User-Agent: WebFetch
Disallow: /

User-Agent: WebFilter
Disallow: /

User-Agent: WebFountain
Disallow: /

User-Agent: WebGo
Disallow: /

User-Agent: WebGo\ IS
Disallow: /

User-Agent: WebHook
Disallow: /

User-Agent: Webinator
Disallow: /

User-Agent: WebInd
Disallow: /

User-Agent: webitpr
Disallow: /

User-Agent: WebLea
Disallow: /

User-Agent: WebLeacher
Disallow: /

User-Agent: Webmaster
Disallow: /

User-Agent: WebmasterWorldForumBot
Disallow: /

User-Agent: WebMin
Disallow: /

User-Agent: WebMirror
Disallow: /

User-Agent: webmole
Disallow: /

User-Agent: webpic
Disallow: /

User-Agent: WebPin
Disallow: /

User-Agent: WebPix
Disallow: /

User-Agent: WebReaper
Disallow: /

User-Agent: WebRipper
Disallow: /

User-Agent: WebRobot
Disallow: /

User-Agent: WebSauger
Disallow: /

User-Agent: WebSite
Disallow: /

User-Agent: Website Quester
Disallow: /

User-Agent: Website.eXtractor
Disallow: /

User-Agent: Website.Quester
Disallow: /

User-Agent: Website\ eXtractor
Disallow: /

User-Agent: Website\ Quester
Disallow: /

User-Agent: WebSnake
Disallow: /

User-Agent: webspider
Disallow: /

User-Agent: Webster
Disallow: /

User-Agent: Webster Pro
Disallow: /

User-Agent: WebStripper
Disallow: /

User-Agent: websucker
Disallow: /

User-Agent: WebTre
Disallow: /

User-Agent: WebVac
Disallow: /

User-Agent: webwalk
Disallow: /

User-Agent: WebWasher
Disallow: /

User-Agent: WebWeasel
Disallow: /

User-Agent: WebWhacker
Disallow: /

User-Agent: WebZIP
Disallow: /

User-Agent: WebZip/4.0
Disallow: /

User-Agent: Wells
Disallow: /

User-Agent: Wells\ Search\ II
Disallow: /

User-Agent: WEP.Search.00
Disallow: /

User-Agent: WEP\ Search
Disallow: /

User-Agent: WEP\_S
Disallow: /

User-Agent: WeRelateBot
Disallow: /

User-Agent: Wget
Disallow: /

User-Agent: Wget/1.5.3
Disallow: /

User-Agent: Wget/1.6
Disallow: /

User-Agent: Whack
Disallow: /

User-Agent: Whacker
Disallow: /

User-Agent: whiz
Disallow: /

User-Agent: WhosTalking
Disallow: /

User-Agent: Widow
Disallow: /

User-Agent: Wildsoft.Surfer
Disallow: /

User-Agent: Win67
Disallow: /

User-Agent: window.location
Disallow: /

User-Agent: Windows.95\)
Disallow: /

User-Agent: Windows.95\;
Disallow: /

User-Agent: Windows.98\)
Disallow: /

User-Agent: Windows.98\;
Disallow: /

User-Agent: WinHT
Disallow: /

User-Agent: winhttp
Disallow: /

User-Agent: WinHttpRequest
Disallow: /

User-Agent: WinHTTrack
Disallow: /

User-Agent: Winnie.Poh
Disallow: /

User-Agent: Winodws
Disallow: /

User-Agent: wire
Disallow: /

User-Agent: WISEbot
Disallow: /

User-Agent: wisenutbot
Disallow: /

User-Agent: wish
Disallow: /

User-Agent: Wizz
Disallow: /

User-Agent: WordP
Disallow: /

User-Agent: Works
Disallow: /

User-Agent: world
Disallow: /

User-Agent: WUMPUS
Disallow: /

User-Agent: Wweb
Disallow: /

User-Agent: WWW.Mechanize
Disallow: /

User-Agent: www.ranks.nl
Disallow: /

User-Agent: WWW\-Collector
Disallow: /

User-Agent: WWWC
Disallow: /

User-Agent: WWW-Collector-E
Disallow: /

User-Agent: WWW-Mechanize
Disallow: /

User-Agent: WWWOFFLE
Disallow: /

User-Agent: wwwster
Disallow: /

User-Agent: x\-Tractor
Disallow: /

User-Agent: X12R1
Disallow: /

User-Agent: Xaldon
Disallow: /

User-Agent: Xaldon\ WebSpider
Disallow: /

User-Agent: Xenu
Disallow: /

User-Agent: Xenu's
Disallow: /

User-Agent: Xenu's Link Sleuth 1.1c
Disallow: /

User-Agent: XGET
Disallow: /

User-Agent: xirq
Disallow: /

User-Agent: Y\!OASIS
Disallow: /

User-Agent: Y\!Tunnel
Disallow: /

User-Agent: yacy
Disallow: /

User-Agent: YaDirectBot
Disallow: /

User-Agent: Yahoo\-MMAudVid
Disallow: /

User-Agent: YahooSeeker
Disallow: /

User-Agent: YahooYSMcm
Disallow: /

User-Agent: Yamm
Disallow: /

User-Agent: Yand
Disallow: /

User-Agent: Yandex
Disallow: /

User-Agent: yandeximages
Disallow: /

User-Agent: yang
Disallow: /

User-Agent: Yeti
Disallow: /

User-Agent: Yoono
Disallow: /

User-Agent: yori
Disallow: /

User-Agent: Yotta
Disallow: /

User-Agent: YTunnel
Disallow: /

User-Agent: Zade
Disallow: /

User-Agent: zagre
Disallow: /

User-Agent: ZBot
Disallow: /

User-Agent: Zeal
Disallow: /

User-Agent: ZeBot
Disallow: /

User-Agent: zermelo
Disallow: /

User-Agent: zerx
Disallow: /

User-Agent: Zeus
Disallow: /

User-Agent: Zeus\.*Webster
Disallow: /

User-Agent: ZIPCode
Disallow: /

User-Agent: Zixy
Disallow: /

User-Agent: zmao
Disallow: /

User-Agent: ZyBorg
Disallow: /