app_dir/urls.py
1 2 3 4 5 6 7 |
from django.views.generic import TemplateView urlpatterns = [ <span class="n">path</span><span class="p">(</span> <span class="s">"robots.txt"</span><span class="p">,</span> <span class="n">TemplateView</span><span class="p">.</span><span class="n">as_view</span><span class="p">(</span><span class="n">template_name</span><span class="o">=</span><span class="s">"robots.txt"</span><span class="p">,</span> <span class="n">content_type</span><span class="o">=</span><span class="s">"text/plain"</span><span class="p">),</span> <span class="p">),</span> |
記述内容(google以外のクローラーを拒否)
1 2 3 4 5 |
User-agent: * Disallow: / User-agent: Googlebot Allow: / |