wintersmith-robots
A Wintersmith plugin to generate a Robots.txt file for sitewide and per-page control over indexing.
Install
npm install wintersmith-robots
Add wintersmith-robots
and wintersmith-contents
to your config.json
Use
Set sitewide options in Wintersmith's config.json
. If noindex
is set globally, your entire site will be blocked from crawlers.
Set per-page options at the top of your Markdown files. For instance, you can prevent an article from being indexed like so:
---noindex: true---