robogen
Robots.txt generator done right.
Takes an array of objects that describe rules. Converts them to an appropriate robots.txt
.
Usage
Generate a robots.txt
:
const robotsTXT = console
This will output
User-agent: *
Allow: /
Disallow: /secret
Disallow: /private
User-agent: Bingbot
User-agent: MSNBot
Disallow: /
Install
With npm installed, run
$ npm install robogen
Acknowledgments
robogen was inspired by robots-generator