robogen

Robots.txt generator done right.

Takes an array of objects that describe rules. Converts them to an appropriate robots.txt.

Usage

Generate a robots.txt:

import robogen from 'robogen'
 
const robotsTXT = robogen([{
    userAgent: '*',
    allow: '/',
    disallow: ['/secret', '/private']
}, {
    userAgent: ['Bingbot', 'MSNBot'],
    disallow: '/'
}])
 
console.log(robotsTXT)

This will output

User-agent: *
Allow: /
Disallow: /secret
Disallow: /private
User-agent: Bingbot
User-agent: MSNBot
Disallow: /

Install

With npm installed, run

$ npm install robogen

Acknowledgments

robogen was inspired by robots-generator

See Also

• noffle/common-readme

License

MIT