❤
npm
Sign UpSign In

Search results

70 packages found

Sort by: Default
  • Default
  • Most downloaded this week
  • Most downloaded this month
  • Most dependents
  • Recently published

robots-parser

A specification compliant robots.txt parser with wildcard (*) matching support.

samclarke
published version 3.0.1, 2 years ago76 dependents licensed under $MIT
4,682,410

generate-robotstxt

Awesome generator robots.txt

evilebottnawi
published version 8.0.3, 5 years ago15 dependents licensed under $MIT
438,239

sitemapper

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

seantomburke
published version 3.2.20, 3 months ago62 dependents licensed under $MIT
125,423

express-robots-txt

Express middleware to serve and generate robots.txt

modosc
published version 1.0.0, 4 years ago17 dependents licensed under $MIT
97,000

robotstxt-webpack-plugin

A webpack plugin to output a robots.txt file

evilebottnawi
published version 8.0.0, 3 years ago4 dependents licensed under $MIT
65,456

astro-robots-txt

Generate a robots.txt for Astro

alextim
published version 1.0.0, 2 years ago3 dependents licensed under $MIT
48,483

robot-directives

Parse robot directives within HTML meta and/or HTTP headers.

stevenvachon
published version 0.4.0, 8 years ago5 dependents licensed under $MIT
68,274

super-sitemap

SvelteKit sitemap focused on ease of use and making it impossible to forget to add your paths.

jasongitmail
published version 1.0.3, 5 months ago0 dependents licensed under $MIT
19,150

astro-robots

A reliable robots.txt generator for Astro projects, offering zero-config setup and Verified Bots support.

jakichen
published version 2.3.1, 3 months ago0 dependents licensed under $ISC
12,638

robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

cakroyd
published version 2.0.3, 3 years ago5 dependents licensed under $MIT
8,122

express-robots

Express middleware for generating your robots.txt

ashaffer88
published version 0.1.6, 9 years ago1 dependents licensed under $MIT
4,139

vite-plugin-robots

Vite plugin for generating robots.txt for production and development builds

kolirt
published version 1.0.5, a year ago0 dependents licensed under $MIT
4,180

@mixmaxhq/express-robots

Express middleware for generating your robots.txt

mixmax-codeship
published version 1.0.0, 5 years ago1 dependents licensed under $MIT
1,466

@bizhours/next-seo-plugin

A Next.js plugin for managing SEO, meta tags, redirects, and robots.txt

cdawg8830
published version 1.0.31, 5 days ago0 dependents licensed under $MIT
1,090

express-seo-robots

Simple robots.txt file generation for express

john-doherty
published version 1.0.2, 3 years ago0 dependents licensed under $MIT
1,183

html-snapshots

A selector-based html snapshot tool using Puppeteer or PhantomJS that sources sitemap.xml, robots.txt, or arbitrary input

localnerve
published version 4.18.0, 19 days ago1 dependents licensed under $MIT
585

vuepress-plugin-robots

Robots.txt file generator plugin for vuepress

hiyue
published version 1.0.1, 6 years ago1 dependents licensed under $MIT
746

@api.global/typedserver

A TypeScript-based project for easy serving of static files with support for live reloading, compression, and typed requests.

lossless
published version 3.0.70, 10 days ago6 dependents licensed under $MIT
755

nextjs-seo-manager

## About

ngwessels
published version 2.0.0-rc.31, 24 days ago0 dependents licensed under $SEE LICENSE IN LICENSE
716

robots

Parser for robots.txt

ekalinin
published version 0.10.1, 8 years ago18 dependents
659