easy-crawl
Few lines describing your project.
📝 Table of Contents
🧐 About
Write about 1-2 paragraphs describing the purpose of your project.
🏁 Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
启动环境变量配置
// 运行模式 默认 true:无头模式, false:有头模式
const headless = (process.env.HEADLESS === 'true')
// 指定chrome执行path 默认 空:自动查询
const chromium_exec_path = process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH || ''
// 指定chrome走代理模式运行 默认 空:不使用代理
const proxyServer = process.env.PROXYSERVER || ''
// 指定定时调度器命令 默认 空:不使用定时调度器
const cron_identity = process.env.CRON || ''
// 启动调试模式 默认 false:使用调试模式
const debug = (process.env.DEBUG === 'true')
// 启动MongoDB数据库初始化实例和释放,还有索引 默认 true: 开启
const save = (process.env.SAVE === 'true')
// 禁用所有CSS样式和js文件的加载,用于节约流量
const disable_css = (process.env.DISABLE_CSS === 'true')
// 禁用所有图片加载
const disable_img = (process.env.DISABLE_IMG === 'true')
// 启动浏览器devtools ,有些版本不支持,会导致错误
const enable_devtools = (process.env.DEVTOOLS === 'true')
// 延缓每个操作的时长
const slowMo = parseInt(process.env.SLOWMO) || 0
// 是否启用browserless
const browserless = (process.env.BROWSERLESS === "true")
// browless远程ws连接 示例: ws://127.0.0.1:7666
const browserlessuri = process.env.BROWSERLESSURI || ''
// browless远程ws连接token
const token = process.env.TOKEN || ''
// 最大失败执行次数,用于超出运行 afterExceedMaxFailureCount 回调事件
const maxFailureCount = parseInt(process.env.SLOWMO) || 50
Installing
A step by step series of examples that tell you how to get a development env running.
Say what the step will be
Give the example
And repeat
until finished
End with an example of getting some data out of the system or using it for a little demo.
🔧 Running the tests
Explain how to run the automated tests for this system.
Break down into end to end tests
Explain what these tests test and why
Give an example
And coding style tests
Explain what these tests test and why
Give an example
🎈 Usage
Add notes about how to use the system.
🚀 Deployment
Add additional notes about how to deploy this on a live system.
⛏️ Built Using
✍️ Authors
- @kylelobo - Idea & Initial work
See also the list of contributors who participated in this project.
🎉 Acknowledgements
- Hat tip to anyone whose code was used
- Inspiration
- References