Longlegs
Dusty Candland |
|
opensource, go, webcrawler
Longlegs was abstracted from [Proofreader](/proofreader/), [CloudSH](/cloudsh/) and [Route 285](/route285/). Each of those projects has some web crawling component and they all use Longlegs for the crawler.
Longlegs was abstracted from Proofreader, CloudSH and Route 285. Each of those projects has some web crawling component and they all use Longlegs for the crawler.
More deailts in the readme: Longlegs
A simple web spider that allows additional processing as the site is crawled.
Development
https://blog.golang.org/publishing-go-modules
Robots.txt
https://github.com/temoto/robotstxt
Deploy
https://go.dev/blog/publishing-go-modules
$ git commit -m "hello: changes for v0.1.0"
$ git tag v0.1.0
$ git push origin v0.1.0
Webmentions
These are webmentions via the IndieWeb and webmention.io. Mention this post from your site:
