AlpenQueue/go.mod
Soldier 1ce45cfe97 Add URL scraping with ethical web crawling
Replace sleep with actual URL fetching. Worker scrapes HTML title from URLs, respects robots.txt, and includes proper User-Agent headers. Scraped titles stored in SQLite and sent via webhook callback.
2025-11-16 08:18:31 +00:00

10 lines
178 B
Modula-2

module alpenqueue
go 1.25.4
require (
github.com/mattn/go-sqlite3 v1.14.32 // indirect
github.com/temoto/robotstxt v1.1.2 // indirect
golang.org/x/net v0.47.0 // indirect
)