maxtheweb/AlpenQueue
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5 Commits 1 Branch 0 Tags 62 KiB
Go 88.9%
Dockerfile 11.1%
1ce45cfe97
Go to file
Soldier 1ce45cfe97 Add URL scraping with ethical web crawling 
Replace sleep with actual URL fetching. Worker scrapes HTML title from URLs, respects robots.txt, and includes proper User-Agent headers. Scraped titles stored in SQLite and sent via webhook callback.
2025-11-16 08:18:31 +00:00
cmd/alpenqueue Add URL scraping with ethical web crawling 2025-11-16 08:18:31 +00:00
pkg Add URL scraping with ethical web crawling 2025-11-16 08:18:31 +00:00
.gitignore Add SQLite persistence and worker 2025-11-16 07:50:59 +00:00
go.mod Add URL scraping with ethical web crawling 2025-11-16 08:18:31 +00:00
go.sum Add URL scraping with ethical web crawling 2025-11-16 08:18:31 +00:00
README.md Initial commit 2025-11-16 07:30:00 +00:00

README.md

AlpenQueue: A lightweigt, self-hosted task queue in Go