Press Space to continue
Building a web scraper for $1 per month instead of $249
Press Space to continue
✨ Okay with lots of help from @javilopen and his Spanish scraping friends I've managed to vibe code my own @Scrapingbee alternative and replace my $249/mo Scrapingbee plan with my own $1/mo scraper running on my VPS! First I tried OpenSERP, an open source project that seems freshly maintained but it didn't work on my VPS IP or laptop IP and got blocked fast Then I tried to use residential IP proxies from IPRoyal but they were already quite tainted IPs meaning Google already knew they were…
@levelsio@levelsioSep 4
I've tried for weeks now to build my own @ScrapingBee and repeatedly AI has now told me that it can't do it 80% of my scraping is Google SERP results for Hotelist and even with residential IP proxies and solve CAPTCHA services and OpenSERP scraper it won't do it well for thousands of queries, it just gets blocked So I will keep paying them $99/mo A good test to see what services might remain now AGI is here My only remaining SaaS now are: - Cloudflare Domains - Cloudflare Email Sending - Cloudflare R2 Storage (for Photo AI mostly and Litestream DB backups) - Backblaze B2 for backups - Hetzner for all my servers for all my sites (VPS) - xAI for all LLMs for all my sites (so many things like moderation, writing emails, anything) - Scrapingbee
View