Vi

VibeScrape – Paste a URL and JSON schema, get working web scraper code

Hacker News

VibeScrape – Paste a URL and JSON schema, get working web scraper code

Hey HN, I built VibeScrape — it takes a website URL and a JSON schema describing the data you want, then analyzes the page, writes real Python code to extract that data, and refines the code until the output is accurate. While there's lots of tools these days (e.g. Firecrawl) that feed the entire HTML of a webpage to an LLM to extract data from it, this always seemed like a really slow & expensive approach to me. On the other hand, handwriting web-scraping code seems archaic at this point. This type of code is incredibly tedious to write, and immediately becomes throwaway code once the webpage's layout changes even a little. VibeScrape aims to automate the process of writing this type of code. 1. Grabs the rendered HTML — the same view a browser sees. 2. Has an LLM extract data from the HTML into your target JSON schema (the “ground truth”). 3. Generates Python scraper code to reproduce that "ground truth" output. 4. Runs and compares results against the ground truth. 5. Refines the code automatically until the outputs match. I've found that letting the LLM take control of both the code generation and iteration process e2e has worked pretty well for producing working scraper code for many of the websites I've tested it on! It still has some limitations in terms of handling pagination, captchas, infinite scrolling, etc. Hoping to get some early feedback from the HN community to see if this is a valuable tool. There's a promo code FIRST5 on the site that gets you 5 credits for free, but am happy to give more credits to anyone that reaches out at contact@vibescrape.ai ! Thanks!

Share card

Actual performance

2points
Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →
Product HuntOn track for Day 1 leaderboard · Strong signals: code · Missing: mac, agents, macos
90%90% predicted probability of success on Product Hunt, based on ML models trained on real launch data.
best fitHighest predicted score across all platforms for this description.
Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting
81%81% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.
Hacker NewsStrong engagement from HN community · Strong signals: lua, io · Missing: https docs, excited, just released
52%52% predicted probability of success on Hacker News, based on ML models trained on real launch data.
nativeThis product was originally launched on this platform.
AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive
38%38% predicted probability of success on AppSumo, based on ML models trained on real launch data.
TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal
34%34% predicted probability of success on TrustMRR, based on ML models trained on real launch data.
Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue
13%13% predicted probability of success on Acquire.com, based on ML models trained on real launch data.
BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto
0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Je
Jev to JSON-Schema36%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Jev to JSON-Schema

Hacker News2
JS
JSON Schema precompiler36%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

JSON Schema precompiler

Hacker News2
FindCouponCode
FindCouponCode48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Paste a URL. Get working coupon codes in seconds.”

Indie Hackerscommitment-side-project
Ge
Generate Java beans from JSON Schema30%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Generate Java beans from JSON Schema

Hacker News1
Ge
Generate Graphql schema from json schema files50%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Generate Graphql schema from json schema files

Hacker News1
De
Declarative Angular Forms with JSON Schema44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Declarative Angular Forms with JSON Schema

Hacker News3
Schema Markup Generator (JSON-LD)
Schema Markup Generator (JSON-LD)23%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Effortlessly Create Schema Markup with JSON-LD Generator

Indie Hackersplatform-web
JS
JSON Schema Viewer32%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

JSON Schema Viewer

Hacker News2
Hu
Huma 2.0, it's like FastAPI for Go built on OpenAPI 3.1 and JSON Schema46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Huma 2.0, it's like FastAPI for Go built on OpenAPI 3.1 and JSON Schema

Hacker News1
Sc
Scraperjs – A versatile web scraper53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Scraperjs – A versatile web scraper

Hacker News192