I built an open-source vision RAG that embeds images and text together

Hacker News

My brother and I have been working on Morphik: an open-source multimodal database. (Github: https://github.com/morphik-org/morphik-core ) After experimenting with various AI models, we realized that they were particularly bad at answering questions which required retrieving over images and other multimodal data. That is, if I uploaded a 10-20 page PDF to ChatGPT, and ask it to get me a result from a particular diagram in the PDF, it would fail and hallucinate instead. I faced the same issue with Claude (not with Gemini, though). Turns out, the issue was with how these systems ingest documents. It seems like both Claude and GPT embed larger PDFs by parsing them into text, and then adding the entire thing to the context of the chat. While this works for text-heavy documents, it fails for queries/documents relating to diagrams, graphs, or infographics. Something that can help solve this is directly embedding the document as a list of images, and performing retrieval over that - getting the closest images to the query, and feeding the LLM exactly those images. This helps reduce the amount of tokens an LLM consumes while also increasing the visual reasoning ability of the model. We've implemented a one-line solution that does exactly this with Morphik. You can check out the specifics in the attached blog, or get started with it through our quick start guide: https://docs.morphik.ai/getting-started Here is an example ingestion pathway: ``` from databridge import DataBridge db = DataBridge() db.ingest_file("report_with_images_and_graphs.pdf", use_colpali=True) ``` And here is an example query pathway: ``` db.query("At what time-step did we see the highest GDP growth rate?", use_colpali=True) ``` Would love to hear your thoughts!

Visit product View on Hacker NewsMar 29, 2025

Share card

Actual performance

4points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: claude, model, models · Missing: mac, agents, macos

92%92% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: started, gemini · Missing: supports, reddit linkedin, podcasting

84%84% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: ide, io · Missing: https docs, excited, just released

59%59% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

42%42% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

31%31% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Strong signals: growth · Missing: arr, mrr, revenue

17%17% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Open-Source RAG with Sharepoint67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-Source RAG with Sharepoint

Hacker News3

Agentset24%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source RAG Platform

Indie Hackerscommitment-full-time

Open-source RAG App [video]54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source RAG App [video]

Hacker News1

Open source colorizing grayscale images with NN70%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open source colorizing grayscale images with NN

Hacker News219

Verba – An open-source RAG application63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Verba – An open-source RAG application

Hacker News7

Supavec75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The open source RAG as a service platform

Product Hunt+670Open Source

MAItion – Open-source RAG with pluggable connectors and chat UI67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

MAItion – Open-source RAG with pluggable connectors and chat UI

Hacker News1

Geniusrise – open-source inference endpoints for text, vision, audio70%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Geniusrise – open-source inference endpoints for text, vision, audio

Hacker News1

I Built an Open-Source RAG API for Docs, GitHub Issues and READMEs71%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I Built an Open-Source RAG API for Docs, GitHub Issues and READMEs

Hacker News2

I built an open source Favicon API67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I built an open source Favicon API

Hacker News1

Browse all products Analyze your launch