AI that uses image context to translate and redraw manga

Hacker News

Share card

AI that uses image context to translate and redraw manga

Hi HN, As a huge manga fan and a translator by training, I've always been in awe of the incredible amount of work that fan translators put in. The process isn't just about translation; it involves manually cleaning the original text from the bubbles, redrawing the background art (inpainting), and then carefully placing the translated text back in (typesetting). It's a true labor of love, but it's incredibly time-consuming. I wondered if modern AI could help automate the most tedious parts of this workflow. So, I built AIFandom. It's a web tool where you upload your manga pages, and it extracts the text from the speech bubbles and translates the text, but with a key difference: The model doesn't just read the text, it also looks at the visual context of the panel to better capture the original tone and nuance. Then, It automatically removes the original text, uses an inpainting model to intelligently redraw the artwork that was behind the text, and then typesets the translated text back into the bubbles. The end result is a fully translated and readable manga page that you can download. I use gemini for the translation process and various models for OCR, inpainting, masking, rendering. It's not a free tool. This whole process is computationally expensive and requires a good amount of GPU time. To cover these costs, I've priced it at $7/month for 1000 pages. However, I really want you to see the quality for yourself, so there is a 50-page free trial for everyone to test it out, no credit card required upfront. I would be incredibly grateful for your feedback. - How is the translation quality and tone? - Does the automatic typesetting look natural to you? - Is the pricing fair for the value it provides? - What features should I prioritize adding next? Thanks for checking it out! I'll be here all day to answer any questions.