Direct AI: Image classification without training data

Hacker News

Share card

Direct AI: Image classification without training data

I'm posting the following on behalf of my son Isaac. His incubator is Neo rather than YCombinator and he hasn't cultivated enough HN karma to post an announcement of his new company, so I'm doing it! He and his partner Ben have demos showing the use of their technology to classify images using only a word (or single example image) to tell the classifier what the you want to draw a box around in a live stream. No training or training data needed. https://directai.io ------------------------------------------------------------------------------------- Hi HN! For the past year we’ve been working to get rid of the last barrier to entry for building custom computer vision algorithms: training data. Today, we’re showing the world our progress! Three years ago I joined an autonomous robotics company as an ML engineer. I was shocked to discover that at most companies the hardest part of ML isn’t designing neural networks; turns out most people just use the open source model that scores the highest on relevant benchmarks. The hardest part was gathering enough data to train that network to work for our use case! I’m really passionate about the math of ML, so I looked into using the knowledge stored in foundation models to directly construct a model with the behavior I had in mind instead of gathering data and training one from scratch. CLIP (the model powering Stable Diffusion and DALL-E) has a lot of knowledge about the world and so people sometimes use it to build zero shot image classifiers with just text. But zero shot methods don’t give you much control over the classifier, so they didn’t solve my use case. I took a shot at building a method that could take a list of positive and negative examples and use CLIP to build a classifier that satisfied those constraints, and DirectAI was born! Looking forward to hearing your thoughts/feedback and growing with community :) thank you! Isaac