Write JSON processor queries (jq) in natural language

Hacker News

Share card

Write JSON processor queries (jq) in natural language

Hey HN! Jaiqu is an AI agent for creating repeatable JSON transforms using jq query language syntax. Jaiqu translates complex JSON structures into repeatable jq queries. jq is a command line utility built for fast, reliable JSON queries. Jaiqu takes advantage of the LLMs for both constructing jq queries as well as using feedback to fix errors/invalid results if those queries are incorrect. Writing code to parse JSON is super annoying. It usually involves manually reading huge JSON balls, parsing the keys in a language of your choice, and pulling the data manually. Also, you're likely to run into key mismatches (i.e. source data might have "customer_id" as a key but my table is "user_id"), so you would have to write the translation dictionaries by hand. This struck me as a task that was analogous to the many AI "natural language to SQL" editors out there. However, there were a lot of consistency issues where the LLMs would not produce functioning code. So I wrote an AI agent that does it for me. It only takes two args: 1. A blob of messy JSON containing data you want 2. A JSON schema formatted in the way you want Jaiqu generates a repeatable jq query that collects the data in the exact JSON schema you specify. This way, you can run this query as many times you want or save it as a config without having to prompt LLMs each and every time. This tool beats the standard "prompt ChatGPT for a jq query" because by and large, LLMs don't perform too well when writing DSLs like jq. Jaiqu essentially works by: 1. Fuzzy matching each key against your schema to make sure it's available to query 2. Writing individual jq queries to fetch and transform those keys 3. Compiling each individual query into a single complex jq query. 4. Feeding back any errors or data mismatches to a "self-healing" loop that adjusts the queries accordingly until it works. Would love to hear thoughts or feedback. This has saved our team a ton of time internally, so hopefully it can help others as well.