StringZilla v3 with C++, Rust, and Swift bindings, and AVX-512 and NEON

Hacker News

Share card

StringZilla v3 with C++, Rust, and Swift bindings, and AVX-512 and NEON

Hi HN! I've spent the last few weeks optimizing the string operations in StringZilla. You'd be surprised how much performance GLibc and the Standard Templates Library of C++ are leaving on the table, especially on Arm and, likely, on RISC-V. I've significantly increased the number of implemented algorithms for different workloads. Searching across English texts on Arm, for example: - is 4.4x faster than STL, - and 16.8x faster in reverse order. The new version of the library now includes a C++ binding largely compatible with the C++ Standard, featuring all 17 overloads of the `replace` function. It also gained bindings for Rust and Swift. On the functionality side, it can now compute Levenshtein distances and Needleman-Wunsch alignment scores using AVX-512. It can generate random strings with a fair, uniform distribution without integer divisions. It can sort strings and compute rolling hashes. It has lazily evaluated ranges to avoid memory allocations on bulk search and split operations over large datasets, which is very common in modern AI pre-training experiments. I wrote over 60KB of English text in the README about all the new bells and whistles, so repeating the whole text here probably doesn't make sense. I've also listed several "good first issues" on GitHub in case someone wants to contribute - it may be a good learning opportunity. Lastly, I'll organize more hackathons for systems engineers in the SF Bay Area. LLM wrappers are fun, but the world needs infra. So, if you are around, join and let's build the future together :)