Every time I need to load a large chunk of data into memory instead of streaming it and processing chunks of it at a time my brain starts buzzing from the code smell.
Also whenever I use blocking IO on a single threaded codebase 😱😱
My poor RAM. It should be caching files instead of buffering!
@mauve the component that I've been debugging today is a streaming JSON parser. I was originally debugging something else but I got a test dataset in JSON and my attempts to use it to debug the original thing exposed a bunch of bugs in my JSON parser.
@jeremy_list Oh wow. What environment are you in that you had to make your own JSON parser? 🤯
@mauve I didn't have to write my own JSON parser: I had to write my own JSON serialiser and figured I'd do the parser as well. There actually are JSON serialisers for the environment in question as well but I needed one that would output exactly the same characters as the one in the V8 JavaScript engine and I didn't know which ones did that.
@jeremy_list Oh jeeze yeah. that's a major mood. i'm guessing you're trying to sign and verify json data? The SSB community had a bit of a reckoning with this a while ago :P Kinda why I like using IPLD's dag-json for stuff now.
@mauve yeah still trying to sign and validate SSB data. Which is a weird problem to be having since I already have it correctly hashing signed SSB data.
@mauve streaming isn't always more efficient 😇