Due to my hubris I am in the process of rewriting my zip file parser from scratch. :P At least now I have experience using the particular APIs and the things I need to watch out for from using Blobs.

Follow

Though TBH my initial approach would have been better for a purely streaming mode where we process and chunk the WARC file on the fly. Now I need to be able to do random access to a WARC either on the FS or via Range queries over the network.

· · Web · 0 · 0 · 0
Sign in to participate in the conversation
Mauvestodon

Escape ship from centralized social media run by Mauve.