Skwish / News: Recent posts

Skwish is a fast, simple, lightweight Java library for storing blobs on the file system. It allows multiple concurrent readers and writers, provides all-or-nothing write semantics, and is designed to survive abnormal, unclean shutdown. Skwish is a structured implementation of storing all blobs in a single file to save on file system I/O. Skwish is premised on the proposition that blob storage ought to be orthogonal to the task of indexing. It is meant to be a clean and simple store on which some other index can be built.... read more

This release now supports referencing an entry's id before it is committed. (Recall, in the general case, a newly inserted entry's id is fixed only after the transaction commits.) This is achieved by introducing a new transaction id, which in combination with a new entry's pre-commit id, uniquely determines the entry's post-commit id. More documentation about the new feature and transactions, in general, is included with the distribution. A demo program is also included that uses the new feature.

Skwish, a small, fast Java library for storing and retrieving blobs, is now licensed under the Apache License, Version 2.0.

See the project website for details.

-Babak Farhang
http://skwish.sourceforge.net/

The 4th release of skwish, a small java library for storing and retrieving blobs, introduces a new experimental non-blocking embedded HTTP server for serving the blobs. Besides providing a good public interface to the contents of a store, this new feature also provides a convenient way to inspect ad hoc blobs (entries, in skwish-terminology) using just a browser--e.g., to debug an application that uses skwish.... read more

Hi everyone,

The last few weeks I've been banging out the code for Skwish, a Java library for storing blobs on the file system. The skeleton has a little meat now, but no so much that it can't easily change direction. So now seemed liked a good time to release: there's enough code there to show it rather than tell it.

But let me tell you why I started this project. My primary interest is in search (indexing), both structured (e.g. rdf), unstructured (e.g. full text search) data. This involves processing and indexing a lot of source files, and you typically need to store these files *somewhere*. Usually that *somewhere* ends up being a deep, heavily populated directory structure on the file system. But this directory structure approach doesn't usually scale well, and it becomes especially annoying when the blobs (files) you're storing have no inherent hierarchical structure. Worse, when the blobs (files) are small and many, and access is heavy, all that file I/O begins to take a toll. ... read more