Outerthought is now NGData

Three months after the highly anticipated proof of architecture release, we're living up to our own promises, and are releasing Lily 'CR' 0.2 today - a fully-distributed, highly scalable and highly available content repository, marrying best-of-breed database and search technology into a powerful, productive and easy-to-use solution for contemporary internet-scale content applications.

For whom

You're building content applications (content management, archiving, asset management, DMS, WCMS, portals, ...) that scale well, either as a product, a project or in the cloud. You need a trustworthy underlying content repository that provides a flexible and easy-to-use content model you can adapt to your own requirements. You have a keen interest in non-relational/HBase technology but need a higher-level API, and scalable indexing and search as well.

Foundations

Lily builds further upon Apache HBase and Apache SOLR. HBase is a faithful implementation of the Google BigTable database, and provides infinite elastic scaling and high-performance access to huge amounts of data. SOLR is the server version of Lucene, the industry-standard search library. Lily joins HBase and SOLR in a single, solidly packaged content repository product, with automated sharding (making use of multiple hardware nodes to provide scaling of volume and performance), high availability and automated index maintenance. Lily adds a sophisticated, yet flexible and surprisingly practical content schema on top of this, providing the structuredness of more classic databases, versioning, secondary indexing, queuing: all the stuff developers care for when fixing real-world problems. All this is made accessible through a powerful Java/AVRO API and a platform-neutral REST interface.

Key features of this release

  • Fully distributed: Lily has a fully-distributed architecture making maximum use of all available hardware for scalability and availability. ZooKeeper is used for distributed process coordination, configuration and locking. Index maintenance is based on an HBase-backed RowLog mechanism allowing fast but reliable updating of SOLR indexes.
  • Index maintenance: Lily offers all the features and functionality of SOLR, but makes index maintenance a breeze, both for interactive as-you-go updating and MapReduce-based full index rebuilds
  • Multi-indexers: for high-load situations, multiple indexers can work in parallel and talk to a sharded SOLR setup
  • REST interface: a flexible and platform-neutral access method for all Lily operations using HTTP and JSON
  • Improved content model: we added URI as a base Lily type as a (small) indication of our interest in semantic technology

More importantly, we commit ourselves to API and data format layout compatibility from this release onwards. Lily offers the final API we want to support in the final release. Lily 0.2 is our contract for content application developers, upgrading to Lily final should require them to do as little code changes as possible.

From where

Download Lily from www.lilyproject.org. Open source, Apache license, no strings attached.

Enterprise support

Together with this release, we're rolling out our commercial support services (and signed up a first customer, yay!) that allows you to use Lily with peace of mind. Also, this release has been fully tested and depends on the latest Cloudera Distribution for Hadoop (CDH3 beta3) - Cloudera being the leading Hadoop enterprise services provider. We're proud to collaborate with a true market leader here.

Next up

Lily 1.0 is planned for March 2011, with an interim release candidate in January. We'll be working on performance enhancements, feature additions, and are happily - eagerly - awaiting your feedback and comments. We'll post a roadmap for Lily 0.3 and onwards by mid November.

Follow us

If you want to keep track of Lily's on-going development, join the Lily discussion list or follow our company Twitter @outerthought.

Thank you

I'd like to thank Bruno and Evert for their hard work so far, the Flemish IWT government fund for their partial financial support, and all of our early Lily adopters and enthusiasts for their much valued feedback. You guys rock!

categories: lily release nosql news
by Steven Noels on 10/29/10
blog comments powered by Disqus