Organisiert durch Jan Lehnardt (CouchDB), Simon Willnauer (Lucene Committer) und Isabel Drost (Co-Founder & Committer of Mahout) wird sich bei der Konferenz alles rund um die Themen NoSQL, Hadoop, Lucene und weiteres aus dem Bereich Scalability drehen. Die Namen der drei dürften übrigens jedem Bekannt sein, der die ein oder andere Mailingliste rund um Lucene abonniert hat.

Neuigkeiten gibt es via @hadoopberlin sowie auf der Veranstaltungswebsite – dort werden übrigens auch noch “Helping Hands” gesucht!

It is already two months since I was announcing the Nutch 1.0 release. Finally after a 1.0 release candidate Nutch was released in version 1.0 last week! Congratulations!

If you are also into Machine Learning you should check out the release of Mahout in its early version 0.3 and wether you are new on it, check out Grant Ingersoll’s presentation on Mahout from ApacheCon Europe 2009 here.

P.S.: Also notice the new Apache Droids Tab in the Lucene navigation bar!

• For the end-user, the interface is the system (Vinayak Hedge)
• “Perfect” is the enemy of “Good enough” (Greg Nyberg)
• Business Drives (Dave Muirhead)

You can order the book at Amazon or checkout the book’s website here.

Lucene community gathering in Amsterdam in March 2009

During the ApacheCon Europe 2009 there will take place a Lucene Meetup beside the venue. If you are interested in meeting Doug Cutting, Grant Ingersoll or Erik Hatcher you should check the Lucene wiki page LuceneMeetupMarch2009! I’m not sure yet wether I’m going to Amsterdam end of march!

Social Networks and Web 2.0 Papers at WWW 2009

You can find a summary of interesting papers regarding Social Networks in this post from Matthew Hurst’s Blog. Let me recommend Social Search in “Small World” Examples, Network Analysis of Collaboration Structure in Wikipedia and How Opinions are Received by Online Communities.

I already posted some kind of spoiler couple weeks ago according to a new Nutch release! Again a message from Andrzej Bialecki appeared at the nutch mailing list announcing the release of Nutch 1.0 maybe in february (this year).

Comment from Andrzej:

We do exist. We plan to release in February – I can’t tell you yet when exactly, we need to review the (few) remaining issues that we want to resolve before the release.

Update: I also heard about a Nutch/SOLR integration coming with this release?! Will start my investigation tommorrow.

Update II, A few important snippets from the CHANGES.txt:

• NUTCH-442 – Integrate Solr/Nutch. (dogacan, original version by siren)
• NUTCH-663 – Upgrade Nutch to use Hadoop 0.19 (kubes)
• NUTCH-662 – Upgrade Nutch to use Lucene 2.4. (kubes)
• NUTCH-665 – Search Load Testing Tool (kubes)
• NUTCH-646 -  New Indexing Framework for Nutch. (kubes)

If you ever wanted to search through the whole Lucene ecosystem (Lucene Java, Mahout, Tika and more) your possible solution of choice is the search engine by Lucid Imagination. It searches through project websites, wikis, mailing-lists (public, dev) and the source code itself – its indexiation performance is also pretty impressive.

Lucid Imagination is a company founded by Marc Krellenstein, Yonik Seeley, Grant Ingersoll and Erik Hatcher (a team of experts) which provides professional search (and beyond) solutions for the enterprise based upon Lucene.

Update: comment about Lucid Imagination from Beyond Search

The registration to ApacheCon Europe 2009 (23-27 March) in Amsterdam was opened last week. I already spent some time watching the available sessions regarding Hadoop, Lucene etc., have a look…

• Hadoop Tools and Tricks for Data Processing Pipelines
• Lucene Boot Camp
• Solr Boot Camp
• Introduction To Hadoop
• Introducing Mahout: Apache Machine Learning
• Hadoop Map-Reduce: Tuning and Debugging
• Lucene Case Studies
• Pig – Making Hadoop Easy
• Advanced Indexing Techniques with Apache Lucene
• Running Hadoop in the Cloud
• Configuring Hadoop for Grid Services
• Dynamic Hadoop Clusters
• Best of breed – httpd, forrest, solr and droids

As you can see, Apache Hadoop is playing a big role this year, and cloud computing starts to take off at Apache too. Unfortunately I didn’t find any Nutch related sessions – whats happening there?

Breakfast Links 12.01.2009

* PyLucene – Python Lucene – Joins Apache Lucene

PyLucene is a python based port of Lucene which is build automatically from the original Lucene source. It just joined Apache Lucene as a sub project. Does this means I can run Lucene in Google’s App Engine?

* Interview with Norbert Weitkämper

Norbert Weitkämper (Weitkämper Technology) interviewed in december 2008 by Stephen E. Arnold from Beyond Search. After the previous interview with Hans-Christian Brockmann, Beyond Search means also a lot of beyond Kentucky!

* Enterprise Search 2008 Wrap-Up

Lynda Moulton (The Gilbane Group) with a short summary on Enterprise Search in the year 2008. Including business developments at the big players (Microsoft and FAST, IBM and OmniFind), search/text mining technologies and a lot of interesting company names in the field of Enterprise Search and beyond.

* New Stuff in Lucene and Solr

Solr insider Grant Ingersoll about the development of new hot stuff for Apache Solr. I’m really excited about the SpatialSearch feature which provides some kind of geo search (there was also an introduction of local search by Chris Hostetter at ApacheCon 2008 here).

Breakfast Links 09.12.2008

* Yahoo BOSS Now Serving 100 Queries Per Second

While Yahoo! was recently struggling with lots of negative issues in terms of their business (who is next Yahoo! CEO?), they kept on publishing and propagating their technologies. According to the Yahoo Search Blog, BOSS is now serving 10 million queries per day! In my opinion Yahoo’s search technology is catching up fast and is far ahead of Google’s APIs.

* If you were ever interested in an UIMA Lucene Cas Consumer, check this out:

at the JULIE Lab, we’ve been working (silently) on a new (and completely
altered) version of our Lucene CAS Indexer consumer (Lucas). We are
planning to make this available soon — preferably in the UIMA sandbox.
In fact, LUCAS now is able to perform offset-based token stream
alignment and merging of UIMA annotations (via position increment) in
the same Lucene field (e.g. “documenttext” or “title”), which we feel is
more appropriate for text indexing — instead of putting each UIMA
annotation into a separate field like the Solr approach (still possible
with the new LUCAS).

(UIMA mailing list, Joachim Wermter)

I wasn’t able to write any Breakfast Links since monday due to projects (like the upcoming relaunch of this blog), birthday parties and efforts for christmas presents (big, big family)! Finally I turned on my news reader today…

* Amazon Launches Public Data Sets To Ease Research

Amazon (AWS) provides public data sets to demonstrate use cases for their (cloud) services. The article on TechCrunch also links to Swivel which is a repository for open data sets. Unfortunately you won’t find any textual data but a lot of data like U.S. Census, historical oil prices, world photovoltaic production and more.

* New Open Source Search Vendor

An interview of Hans-Christian Brockmann (Brox IT-Solutions GmbH) about Brox Open Source strategy in the field of enterprise search. Really interesting is Brockmann’s number of 180.000 lucene projects (more or less productive) out there! Brox already came up with their Eclipse project SMILA, which is going to be a base architecture to better integrate search and text mining into common systems. Nice to hear from those kind of projects in my business area!

* Z! – Episode 134: Qimaya – Anwendung neuronaler Netze (german)

Interesting interview with Roy Uhlmann (Qimaya aka former Queap) about neuronal networks in their search engine Qimaya. If you are interested in testing Qimaya you can request access to the demo – next invitation and access delivery is even today! As I can see, Qimaya is currently a german language only project!